Skip to content

Flash: add MPP sql digest and connection metadata tracing#10735

Open
JaySon-Huang wants to merge 2 commits intomasterfrom
feature/short-query-sql-digest
Open

Flash: add MPP sql digest and connection metadata tracing#10735
JaySon-Huang wants to merge 2 commits intomasterfrom
feature/short-query-sql-digest

Conversation

@JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Mar 8, 2026

What problem does this PR solve?

Issue Number: ref pingcap/tidb#66762

rely on pingcap/kvproto#1428

Problem Summary:

  • MPP task tracing logs miss SQL digest and connection metadata, making it harder to correlate TiDB session/query information during diagnosis.
  • Dispatch logs also do not include SQL digest, reducing observability on request ingress.

What is changed and how it works?

Flash: add MPP sql digest logging and update kvproto
Flash: include connection metadata in MPP task tracing logs
  • Plumb TaskMeta.sql_digest into DAGContext and expose it via getter.
  • Extend MPP tracing JSON (MPPTaskStatistics::logTracingJson) to print sql_digest, connection_id, and connection_alias.
  • Extend FlashService::DispatchMPPTask handling log with sql_digest.
  • Update contrib/kvproto submodule pointer to include matched proto changes.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced diagnostic capabilities with SQL digest tracking to enable improved query identification and monitoring across distributed tasks.
    • Added connection context information (connection ID and alias) to task logging for better request traceability and troubleshooting.
  • Chores

    • Updated dependency references.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 8, 2026
@pantheon-ai
Copy link

pantheon-ai bot commented Mar 8, 2026

⚠️ Review Failed

Environment preparation failed after 3 attempts due to upstream infrastructure overload (503 Service Unavailable). Please retry the review when the service recovers.

ℹ️ Learn more details on Pantheon AI.

@ti-chi-bot ti-chi-bot bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Mar 8, 2026
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 8, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jiaqizho for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai
Copy link

coderabbitai bot commented Mar 8, 2026

📝 Walkthrough

Walkthrough

This pull request propagates SQL digest and connection context information through the MPP task execution pipeline by introducing new member variables in DAGContext and MPPTaskStatistics to track sql_digest, connection_id, and connection_alias, alongside corresponding logging enhancements.

Changes

Cohort / File(s) Summary
DAGContext Extension
dbms/src/Flash/Coprocessor/DAGContext.h, dbms/src/Flash/Coprocessor/DAGContext.cpp
Added sql_digest member variable and getSQLDigest() accessor to DAGContext; initialized from meta_.sql_digest() in MPP constructor.
MPPTaskStatistics Context Tracking
dbms/src/Flash/Mpp/MPPTaskStatistics.h, dbms/src/Flash/Mpp/MPPTaskStatistics.cpp
Added three private members (connection_id, connection_alias, sql_digest) and initialized them from DAGContext in InitializeExecutorDAG; added same fields to JSON tracing log output.
Logging Integration
dbms/src/Flash/FlashService.cpp
Extended DispatchMPPTask log message to include sql_digest field and value.
Subproject Update
contrib/kvproto
Updated kvproto commit reference (no code changes).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 A rabbit hops through logs so grand,
With digests swift and context spanned,
Connection threads now traced with care,
Through MPP tasks floating in the air! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding MPP SQL digest and connection metadata tracing to Flash logs.
Description check ✅ Passed The pull request description follows the required template with all necessary sections completed, including problem statement, technical changes, commit messages, and checklist.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/short-query-sql-digest

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 8, 2026
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 8, 2026

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant