Skip to content

ddl: do not fill TiKV block cache for reorg coprocessor scans#69876

Open
sanketkedia wants to merge 1 commit into
pingcap:masterfrom
sanketkedia:ddl-reorg-scan-not-fill-cache
Open

ddl: do not fill TiKV block cache for reorg coprocessor scans#69876
sanketkedia wants to merge 1 commit into
pingcap:masterfrom
sanketkedia:ddl-reorg-scan-not-fill-cache

Conversation

@sanketkedia

@sanketkedia sanketkedia commented Jul 15, 2026

Copy link
Copy Markdown

What problem does this PR solve?

Issue Number: close #69875

Problem Summary:

Reorg coprocessor scans are single-pass bulk reads over the whole table/index
range. They are sent with NotFillCache = false, so every scanned block is
inserted into the TiKV block cache. Those blocks are never re-read by the reorg,
so caching them brings no benefit while risking eviction of the working set that
concurrent foreground traffic relies on — regressing foreground read
latency during an online ADD INDEX on a large table.

What changed and how does it work?

Set NotFillCache = true on the shared reorg DistSQLContext
(newDefaultReorgDistSQLCtx). All reorg coprocessor scans are built from this
context (via SetFromSessionVars), so they now skip filling the block cache.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Reduce TiKV block cache pollution during DDL reorg (e.g. `ADD INDEX`) by not filling the block cache for reorg coprocessor scans.

Reorg coprocessor scans (add index / add primary key backfill, and the
table-max-handle probe) are single-pass bulk reads over the whole
table/index range. The blocks they touch are never re-read by the reorg,
so inserting them into the TiKV block cache yields no benefit while
risking eviction of the working set that concurrent foreground traffic
relies on.

Set NotFillCache on the shared reorg DistSQLContext so all reorg
coprocessor scans skip filling the block cache. The table-max-handle
probe already set this flag directly; this makes the behavior consistent
across every reorg coprocessor scan.

Signed-off-by: Sanket Kedia <kediasanket11121993@gmail.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-tests-checked release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jul 15, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tiancaiamao for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. contribution This PR is from a community contributor. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels Jul 15, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

Hi @sanketkedia. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

An error occurred during the review process. Please try again later.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="Running error: context loading failed: failed to load packages: failed to load packages: failed to load with go/packages: context deadline exceeded"
level=error msg="Timeout exceeded: try increasing it by passing --timeout option"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@joechenrh

Copy link
Copy Markdown
Contributor

/ok-to-test

@ti-chi-bot ti-chi-bot Bot added ok-to-test Indicates a PR is ready to be tested. and removed needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels Jul 16, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 16, 2026

Copy link
Copy Markdown

@sanketkedia: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
idc-jenkins-ci-tidb/check_dev_2 ace2eed link true /test check-dev2
pull-integration-realcluster-test-next-gen ace2eed link true /test pull-integration-realcluster-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.6140%. Comparing base (531e40c) to head (ace2eed).

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #69876        +/-   ##
================================================
- Coverage   76.3207%   75.6140%   -0.7067%     
================================================
  Files          2041       2084        +43     
  Lines        559975     582365     +22390     
================================================
+ Hits         427377     440350     +12973     
- Misses       131697     139884      +8187     
- Partials        901       2131      +1230     
Flag Coverage Δ
integration 45.6804% <100.0000%> (+5.9751%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4471% <ø> (ø)
parser ∅ <ø> (∅)
br 63.9686% <ø> (+1.2473%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. ok-to-test Indicates a PR is ready to be tested. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ddl: skip filling the TiKV block cache for reorg coprocessor scans

2 participants