Skip to content

Private/issue 2677 decision statistics pr desc clean#10948

Draft
ljluestc wants to merge 3 commits into
tikv:masterfrom
ljluestc:private/issue-2677-decision-statistics-pr-desc-clean
Draft

Private/issue 2677 decision statistics pr desc clean#10948
ljluestc wants to merge 3 commits into
tikv:masterfrom
ljluestc:private/issue-2677-decision-statistics-pr-desc-clean

Conversation

@ljluestc

Copy link
Copy Markdown

What problem does this PR solve?

Issue Number: Close #1125

When a TiKV store disconnects or remains unavailable for a long time, the
pd/api/v1/stores response may return last_heartbeat_ts as nil.

The root cause is that watcher/meta synchronization can overwrite in-memory
store metadata with an incoming LastHeartbeat that is missing (0) or older,
which loses the previously observed latest heartbeat timestamp.

What is changed and how does it work?

core: preserve last heartbeat when syncing store meta

When store metadata is refreshed from watcher updates, incoming meta may miss
LastHeartbeat or carry an older value. This could overwrite in-memory store
metadata and make last_heartbeat_ts disappear in pd/api/v1/stores for
long-disconnected stores.

Keep LastHeartbeat monotonic in SetStoreMeta:
- preserve existing heartbeat when incoming value is zero/missing
- update only when incoming heartbeat is newer

Add regression test TestSetStoreMetaPreservesLastHeartbeat.

Changes in this PR:

  1. Update core.SetStoreMeta to merge LastHeartbeat monotonically:
    • keep the current value by default;
    • only apply incoming LastHeartbeat when it is greater than the current one.
  2. Add TestSetStoreMetaPreservesLastHeartbeat in pkg/core/store_test.go to
    verify preserve/update/non-regression behavior.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

Release note

Fix an issue where `last_heartbeat_ts` in `pd/api/v1/stores` could be lost
after long store disconnections.

Non-default keyspace group TSO services now read all keyspace group
timestamps at startup and use the maximum value, ensuring TSO
monotonicity without requiring metadata cleanup during TSO node
start/stop operations.

Signed-off-by: Jiale Lin <63439129+ljluestc@users.noreply.github.com>
Use the effective TSO timeout when resetting the dispatcher stream loop timer on each batch, so timeout behavior no longer falls back to pd-server-timeout.

Add a dispatcher unit test that verifies stream-loop retries honor TSO timeout instead of the PD timeout, and align suite setup timeout defaults with the new semantics.

Remove local PR description artifact from tracked files.

Closes: tikv#9469

Signed-off-by: Jiale Lin <63439129+ljluestc@users.noreply.github.com>
@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Jun 28, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign bufferflies for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jun 28, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: beef5655-6b62-4b97-b4da-020e20d574ed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. dco-signoff: no Indicates the PR's author has not signed dco. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels Jun 28, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Hi @ljluestc. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot

ti-chi-bot Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.

📝 Please follow instructions in the contributing guide to update your commits with the DCO

Full details of the Developer Certificate of Origin can be found at developercertificate.org.

The list of commits missing DCO signoff:

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ti-chi-bot ti-chi-bot Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. dco-signoff: no Indicates the PR's author has not signed dco. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the LastHeartbeatTS may lost sometime

1 participant