metrics: hide checkpoint lag for failed changefeeds

[ekexium]

What problem does this PR solve?

Issue Number: close #4053

A failed changefeed no longer advances its checkpoint, but the coordinator kept
publishing checkpoint lag based on the current PD time. As a result, the lag
continued to grow even though it no longer represented active replication
latency.

What is changed and how it works?

• Delete owner checkpoint timestamp and lag metrics when a changefeed is failed.
• Decouple checkpoint metric publication from the remaining per-changefeed
  metric synchronization, so failed changefeed error information is still
  reported.
• Extend the checkpoint metric test to cover failed, stopped, finished, and
  removed changefeeds.

Check List

Tests

• Unit test
  • go test ./coordinator -run '^TestUpdateChangefeedCheckpointMetricsDeletesInactiveLabels$' -count=1
  • go test ./pkg/metrics -count=1
  • go test ./coordinator -count=1
  • make unit_test_pkg PKG=./coordinator

Questions

Will it cause performance regression or break compatibility?

No.

Do you need to update user documentation, design documentation or monitoring documentation?

No. The existing dashboard query stops showing the series after Prometheus
marks the deleted metric stale.

Release note

Fix an issue where checkpoint lag continued to be reported for failed changefeeds.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lidezhu for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Warning

Review limit reached

@ekexium, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 47 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 70572f01-9b26-4e5d-8ac4-4a39c7e62450

📥 Commits

Reviewing files that changed from the base of the PR and between 369674c and e0a0f31.

📒 Files selected for processing (2)

• coordinator/controller.go
• coordinator/controller_test.go

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[pingcap-cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[gemini-code-assist]

Code Review

This pull request refactors the metrics collection logic in the coordinator controller. Specifically, updateChangefeedCheckpointMetrics has been updated to no longer return a boolean value, and it now handles the StateFailed state in addition to other inactive states (StateStopped, StateFinished, StateRemoved) by deleting the changefeed checkpoint metrics. The corresponding unit tests have been updated and expanded to cover all of these inactive states. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Copilot]

Pull request overview

This PR fixes misleading owner checkpoint lag reporting by deleting checkpoint timestamp/lag metrics when a changefeed enters a non-running terminal state (most importantly failed), preventing lag from continuously growing based on PD time when replication has stopped.

Changes:

• Delete changefeed checkpoint timestamp and lag metrics when the changefeed state is failed (in addition to already-handled stopped/finished/removed).
• Decouple checkpoint metric updates from error-metric synchronization so failed changefeed error information continues to be published.
• Expand the unit test to validate checkpoint metric deletion across inactive states (failed/stopped/finished/removed).

Validation:

• Not run in this review environment (PR description lists targeted go test and make unit_test_pkg runs).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
coordinator/controller.go	Deletes checkpoint metrics for failed changefeeds and ensures error-metric sync still runs.
coordinator/controller_test.go	Extends checkpoint-metric deletion test coverage for inactive changefeed states.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.