[fix](be) Avoid rowset map race in backend tablets scanner#65288
[fix](be) Avoid rowset map race in backend tablets scanner#65288zclllyybb wants to merge 1 commit into
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Root cause:\nThe backend tablets schema scanner reads tablet rowset maps while compaction can rewrite the same maps under the tablet header lock. Some columns also depend on the max-version rowset and rowset-derived compaction metadata, so reading them without the lock can race with compaction.\n\nFix:\nTake a short shared header lock per tablet, snapshot the rowset metadata pointers, row-binlog metadata pointers, max-version rowset pointer, and scalar tablet fields, then release the lock before aggregating sizes, counts, compaction score, overlap state, and filling the result block. This uses the same synchronization boundary as compaction while keeping the lock hold time small.\n\nVerification:\n- git diff --check\n- GIT_ALLOW_PROTOCOL=file ./build.sh --be -j 90
6a4f528 to
7b451b2
Compare
|
run buildall |
|
/review |
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
There was a problem hiding this comment.
Automated review summary for PR 65288.
I reviewed the single changed file, be/src/information_schema/schema_tablets_scanner.cpp, plus the surrounding tablet, rowset metadata, local tablet, cloud tablet, FE schema, and regression-test context. I did not find a substantiated issue that needs an inline review comment.
Critical checkpoint conclusions:
- Goal and tests: the PR snapshots backend-tablets scanner rowset metadata under the tablet header lock to avoid races with compaction. The implementation does that and preserves the prior
backend_tabletsoutput formulas. Existing coverage includes backend-tablets shape and cloud tablet size semantics; I did not run tests in this checkout. - Scope: the change is focused to the backend tablets scanner and does not introduce unrelated behavior.
- Concurrency and lifecycle: the relevant concurrency is schema scan reads versus compaction/publish/sync mutation of tablet rowset maps. The scanner now copies rowset meta shared pointers and the max-version rowset while holding the same header lock used by those mutations, then releases it before aggregation and block filling.
- Config, compatibility, and protocol: no new configs, persisted formats, FE-BE thrift fields, or schema columns are introduced.
- Parallel paths: local and cloud tablet paths remain covered. Cloud
IS_USEDstill returns true as before, and local mode still usesTablet::is_used(). - Data writes and persistence: this is a read-only information-schema scanner path; transaction and persistence write paths are not changed.
- Performance and observability: the per-tablet lock scope is limited to snapshot copying. Heavy aggregation and block writes happen after unlock. No new observability looked necessary for this race fix.
- Validation: static review only.
.worktree_initialized,thirdparty/installed, andthirdparty/installed/bin/protocare missing in this runner, so I did not run BE build or regression tests.git show --checkfor the changed file reported no whitespace errors.
User focus: no additional user-provided review focus was supplied.
Subagent conclusions:
optimizer-rewrite: no candidates became inline comments; convergence round 1 returnedNO_NEW_VALUABLE_FINDINGS.tests-session-config: no candidates became inline comments; convergence round 1 returnedNO_NEW_VALUABLE_FINDINGS.- No duplicates were merged, and the final proposed inline comment set was empty.
|
Branch-4.0 backport PR: #65301 |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
skip check_coverage |
What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
information_schema.backend_tabletsreads tablet rowset metadata while compaction can rewrite the rowset maps under the tablet header lock. The scanner also derives fields such as tablet size, version count, segment count, compaction score, update time, and overlap state from those rowset maps or the max-version rowset.This PR takes a short shared tablet header lock per tablet, snapshots the rowset metadata pointers, row-binlog metadata pointers, max-version rowset pointer, and scalar tablet fields, then releases the lock before doing aggregation and filling the result block. This keeps the scanner synchronized with compaction without holding the lock during block materialization.
Release note
None
Check List (For Author)
Test
git diff --checkGIT_ALLOW_PROTOCOL=file ./build.sh --be -j 90Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)