[fix](filecache) skip redundant ttl scans for non-TTL tablets#65429
Open
freemandealer wants to merge 1 commit into
Open
[fix](filecache) skip redundant ttl scans for non-TTL tablets#65429freemandealer wants to merge 1 commit into
freemandealer wants to merge 1 commit into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
Member
Author
|
/review |
Contributor
There was a problem hiding this comment.
Requesting changes for one file-cache TTL correctness issue.
Critical checkpoints:
- Changed files reviewed:
be/src/io/cache/block_file_cache_ttl_mgr.cppandbe/test/io/cache/block_file_cache_ttl_mgr_test.cpp, plus relevant cache load and cache-type transition paths. - Existing review context: no existing inline review threads or replies were present, so this is not a duplicate.
- User focus: no additional user-provided review focus.
- Repo instructions: no AGENTS.md files were listed; a repo-local code-review SKILL.md was not present in the expected paths, so I followed the review prompt and ledger workflow.
- Subagent conclusions:
optimizer-rewriteproduced no candidates and was dismissed with evidence;tests-session-configproducedmain-001, accepted as the inline comment below; convergence round 1 ended with both subagents replyingNO_NEW_VALUABLE_FINDINGSfor the same ledger/comment set. - Validation: static review only. The checkout is grafted and missing the base object for local diff checks, and
thirdparty/installedplusthirdparty/installed/bin/protocare absent, so I did not run BE unit tests.
35323a5 to
7ae7df1
Compare
Member
Author
|
/review |
Member
Author
|
run buildall |
Contributor
There was a problem hiding this comment.
Automated review result: request changes.
Critical checkpoints:
- Scope: live PR head 7ae7df1; GitHub reports two changed files: be/src/io/cache/block_file_cache_ttl_mgr.cpp and be/test/io/cache/block_file_cache_ttl_mgr_test.cpp.
- Existing review context: I considered the existing restart/persisted-TTL thread. The inline comment below is a distinct current-patch follow-up because the new periodic reconcile addresses the old _ttl_info_map gate but still depends on missing tablet-id registration.
- User focus: no additional user-provided review focus was supplied.
- Validation: git show --format= --check HEAD -- be/src/io/cache/block_file_cache_ttl_mgr.cpp be/test/io/cache/block_file_cache_ttl_mgr_test.cpp passed. I did not run the BE unit test locally because this grafted checkout is missing thirdparty/installed and thirdparty/installed/bin/protoc; the PR body reports the author's BlockFileCacheTtlMgrTest run.
Subagent conclusions:
- optimizer-rewrite proposed optimizer-rewrite-001, accepted as inline-001.
- tests-session-config proposed tests-session-config-001, merged as duplicate/supporting evidence for inline-001 rather than a second comment.
- Convergence round 1 ended with both live subagents reporting NO_NEW_VALUABLE_FINDINGS for the same final ledger/comment set.
Member
Author
|
run beut |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
The file cache TTL manager keeps registered tablet ids in
_tablet_id_set. For ordinary non-TTL tablets, the oldttl_seconds <= 0path always setneed_convert_from_ttl = true, so every update round scanned cached blocks even when the tablet had never been recorded in_ttl_info_map.This PR reduces the repeated CPU work for registered non-TTL tablets:
With the default
file_cache_background_ttl_info_update_interval_ms = 180000, ordinary non-TTL tablets avoid the repeated per-round block scan that previously ran every 3 minutes.Release note
None
Check List (For Author)
Unit test:
DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON ENABLE_INJECTION_POINT=ON ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh run-be-ut.sh --run --filter='BlockFileCacheTtlMgrTest.*'Result: 5 tests passed.
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)