Skip to content

branch-4.1: [fix](filecache) fix flaky be UT for LRU dump (pick#65427)#65425

Open
freemandealer wants to merge 1 commit into
apache:branch-4.1from
freemandealer:task-branch-4-1-analyze-jira-doris-26721
Open

branch-4.1: [fix](filecache) fix flaky be UT for LRU dump (pick#65427)#65425
freemandealer wants to merge 1 commit into
apache:branch-4.1from
freemandealer:task-branch-4-1-analyze-jira-doris-26721

Conversation

@freemandealer

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: BlockFileCacheTest.version3_add_remove_restart relied on sleeping for the background LRU dump thread after replaying remove records. If the background dump did not run before the restart phase, stale LRU tail records could be restored and the restarted cache could report non-empty queues. The test now explicitly dumps LRU queues after replaying the remove logs, so persisted LRU state is synchronized before the restart check without changing file cache runtime code.

Release note

None

Check List (For Author)

  • Test:
    • Unit Test: DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON ENABLE_INJECTION_POINT=ON ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh run-be-ut.sh --run --filter=BlockFileCacheTest.version3_add_remove_restart
    • Style check: git diff --check -- be/test/io/cache/block_file_cache_test_meta_store.cpp
  • Behavior changed: No
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: BlockFileCacheTest.version3_add_remove_restart relied on sleeping for the background LRU dump thread after replaying remove records. If the background dump did not run before the restart phase, stale LRU tail records could be restored and the restarted cache could report non-empty queues. The test now explicitly dumps LRU queues after replaying the remove logs, so persisted LRU state is synchronized before the restart check without changing file cache runtime code.

### Release note

None

### Check List (For Author)

- Test:
    - Unit Test: DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON ENABLE_INJECTION_POINT=ON ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh run-be-ut.sh --run --filter=BlockFileCacheTest.version3_add_remove_restart
    - Style check: git diff --check -- be/test/io/cache/block_file_cache_test_meta_store.cpp
- Behavior changed: No
- Does this need documentation: No
@freemandealer freemandealer requested a review from yiguolei as a code owner July 9, 2026 12:27
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer

Copy link
Copy Markdown
Member Author

run buildall

@freemandealer

Copy link
Copy Markdown
Member Author

/review

@freemandealer freemandealer changed the title [fix](be) Make file cache meta store UT deterministic [fix](filecache) fix flaky be UT for LRU dump Jul 9, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review summary for PR 65425 at 7c0c876.

I found no blocking or inline-worthy issues in this patch.

Critical checkpoints:

  • Scope/head: verified the live PR still points to head 7c0c876 on base 7126cf6, with one changed file: be/test/io/cache/block_file_cache_test_meta_store.cpp.
  • Existing review context: no existing inline review comments or replies were present, so there were no duplicate issues to suppress.
  • Test behavior: the changed test now calls cache.dump_lru_queues(true) after replay_lru_logs_once() has drained the pending removal records and before the restart phase. The related implementation synchronously dumps the shadow LRU queues under _dump_lru_queues_mtx, so this matches the persisted restart state the test is validating and removes the previous timing dependency on the background dump interval.
  • Config/session compatibility: the modified test already saves and restores the affected file-cache config values through Defer, and this patch does not touch runtime configuration propagation or production behavior.
  • Validation: git diff --check FETCH_HEAD..HEAD -- be/test/io/cache/block_file_cache_test_meta_store.cpp passed. I did not run the targeted BE UT in this runner because thirdparty/installed and thirdparty/installed/bin/protoc are absent.
  • User focus: no additional user-provided review focus was specified.

Subagent conclusions:

  • optimizer-rewrite: found no optimizer/rewrite, semantic-equivalence, join, aggregate, or parallel plan-path issue; convergence round 1 returned NO_NEW_VALUABLE_FINDINGS.
  • tests-session-config: found no regression-test, expected-output, session/config propagation, compatibility, or basic style issue; convergence round 1 returned NO_NEW_VALUABLE_FINDINGS.

Overall opinion: the patch is a narrowly scoped deterministic test fix and looks good from static review.

@freemandealer freemandealer changed the title [fix](filecache) fix flaky be UT for LRU dump branch-4.1: [fix](filecache) fix flaky be UT for LRU dump (pick#65427) Jul 9, 2026
@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.06% (26934/37375)
Line Coverage 55.16% (287291/520847)
Region Coverage 52.69% (240143/455781)
Branch Coverage 53.75% (103558/192649)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants