refactor(fuse): single cache with Arc fields and moka atomic upsert#700
refactor(fuse): single cache with Arc fields and moka atomic upsert#700
Conversation
bigbigxu
commented
Mar 6, 2026
- meta_cache: use single FastSyncCache<String, Arc>; wrap list/status/blocks in Arc so merge only does Arc::clone; use entry().and_upsert_with() for atomic read-modify-write (moka key-level lock), remove custom stripe lock
- node_state: pass path to put_open in new_reader, use if let for Cv branch
- meta_cache: use single FastSyncCache<String, Arc<CacheValue>>; wrap list/status/blocks in Arc so merge only does Arc::clone; use entry().and_upsert_with() for atomic read-modify-write (moka key-level lock), remove custom stripe lock - node_state: pass path to put_open in new_reader, use if let for Cv branch
There was a problem hiding this comment.
Pull request overview
Refactors metadata caching and FUSE reader creation to reduce lock contention and avoid repeated deep clones by consolidating cache entries and caching CV file block metadata for faster opens.
Changes:
- Refactor
MetaCacheto a singleFastSyncCache<String, Arc<CacheValue>>and use moka’s atomicentry().and_upsert_with()for updates. - Update FUSE
NodeState::new_readerto reuse cachedFileBlocksto construct a CVFsReaderwithout callingUnifiedFileSystem::open. - Extend
FsReaderto retainFileBlocks(and exposefile_blocks()), and increase the defaultmaster_conn_pool_sizeto 3.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| curvine-fuse/src/fs/state/node_state.rs | Uses cached FileBlocks to fast-path CV reader creation and populates the cache on open. |
| curvine-common/src/fs/meta_cache.rs | Consolidates list/status/blocks caches into a single moka-backed cache value with atomic upserts. |
| curvine-common/src/conf/client_conf.rs | Raises default master connection pool size from 1 to 3. |
| curvine-client/src/file/fs_reader.rs | Stores FileBlocks in FsReader and exposes it for downstream caching. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| let reader = match self.get_cached_blocks(path) { | ||
| Some(blocks) => { | ||
| let reader = FsReader::new(path.clone(), self.fs.fs_context().clone(), blocks)?; | ||
| UnifiedReader::Cv(reader) | ||
| } | ||
|
|
||
| None => { | ||
| let reader = self.fs.open(path).await?; | ||
|
|
||
| if self.conf.enable_meta_cache { | ||
| if let UnifiedReader::Cv(cv_reader) = &reader { | ||
| self.meta_cache | ||
| .put_open(path, cv_reader.file_blocks().clone()); | ||
| } | ||
| } | ||
|
|
||
| reader | ||
| } | ||
| }; |
There was a problem hiding this comment.
When cached blocks exist, new_reader builds UnifiedReader::Cv(FsReader) directly instead of calling UnifiedFileSystem::open. This bypasses unified mount/cache behavior (notably cache-mode validity checks like read_verify_ufs / check_cache_validity) and can serve stale CV data when the underlying UFS has changed. Consider only using cached blocks when the path is not mounted (or mount is fs-mode / read_verify_ufs is false), otherwise fall back to self.fs.open(path).await (or re-run an equivalent validity check before trusting cached blocks).
| let reader = match self.get_cached_blocks(path) { | |
| Some(blocks) => { | |
| let reader = FsReader::new(path.clone(), self.fs.fs_context().clone(), blocks)?; | |
| UnifiedReader::Cv(reader) | |
| } | |
| None => { | |
| let reader = self.fs.open(path).await?; | |
| if self.conf.enable_meta_cache { | |
| if let UnifiedReader::Cv(cv_reader) = &reader { | |
| self.meta_cache | |
| .put_open(path, cv_reader.file_blocks().clone()); | |
| } | |
| } | |
| reader | |
| } | |
| }; | |
| // Always go through UnifiedFileSystem::open to respect mount and cache validity checks. | |
| let reader = self.fs.open(path).await?; | |
| if self.conf.enable_meta_cache { | |
| if let UnifiedReader::Cv(cv_reader) = &reader { | |
| self.meta_cache | |
| .put_open(path, cv_reader.file_blocks().clone()); | |
| } | |
| } |
There was a problem hiding this comment.
The cache itself carries the risk of reading dirty data; opening it first defeats the purpose of caching.
| .entry(path.clone_uri()) | ||
| .and_upsert_with(|maybe_entry| { | ||
| let prev = maybe_entry.map(|e| e.into_value()); | ||
| Arc::new(CacheValue { | ||
| list: None, |
There was a problem hiding this comment.
invalidate_list uses entry(...).and_upsert_with(...), which will insert a new empty CacheValue if the key is missing. That means an invalidate call can add entries (and refresh TTL) instead of being a no-op. Consider only modifying existing entries (e.g., get then update) or using an entry API that avoids insertion/removes the key when appropriate (same applies to invalidate_status/invalidate_open).
There was a problem hiding this comment.
Using the GET method first can lead to issues with concurrent updates.