Skip to content

Commit 7682557

Browse files
justindbaurwithinfocus
justindbaur
and
withinfocus
authored
Add ADR about adopting FusionCache for all our caching needs (#722)
* Add ADR about adopting FusionCache for all our caching needs * Address Claude feedback * Address Claude feedback * Apply suggestions from code review Co-authored-by: Matt Bishop <[email protected]> * Address PR feedback * Update titles * More feedback Co-authored-by: Matt Bishop <[email protected]> --------- Co-authored-by: Matt Bishop <[email protected]>
1 parent 719c8a2 commit 7682557

File tree

2 files changed

+178
-0
lines changed

2 files changed

+178
-0
lines changed

‎custom-words.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
# by the spellchecker. Please keep the list sorted alphabetically.
44
AndroidX
55
AOSP
6+
backplane
67
Bitwarden
78
bitwardensecret
89
bytemark

‎docs/architecture/adr/0028-adopt-fusion-cache.md‎

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
---
2+
adr: "0028"
3+
status: Accepted
4+
date: 2025-12-01
5+
tags: [server, server-sdk]
6+
---
7+
8+
# 0028 - Adopt FusionCache
9+
10+
<AdrTable frontMatter={frontMatter}></AdrTable>
11+
12+
## Context and problem statement
13+
14+
Numerous caching approaches currently exist in the `server` codebase, generally entailing the setup
15+
of an instance of `IDistributedCache` through keyed services. That cache is then injected into some
16+
service and some of the following things are done:
17+
18+
- A hard-coded prefix is added to every `Get` or `Set` style call
19+
- Entry options are hard-coded, or have minimal customization from `GlobalSettings`
20+
- The value is serialized, often in JSON format
21+
- If a value isn't found in the cache, a hard-coded value is used instead
22+
- If a value isn't found in the cache, a value from the database is retrieved and used instead
23+
- The value from the database is often cached in the distributed cache
24+
- An even faster copy of the data is stored in memory
25+
- Either in a local field or in `IMemoryCache`
26+
- Messages from other nodes are subscribed to in order to keep the memory copy in lockstep
27+
28+
One of the key things that is missing from that list is that no metrics are ever recorded on if the
29+
cache was useful, the most important metric being _"Was the value stored in the cache ever retrieved
30+
before it expired?"_
31+
32+
## Considered options
33+
34+
- Continue implementing custom caching solutions
35+
- Adopt `HybridCache`
36+
- Adopt `FusionCache`
37+
- Adopt `FusionCache` as `HybridCache`
38+
- Implement other third-party libraries
39+
40+
### Continue implementing custom caching solutions
41+
42+
**Pros**
43+
44+
- Maximum customizability by building everything yourself
45+
46+
**Cons**
47+
48+
- A lot of boilerplate that's easy to get wrong
49+
- No automatic metrics
50+
- Have to manually configure connection strings and TTL
51+
- No standard on key prefixes
52+
53+
### Adopt `HybridCache`
54+
55+
Adopt out-of-box library from Microsoft: [HybridCache][hybrid-cache].
56+
57+
**Pros**
58+
59+
- Great support and docs
60+
- L1 (in-memory, fast) and L2 (distributed, persistent) cache
61+
- Cache stampede protection (multiple requests simultaneously computing the same value)
62+
- Serialization configured through dependency injection (DI)
63+
64+
**Cons**
65+
66+
- Key prefixing is still manual
67+
- No memory cache synchronization of nodes, often called a backplane
68+
69+
### Adopt `FusionCache`
70+
71+
Adopt third-party package: [FusionCache][fusion-cache].
72+
73+
**Pros**
74+
75+
- Automatic key prefixing (setup in DI)
76+
- L1 and L2 cache
77+
- Cache stampede protection
78+
- Memory cache node synchronization mechanism
79+
- Very customizable
80+
81+
**Cons**
82+
83+
- Third-party package introduces potential support / maintenance risks
84+
85+
### Adopt `FusionCache` as `HybridCache`
86+
87+
It's possible to use FusionCache under the hood but inject and interact with `HybridCache`.
88+
89+
**Pros**
90+
91+
- All pros from [Adopt `FusionCache`](#adopt-fusioncache)
92+
- If we ever switched to `HybridCache`, it would be a DI-only change
93+
94+
**Cons**
95+
96+
- All cons from [Adopt `FusionCache`](#adopt-fusioncache)
97+
- Abstraction overhead
98+
- Hides the true implementation
99+
100+
### Implement other third-party libraries
101+
102+
[`CacheManager`][cache-manager] is not as popular or active compared to FusionCache. It does have
103+
built in serialization but it relies on `Newtonsoft.Json` and therefore would not be AOT friendly.
104+
We'd likely want to implement our own using `System.Text.Json`. It isn't clear if it has cache
105+
stampede protection. It also doesn't have built in metrics.
106+
107+
[`LazyCache`][lazy-cache] does not fit our needs in a few ways, most importantly it is only a memory
108+
cache. It has also not released a new version since September 2021.
109+
110+
[`CacheTower`][cache-tower] is less popular and less active than `FusionCache`. The only feature it
111+
is missing that we want is built in metrics.
112+
113+
## Decision outcome
114+
115+
Chosen option: **Adopt `FusionCache`**, because the `FusionCache` library contains all the features
116+
we need for our most complex scenarios and has enough customizability for our simpler scenarios.
117+
While `HybridCache` is impressive and would have the support of Microsoft it lacks the backplane
118+
extensibility so that we can synchronize the L1 cache of all our nodes.
119+
120+
### Positive consequences
121+
122+
- Able to get started quickly; nothing but injecting `IFusionCache` is needed for most cases
123+
- Consolidated documentation, guidance, and metrics by consuming it from a package
124+
- Usable outside of the `server` monorepo
125+
- Customizable without any code changes needed
126+
127+
### Negative consequences
128+
129+
- Could make caching too easy to use when caching isn't the right solution all the time
130+
131+
### Plan
132+
133+
- New features desiring cache should use `IFusionCache`
134+
- Finish up Caching package in server SDK
135+
- Individual migration plans for existing cache uses
136+
- If only `IDistributedCache` is needed memory cache and backplane can be turned off
137+
- If no `IDistributedCache` is needed it can be turned off and only memory and the backplane will
138+
be used.
139+
- Configure and document how to view caches hits and misses for your `FusionCache` usages
140+
141+
#### Today
142+
143+
We will use the [`AddExtendedCache`][add-extended-cache] available in `server`. You are able to call
144+
it with your own name and settings and then inject your own `IFusionCache` from keyed services using
145+
the given name.
146+
147+
#### In the near future
148+
149+
Caching will be built into our server SDK and supplied through a `Bitwarden.Server.Sdk.Caching`
150+
library. With no additional DI registration you will be able to inject `IFusionCache` using keyed
151+
services. You will then be able to configure your specific instance using named options or through
152+
configuration like such:
153+
154+
```json
155+
{
156+
"Caching": {
157+
"Uses": {
158+
"MyFeature": {
159+
"Fusion": {
160+
"DefaultEntryOptions": {
161+
"Duration": "00:01:00"
162+
}
163+
}
164+
}
165+
}
166+
}
167+
}
168+
```
169+
170+
[hybrid-cache]:
171+
https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid?view=aspnetcore-10.0
172+
[fusion-cache]: https://github.com/ZiggyCreatures/FusionCache
173+
[add-extended-cache]:
174+
https://github.com/bitwarden/server/blob/de5a81bdc4beea752de72539627521d840dd1976/src/Core/Utilities/ExtendedCacheServiceCollectionExtensions.cs#L25
175+
[cache-manager]: https://github.com/MichaCo/CacheManager
176+
[lazy-cache]: https://github.com/alastairtree/LazyCache
177+
[cache-tower]: https://github.com/TurnerSoftware/CacheTower

0 commit comments

Comments
 (0)