Does prompt caching support LORA?

[tranlm]

Hi all,

I know that llama cpp "automatically" supports prompt caching by looking for the longest prefix in the string for each follow-up query. I'm wondering if this also works with LORA adapters? For example, say I have adapters A and B. I then do the following:

1. Call the base model with prompt t_1, ..., t_k.
2. Call the base model with adapter A with t_1, ..., t_k, t_{k+1}, ..., t_l.
3. Call the base model with adapter B with t_1, ..., t_k, t_{k+1}, ..., t_l, t_{l+1}, ..., t_m.

where 0 < k < l < m.

Would the prompt still be cached for the underlying base model (e.g. only the lora adapter operations are applied on each follow-up query)?