Prompt caching

[planger]

PO @eneufeld

With our system prompts and the amount of tool calls growing larger we should make sure to add support for caching.

OpenAI seems to have enabled caching by default: https://platform.openai.com/docs/guides/prompt-caching
For Anthropic we need to manually add caching markers: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
For Gemini there is implicit caching but for cost saving explicit caching is recommended: https://ai.google.dev/gemini-api/docs/caching?lang=nodene