Image Understanding with RAG Cookbook

[robtinn]

Summary

The goal of this PR is to add a new cookbook to the multimodal examples in this repo. The new OpenAI image understanding capabilities have unlocked new use cases for systems that analyse multimodal data and many real-world datasets are multimodal. The cookbook covers creating a model which can leverage text and image context to answer questions about a synthetic dataset, with evals to test performance for different models.

Motivation

Why are these changes necessary? How do they improve the cookbook?

• There has been a lot of interest in the new OpenAI image understanding capabilities recently and this notebook explores how you can leverage them as part of a RAG workflow.
• In addition, many teams are still using the chat completions API when a lot of new functionality is available in the Responses API which this cookbook focuses on, in particular File Search capabilities with image analysis context.
• I am also helping organise content for a hackathon around customer service and many of the use cases that are popular projects for the hackathon are covered in this notebook, (I hope to share this cookbook as a template for starting the hackathon).
• Generally, a lot of the examples for the Evals API are out-of-date or incomplete and this cookbook also covers a simple example of the Evals API end-to-end.

---

TODO: add authors in authors.yaml