Skip to content

model selection cookbook with image updates #1815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 34 additions & 13 deletions examples/partners/model_selection_guide/model_selection_guide.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@
"\n",
"### OpenAI Model Evolution \n",
"\n",
"![OpenAI Model Evolution](./images/2.2_model_evolution.png)\n",

"![OpenAI Model Evolution](../../../images/2.2_model_evolution.png)\n",

"\n",
"### Key Characteristics\n",
"\n",
Expand All @@ -79,7 +81,9 @@
"\n",
"## 3A. Use Case: Long-Context RAG for Legal Q&A\n",
"\n",
"![Long-Context RAG for Legal Q&A](images/3A_rag_task_card.png)\n",

"![Long-Context RAG for Legal Q&A](../../../images/3A_rag_task_card.png)\n",

"## 🗂️ TL;DR Matrix\n",
"\n",
"This table summarizes the core technology choices and their rationale for **this specific Long-Context Agentic RAG implementation**.\n",
Expand Down Expand Up @@ -133,7 +137,9 @@
"id": "db9bad1b",
"metadata": {},
"source": [
"![Hierarchical Router](images/3A_rag_hierarchical_router.png)\n",

"![Hierarchical Router](../../../images/3A_rag_hierarchical_router.png)\n",

"\n",
"\n",
"## Agentic RAG System: Model Usage\n",
Expand Down Expand Up @@ -1815,7 +1821,9 @@
"================================================================================\n",
"\n",
"## 3B. Use Case: AI Co-Scientist for Pharma R&D\n",
"![AI Co-Scientist for Pharma R&D](images/3B_reasoning_task_card.png)\n",

"![AI Co-Scientist for Pharma R&D](../../../images/3B_reasoning_task_card.png)\n",

"\n",
"This section details how to build an AI system that functions as a \"co-scientist\" to accelerate experimental design in pharmaceutical R&D, focusing on optimizing a drug synthesis process under specific constraints.\n",
"\n",
Expand Down Expand Up @@ -1855,7 +1863,9 @@
"\n",
"The system employs a multi-agent architecture that emulates a high-performing scientific team. Different AI components, acting in specialized roles (such as ideation, critique, and learning from outcomes), collaborate using various models and tools to execute the workflow.\n",
"\n",
"![AI Co-Scientist Architecture](images/3B_coscientist_architecture.png)\n",

"![AI Co-Scientist Architecture](../../../images/3B_coscientist_architecture.png)\n",

"\n",
"### 2.1. **Scientist Input & Constraints:** \n",
"The process starts with the scientist defining the goal, target compound, and constraints."
Expand Down Expand Up @@ -2463,7 +2473,9 @@
"\n",
"## 3C. Use Case: Insurance Claim Processing\n",
"\n",
"![](./images/3C_insurance_task_card.png)\n",

"![](../../../images/3C_insurance_task_card.png)\n",

"\n",
"Many businesses are faced with the task of digitizing hand filled forms. In this section, we will demonstrate how OpenAI can be used to digitize and validate a hand filled insurance form. While this is a common problem for insurance, the same techniques can be applied to a variety of other industries and forms, for example tax forms, invoices, and more.\n",
"\n",
Expand Down Expand Up @@ -2493,7 +2505,9 @@
"\n",
"The high level basic architecture of the solution is shown below.\n",
"\n",
"![](./images/3C_insurance_architecture.png)\n",

"![](../../../images/3C_insurance_architecture.png)\n",

"\n",
"This task is complex and requires a wide variety of model capabilities, including vision, function calling, reasoning, and structured output. While `o3` is capable of doing all of these at once, we found during experimentation that `o4-mini` alone was not sufficient to achieve the necessary performance. Due to the higher relative costs of `o3`, we instead opted for a two-stage approach.\n",
"\n",
Expand All @@ -2503,7 +2517,9 @@
"\n",
"To demonstrate concretely how this works, let's look at a sample image of an insurance form.\n",
"\n",
"![](./images/3C_insurance_form.png)\n",

"![](../../../images/3C_insurance_form.png)\n",

"\n",
"While the form itself is fairly straightforward, there is missing data and ambiguous information that will be difficult for a traditional OCR system to fill out correctly. First, notice that the zip code and county have been omitted. Second, the email address of the user is ambiguous \\-- it could be `[email protected]` or `[email protected]`. In the following sections, we will walk through how a well-designed solution can handle these ambiguities and return the correct form results.\n",
"\n",
Expand Down Expand Up @@ -3186,7 +3202,10 @@
"\n",
"## Adaptation Decision Tree\n",
"\n",
"![Model Selection Decision Tree](images/3D_model_selection_flowchart.png)\n",

"![Model Selection Decision Tree](../../../images/3D_model_selection_flowchart.png)\n",


"\n",
"## Communicating Model Selection to Non-Technical Stakeholders\n",
"\n",
Expand Down Expand Up @@ -3286,10 +3305,12 @@
"\n",
"## Contributors\n",
"\n",
"- Kashyap Coimbatore Murali\n",
"- Nate Harada \n",
"- Sai Prashanth Soundararaj \n",
"- Shikhar Kwatra "

"- [Kashyap Coimbatore Murali](https://www.linkedin.com/in/kashyap-murali/)\n",
"- [Nate Harada](https://www.linkedin.com/in/nate-harada/) \n",
"- [Sai Prashanth Soundararaj](https://www.linkedin.com/in/saiprashanths/)\n",
"- [Shikhar Kwatra](https://www.linkedin.com/in/shikharkwatra/)"

]
}
],
Expand Down
Binary file added images/2.2_model_evolution.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3A_rag_hierarchical_router.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3A_rag_task_card.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3B_coscientist_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3B_reasoning_task_card.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3C_insurance_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3C_insurance_form.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3C_insurance_task_card.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/3D_model_selection_flowchart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.