Skip to content

Commit 890b706

Browse files
committed
SAS-1317: update after document chat support for OpenAI
1 parent cc87811 commit 890b706

File tree

4 files changed

+21
-9
lines changed

4 files changed

+21
-9
lines changed

content/en/docs/appstore/use-content/platform-supported-content/modules/genai/_index.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,7 @@ Mendix connectors offer direct support for the following models:
7171
|--------------|---------------------|---------------------|-------------------|-----------|-------------------------|
7272
| Mendix Cloud GenAI | Anthropic Claude 3.5 Sonnet | Chat Completions | text, image, document | text | Function calling |
7373
| | Cohere Embed English, Cohere Embed Multilingual | Embeddings | text | embeddings | |
74-
| Azure / OpenAI | gpt-3.5-turbo | Chat completions | text | text | Function calling |
75-
| | gpt-4, gpt-4-turbo, gpt-4o, gpt-4o mini, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4.5-preview | Chat completions | text, image | text | Function calling |
74+
| Azure / OpenAI | gpt-4, gpt-4-turbo, gpt-4o, gpt-4o mini, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4.5-preview | Chat completions | text, image, document (OpenAI only) | text | Function calling |
7675
| | DALL·E 2, DALL·E 3, gpt-image-1 | Image generation | text | image | |
7776
| | text-embedding-ada-002, text-embedding-3-small, text-embedding-3-large | Embeddings | text | embeddings| |
7877
| Amazon Bedrock | Amazon Titan Text G1 - Express, Amazon Titan Text G1 - Lite, Amazon Titan Text G1 - Premier | Chat Completions | text, document (except Titan Premier) | text | |
@@ -104,4 +103,4 @@ In addition to the models listed above, you can also connect to other models by
104103

105104
* To connect to other [foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html) and implement them in your app, use the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/).
106105
* To connect to [Snowflake Cortex LLM](https://docs.snowflake.com/en/sql-reference/functions/complete-snowflake-cortex) functions, [configure the Snowflake REST SQL connector for Snowflake Cortex Analyst](/appstore/connectors/snowflake/snowflake-rest-sql/#cortex-analyst).
107-
* To implement your connector compatible with the other components, use the [GenAI Commons](/appstore/modules/genai/commons/) interface.
106+
* To implement your connector compatible with the other components, use the [GenAI Commons](/appstore/modules/genai/commons/) interface and follow the how-to [Build Your Own GenAI Connector](/appstore/modules/genai/how-to/byo-connector/).

content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/Mx GenAI Connector.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -161,9 +161,7 @@ For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/co
161161
In the entire conversation, you can pass up to five documents that are smaller than 4.5 MB each. The following file types are accepted: PDF, CSV, DOC, DOCX, XLS, XLSX, HTML, TXT, and MD.
162162

163163
{{% alert color="info" %}}
164-
When adding a document to the `FileCollection`, you can optionally use the `TextContent` parameter to pass the file name. Ensure the file name excludes its extension before passing it to the file collection.
165-
166-
Note that the model uses the file name when analyzing documents, which could make it vulnerable to prompt injection. Depending on your use case, you may choose to modify the string or not pass it at all.
164+
Note that the model uses the file name when analyzing documents, which could make it vulnerable to prompt injection. Depending on your use case, you may choose to modify the file's name before adding it to the request.
167165
{{% /alert %}}
168166

169167
### Knowledge Base Operations

content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/openai.md

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -219,6 +219,20 @@ When you use Azure OpenAI, it is recommended to set the optional `MaxTokens` inp
219219

220220
For more information on vision, see [OpenAI](https://platform.openai.com/docs/guides/vision) and [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision) documentation.
221221

222+
#### Document Chat {#chatcompletions-document}
223+
224+
Document chat enables the model to interpret and analyze PDF documents, allowing them to answer questions and perform tasks related to the content. To use document chat, an optional [FileCollection](/appstore/modules/genai/genai-for-mx/commons/#filecollection) containing one or multiple documents must be sent along with a single message.
225+
226+
For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/genai-for-mx/commons/#chat-add-message-to-request).
227+
228+
In the entire conversation, you can pass up 100 pages across multiple files and a maximum of 32MB. Currently, processing multiple files with OpenAI is not always guaranteed and can lead to unexpected behavior (e.g., only using one document).
229+
230+
{{% alert color="info" %}}
231+
Azure OpenAI does not support the use of file input at the moment.
232+
233+
Note that the model uses the file name when analyzing documents, which could make it vulnerable to prompt injection. Depending on your use case, you may choose to modify the string or not pass it at all.
234+
{{% /alert %}}
235+
222236
#### Image Generations {#image-generations-configuration}
223237

224238
OpenAI also provides image generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [image generation operation from GenAI Commons](/appstore/modules/genai/genai-for-mx/commons/#generate-image).
@@ -257,11 +271,11 @@ OpenAI-specific exposed microflow actions to construct requests via drag-and-dro
257271

258272
This microflow changes the `ResponseFormat` of the `OpenAIRequest_Extension` object, which will be created for a `Request` if not present. This describes the format that the chat completions model must output. The default behavior for OpenAI's models currently is `Text`. This operation must be used to enable JSON mode by providing the value `JSONObject` as input.
259273

260-
#### Files: Initialize Collection with OpenAI File {#initialize-filecollection}
274+
#### Files: Initialize Collection with OpenAI Image {#initialize-filecollection}
261275

262276
This microflow initializes a new `FileCollection` and adds a new `FileDocument` or URL. Optionally, the `Image Detail` or a description using `TextContent` can be passed.
263277

264-
#### Files: Add OpenAI File to Collection {#add-file}
278+
#### Files: Add OpenAI Image to Collection {#add-file}
265279

266280
This microflow adds a new `FileDocument` or URL to an existing `FileCollection`. Optionally, the `Image Detail` or a description using `TextContent` can be passed.
267281

content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-commons.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,9 @@ This is a file in a collection of files that belongs to a message. Each instance
141141
| `FileContent` | Depending on the `ContentType`, this is either a URL or the base64-encoded file data. |
142142
| `ContentType` | This describes the type of file data. Supported content types are either URL or base64-encoded file data. For more information, see the [ENUM_ContentType](#enum-contenttype) section.
143143
| `FileType` | Currently only images and documents are supported file types. In general, not all file types might be supported by all AI providers or models. For more information, see the [ENUM_FileType](#enum-filetype).
144-
| `TextContent` | An optional text content describing the file content or giving it a specific name. This can be used to refer to specific files in the prompt of the message. |
144+
| `TextContent` | An optional text content describing the file content. |
145145
| `FileExtension` | Extension of the file, e.g. *png* or *pdf*. Note that this attribute may only be filled if the ContentType equals *Base64* and can be empty. |
146+
| `FileName` | If a FileDocument is added, the Filename is extracted automatically. |
146147

147148
#### `ToolCollection` {#toolcollection}
148149

0 commit comments

Comments
 (0)