You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/aws/amazon-bedrock.md
+11Lines changed: 11 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -228,6 +228,17 @@ Some capabilities of the chat completions operations are currently only availabl
228
228
229
229
***Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/appstore/modules/genai/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/appstore/modules/genai/genai-for-mx/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**.
230
230
231
+
##### Tool Choice
232
+
233
+
All [tool choice types](/appstore/modules/genai/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/appstore/modules/genai/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below:
234
+
235
+
| GenAI Commons (Mendix) | Amazon Bedrock |
236
+
| --- | --- |
237
+
| Auto | Auto |
238
+
| Any | Any |
239
+
| None | Tools removed from request |
240
+
| Tool | Tool |
241
+
231
242
#### RetrieveAndGenerate {#retrieve-and-generate}
232
243
233
244
The `Retrieve and Generate` activity can be used for conversations leveraging Retrieval Augmented Generation through a knowledge base. This operation corresponds to the *RetrieveAndGenerate* microflow.
|| text-embedding-ada-002, text-embedding-3-small, text-embedding-3-large | Embeddings | text | embeddings||
78
77
| Amazon Bedrock | Amazon Titan Text G1 - Express, Amazon Titan Text G1 - Lite, Amazon Titan Text G1 - Premier | Chat Completions | text, document (except Titan Premier) | text ||
@@ -104,4 +103,4 @@ In addition to the models listed above, you can also connect to other models by
104
103
105
104
* To connect to other [foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html) and implement them in your app, use the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/).
106
105
* To connect to [Snowflake Cortex LLM](https://docs.snowflake.com/en/sql-reference/functions/complete-snowflake-cortex) functions, [configure the Snowflake REST SQL connector for Snowflake Cortex Analyst](/appstore/connectors/snowflake/snowflake-rest-sql/#cortex-analyst).
107
-
* To implement your connector compatible with the other components, use the [GenAI Commons](/appstore/modules/genai/commons/) interface.
106
+
* To implement your connector compatible with the other components, use the [GenAI Commons](/appstore/modules/genai/commons/) interface and follow the how-to [Build Your Own GenAI Connector](/appstore/modules/genai/how-to/byo-connector/).
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/Mx GenAI Connector.md
+12-3Lines changed: 12 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -161,9 +161,7 @@ For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/co
161
161
In the entire conversation, you can pass up to five documents that are smaller than 4.5 MB each. The following file types are accepted: PDF, CSV, DOC, DOCX, XLS, XLSX, HTML, TXT, and MD.
162
162
163
163
{{% alert color="info" %}}
164
-
When adding a document to the `FileCollection`, you can optionally use the `TextContent` parameter to pass the file name. Ensure the file name excludes its extension before passing it to the file collection.
165
-
166
-
Note that the model uses the file name when analyzing documents, which could make it vulnerable to prompt injection. Depending on your use case, you may choose to modify the string or not pass it at all.
164
+
The model uses the file name when analyzing documents, which may introduce a potential vulnerability to prompt injection. To reduce this risk, consider modifying file names before including them in the request.
167
165
{{% /alert %}}
168
166
169
167
### Knowledge Base Operations
@@ -254,6 +252,17 @@ The **Documentation** pane displays the documentation for the currently selected
All [tool choice types](/appstore/modules/genai/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/appstore/modules/genai/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below:
For more guidance on how to use microflows in your logic, Mendix recommends downloading the [GenAI Showcase App](https://marketplace.mendix.com/link/component/220475), which demonstrates a variety of example use cases and applies almost all of the Mendix Cloud GenAI operations. The starter apps in the [Mendix Components](/appstore/modules/genai/#mendix-components) list can also be used as inspiration or simply adapted for a specific use case.
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/openai.md
+27-2Lines changed: 27 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -219,6 +219,20 @@ When you use Azure OpenAI, it is recommended to set the optional `MaxTokens` inp
219
219
220
220
For more information on vision, see [OpenAI](https://platform.openai.com/docs/guides/vision) and [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision) documentation.
221
221
222
+
#### Document Chat {#chatcompletions-document}
223
+
224
+
Document chat enables the model to interpret and analyze PDF documents, allowing it to answer questions and perform tasks based on the document content. To use document chat, you can send an optional [FileCollection](/appstore/modules/genai/genai-for-mx/commons/#filecollection) containing one or more documents along with a single message.
225
+
226
+
For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/genai-for-mx/commons/#chat-add-message-to-request).
227
+
228
+
You can send up to 100 pages across multiple files, with a maximum combined size of 32 MB per conversation. Currently, processing multiple files with OpenAI is not always guaranteed and can lead to unexpected behavior (for example, only one file being processed).
229
+
230
+
{{% alert color="info" %}}
231
+
Azure OpenAI does not currently support file input.
232
+
233
+
Note that the model uses the file name when analyzing documents, which may introduce a potential vulnerability to prompt injection. To reduce this risk, consider modifying the string or not passing it at all.
OpenAI also provides image generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [image generation operation from GenAI Commons](/appstore/modules/genai/genai-for-mx/commons/#generate-image).
@@ -257,11 +271,11 @@ OpenAI-specific exposed microflow actions to construct requests via drag-and-dro
257
271
258
272
This microflow changes the `ResponseFormat` of the `OpenAIRequest_Extension` object, which will be created for a `Request` if not present. This describes the format that the chat completions model must output. The default behavior for OpenAI's models currently is `Text`. This operation must be used to enable JSON mode by providing the value `JSONObject` as input.
259
273
260
-
#### Files: Initialize Collection with OpenAI File {#initialize-filecollection}
274
+
#### Files: Initialize Collection with OpenAI Image {#initialize-filecollection}
261
275
262
276
This microflow initializes a new `FileCollection` and adds a new `FileDocument` or URL. Optionally, the `Image Detail` or a description using `TextContent` can be passed.
263
277
264
-
#### Files: Add OpenAI File to Collection {#add-file}
278
+
#### Files: Add OpenAI Image to Collection {#add-file}
265
279
266
280
This microflow adds a new `FileDocument` or URL to an existing `FileCollection`. Optionally, the `Image Detail` or a description using `TextContent` can be passed.
267
281
@@ -280,6 +294,17 @@ The **Documentation** pane displays the documentation for the currently selected
All [tool choice types](/appstore/modules/genai/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/appstore/modules/genai/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below:
For more inspiration or guidance on how to use those microflows in your logic, Mendix recommends downloading the [GenAI Showcase App](https://marketplace.mendix.com/link/component/220475), which demonstrates a variety of example use cases.
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-commons.md
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -141,8 +141,9 @@ This is a file in a collection of files that belongs to a message. Each instance
141
141
|`FileContent`| Depending on the `ContentType`, this is either a URL or the base64-encoded file data. |
142
142
| `ContentType` | This describes the type of file data. Supported content types are either URL or base64-encoded file data. For more information, see the [ENUM_ContentType](#enum-contenttype) section.
143
143
| `FileType` | Currently only images and documents are supported file types. In general, not all file types might be supported by all AI providers or models. For more information, see the [ENUM_FileType](#enum-filetype).
144
-
|`TextContent`| An optional text content describing the file content or giving it a specific name. This can be used to refer to specific files in the prompt of the message. |
144
+
|`TextContent`| An optional text content describing the file content. |
145
145
|`FileExtension`| Extension of the file, e.g. *png* or *pdf*. Note that this attribute may only be filled if the ContentType equals *Base64* and can be empty. |
146
+
|`FileName`| If a FileDocument is added, the `Filename` is extracted automatically. |
0 commit comments