[BUG] Unknown parameter: 'attachments[0].tools[0].file_search' when trying to attach a file to a message for the FileSearch tool.

[MickaelBNeron]

Describe the bug

My code (below) works for CodeInterpreter attachments, but gives the Unknown parameter: 'attachments[0].tools[0].file_search' error for FileSearch attachments. That should be incredibly easy to reproduce. My assistant is configured with both File Search and Code Interpreter enabled.

public async Task PostChat(
    string threadId,
    string model,
    string? instructions,
    bool webSearch,
    string prompt,
    List<(string FileName, Stream Stream)> fileSearch,
    List<(string FileName, Stream Stream)> codeInterpreter)
{
    var uploadFileSearchTasks = fileSearch.Select(async attachment => (
        File: await _fileClient.UploadFileAsync(attachment.Stream, attachment.FileName, FileUploadPurpose.Assistants),
        Tool: new List<ToolDefinition>() { ToolDefinition.CreateFileSearch() }
    ));

    var uploadCodeInterpreterTasks = codeInterpreter.Select(async attachment => (
        File: await _fileClient.UploadFileAsync(attachment.Stream, attachment.FileName, FileUploadPurpose.Assistants),
        Tool: new List<ToolDefinition>() { ToolDefinition.CreateCodeInterpreter() }
    ));

    var tasks = await Task.WhenAll(uploadFileSearchTasks.Concat(uploadCodeInterpreterTasks));

    var attachments = tasks.Select(x => new MessageCreationAttachment(x.File.Value.Id, x.Tool));

    var message = await _assistantClient.CreateMessageAsync(
        threadId,
        MessageRole.User,
        new[] {
            MessageContent.FromText(prompt.ToString()),
        },
        new MessageCreationOptions() {
            Attachments = new List<MessageCreationAttachment>(attachments),
        });

    var updates = _assistantClient.CreateRunStreamingAsync(threadId, _assistantId);
}

Steps to reproduce

1. Upload a file:

var fileId = await openAIFileClient.UploadFileAsync(stream, fileName FileUploadPurpose.Assistants);

2. Create an attachment with the Id of the uploaded file and the FileSearch tool:

var attachment = new MessageCreationAttachment(fileId, new List<ToolDefinition>() { ToolDefinition.CreateFileSearch() });

3. Send a message with that attachment:

var message = await _assistantClient.CreateMessageAsync(
    threadId,
    MessageRole.User,
    new[] {
        MessageContent.FromText(message),
    },
    new MessageCreationOptions() {
        Attachments = new List<MessageCreationAttachment>(attachments),
    });

Code snippets

public async Task PostChat(
    string threadId,
    string model,
    string? instructions,
    bool webSearch,
    string prompt,
    List<(string FileName, Stream Stream)> fileSearch,
    List<(string FileName, Stream Stream)> codeInterpreter)
{
    var uploadFileSearchTasks = fileSearch.Select(async attachment => (
        File: await _fileClient.UploadFileAsync(attachment.Stream, attachment.FileName, FileUploadPurpose.Assistants),
        Tool: new List<ToolDefinition>() { ToolDefinition.CreateFileSearch() }
    ));

    var uploadCodeInterpreterTasks = codeInterpreter.Select(async attachment => (
        File: await _fileClient.UploadFileAsync(attachment.Stream, attachment.FileName, FileUploadPurpose.Assistants),
        Tool: new List<ToolDefinition>() { ToolDefinition.CreateCodeInterpreter() }
    ));

    var tasks = await Task.WhenAll(uploadFileSearchTasks.Concat(uploadCodeInterpreterTasks));

    var attachments = tasks.Select(x => new MessageCreationAttachment(x.File.Value.Id, x.Tool));

    var message = await _assistantClient.CreateMessageAsync(
        threadId,
        MessageRole.User,
        new[] {
            MessageContent.FromText(prompt.ToString()),
        },
        new MessageCreationOptions() {
            Attachments = new List<MessageCreationAttachment>(attachments),
        });

    var updates = _assistantClient.CreateRunStreamingAsync(threadId, _assistantId);
}

OS

Windows

.NET version

8.0

Library version

2.2.0

[jsquire]

Hi @MickaelBNeron. Thanks for reaching out and we regret that you're experiencing difficulties. This looks to be a serialization bug in the library, where the tool registration is being formed incorrectly. For now, you can work around this by dropping to the protocol overload. Your example would look similar to:

var protocolFiles = uploadFileSearchTasks.Select(x => new
{
    file_id = x.File.Value.Id,
    tools = new[]
    {
        new
        {
            type = "file_search"
        }
    }
});

var protocolInterpreters = uploadCodeInterpreterTasks.Select(x => new
{
    file_id = x.File.Value.Id,
    tools = new[]
    {
        new
        {
            type = "code_interpreter"
        }
    }
});

var response = await client.CreateMessageAsync(
    threadId, 
    BinaryContent.Create(BinaryData.FromObjectAsJson(new
   {
        content = prompt.ToString(),
        role = "user",
        attachments = protocolFile.Concat(protocolInterpreters)s.ToArray()
    })));
        
var message = ModelReaderWriter.Read<ThreadMessage>(
    response.GetRawResponse().Content, 
    ModelReaderWriterOptions.Json);

[jsquire]

@ShivangiReja: This looks to be related to the way that the tools element of the attachments are serialized. The platform docs define this as:

"attachments": [
  {
    "file_id": "file-3mJqVZbeQnw2KGD3gSxV6q",
    "tools": [
      {
        "type": "file_search",
      }
    ]
  }
],

Currently, we're serializing as:

"attachments": [
  {
    "file_id": "file-3mJqVZbeQnw2KGD3gSxV6q",
    "tools": [
      {
        "type": "file_search",
        "file_search": {}
      }
    ]
  }
],

Note the extra file_search object in the tool.

[MickaelBNeron]

Thank you @jsquire . I can confirm that this workaround works.