Add capabilities for llms

[planger]

PO: @eneufeld

Each llm needs to be able to define capabilities.
The reason for this is that not every model has the same capabilities and requests can then fail.
Here an example

• make a request with sonnet 3.7 with thinking
• switch the model in the chat
• make a new request

There is a point for this in the AI Epic: Models need to define capabilities (this is needed for features like image, audio and thinking)

Feature: Adaptive UI based on LLM capabilities

  Scenario: Hide picture input when LLM does not support pictures
    Given the selected LLM does not support image input
    When the user opens the chat interface
    Then the option to add pictures to the chat should not be visible

  Scenario: Reject dropped image with feedback when LLM does not support pictures
    Given the selected LLM does not support image input
    When the user drops a picture onto the chat interface
    Then the picture should not be added to the chat
    And the user should receive feedback that pictures are not supported by the current model

  Scenario: Remove thinking blocks when LLM does not support thinking
    Given the selected LLM does not support thinking blocks
    When the user composes a request
    Then the UI should not allow adding thinking blocks to the request
    And any existing thinking blocks should be removed from the request

[CamilleLetavernier]

I tried adding "user-specified" Capabilities to LLMs, but that didn't go well: currently, LLMs are defined as a simple string in the preferences, making it very difficult to describe any additional data.

Once we take the model ID and instantiate it, it's then easier to add some structure, as we have a proper class to work with. So instead of manually specifying capabilities on each model, I looked for a way to make it dynamic.

I noticed that Ollama has an endpoint listing the Model Capabilities; so the capabilities can easily be defined dynamically at runtime for each model. However, according to ChatGPT, neither OpenAI nor Anthropic provide a similar API (I haven't verified this):

OpenAI:

OpenAI provides an API endpoint to list available models, but there is no official endpoint that returns a detailed list of each model’s capabilities (e.g., supports function calling, vision, JSON mode, etc.) directly via API at this time.

Anthropic:

There is:

No /models endpoint to enumerate all available model IDs.

No metadata endpoint to query model capabilities dynamically.

This makes it very difficult to implement capabilities, especially for user-defined models. For default models, we could have a method identifying "model families", and associate known capabilities to them. However this wouldn't work for user-defined models, especially future models with unknown capabilities; and we'd still need a more complex mechanism to define the models+capabilities.