Skip to content

Support empty thinking tags for Nemotron outputs #4740

@MaxFrax

Description

@MaxFrax

Pydantic-AI thinking separator issue

Problem

nvidia/Nemotron-120B-A12B emits thinking content at the start of the message and only closes it with an end tag like </think>.
There is no opening tag in the content.

Pydantic-AI’s thinking splitter expects both a start and end tag. If a profile is configured with an empty start tag (""), the splitter loops indefinitely and the output is not parsed correctly.

Impact

  • The agent’s final response includes the model’s internal thinking.
  • There is no clean separation between ThinkingPart and TextPart for these models.

Example

Raw content starts with thinking and ends with </think>, followed by the user-visible response.

Desired behavior

Treat content from the beginning until </think> as thinking, and return the remainder as the user-visible response.

Profile configuration

profile = OpenAIModelProfile(thinking_tags=('', '</think>'))

model = OpenAIChatModel(
    MODEL_NAME,
    provider=OpenAIProvider(api_key=settings.openai_api_key, base_url=BASE_URL),
    profile=profile,
)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions