v2.17.0

⭐️ Highlights

🖼️ Image support for several model providers

Following the introduction of image support in Haystack 2.16.0, we've expanded this to more model providers in Haystack and Haystack Core integrations.

Now supported: Amazon Bedrock, Anthropic, Azure, Google, Hugging Face API, Meta Llama API, Mistral, Nvidia, Ollama, OpenAI, OpenRouter, STACKIT.

🧩 Extended components

We've improved several components to make them more flexible:

• MetadataRouter, which is used to route Documents based on metadata, has been extended to also support routing ByteStream objects.
• The SentenceWindowRetriever, which retrieves neighboring sentences around relevant Documents to provide full context, is now more flexible. Previously, its source_id_meta_field parameter accepted only a single field containing the ID of the original document. It now also accepts a list of fields, so that only documents matching all of the specified meta fields will be retrieved.

⬆️ Upgrade Notes

• MultiFileConverter outputs a new key failed in the result dictionary, which contains a list of files that failed to convert. The documents output is included only if at least one file is successfully converted. Previously, documents could still be present but empty if a file with a supported MIME type was provided but did not actually exist.

• The finish_reason field behavior in HuggingFaceAPIChatGenerator has been updated. Previously, the new finish_reason mapping (introduced in Haystack 2.15.0 release) was only applied when streaming was enabled. When streaming was disabled, the old finish_reason was still returned. This change ensures the updated finish_reason values are consistently returned regardless of streaming mode.

  How to know if you're affected: If you rely on finish_reason in responses from HuggingFaceAPIChatGenerator with streaming disabled, you may see different values after this upgrade.

  What to do: Review the updated mapping:

  • length → length
  • eos_token → stop
  • stop_sequence → stop
  • If tool calls are present → tool_calls

🚀 New Features

• Add support for ByteStream objects in MetadataRouter. It can now be used to route list[Documents] or list[ByteStream] based on metadata.
• Add support for the union type operator | (added in python 3.10) in serialize_type and Pipeline.connect(). These functions support both the typing.Union and | operators and mixtures of them for backwards compatibility.
• Added ReasoningContent as a new content part to the ChatMessage dataclass. This allows storing model reasoning text and additional metadata in assistant messages. Assistant messages can now include reasoning content using the reasoning parameter in ChatMessage.from_assistant(). We will progressively update the implementations for Chat Generators with LLMs that support reasoning to use this new content part.
• Updated SentenceWindowRetriever's source_id_meta_field parameter to also accept a list of strings. If a list of fields are provided, then only documents matching both fields will be retrieved.

⚡️ Enhancement Notes

• Added multimodal support to HuggingFaceAPIChatGenerator to enable vision-language model (VLM) usage with images and text. Users can now send both text and images to VLM models through Hugging Face APIs. The implementation follows the HF VLM API format specification and maintains full backward compatibility with text-only messages.
• Added serialization/deserialization methods for TextContent and ImageContent parts of ChatMessage.
• Made the lazy import error message clearer explaining that the optional dependency is missing.
• Adopted modern type hinting syntax using PEP 585 throughout the codebase. This improves readability and removes unnecessary imports from the typing module.
• Support subclasses of ChatMessage in Agent state schema validation. The validation now checks for issubclass(args[0], ChatMessage) instead of requiring exact type equality, allowing custom ChatMessage subclasses to be used in the messages field.
• The ToolInvoker run method now accepts a list of tools. When provided, this list overrides the tools set in the constructor, allowing you to switch tools at runtime in previously built pipelines.

🐛 Bug Fixes

• The English and German abbreviation files used by the SentenceSplitter are now included in the distribution. They were previously missing due to a config in the .gitignore file.

• Add encoding format keyword argument to OpenAI client when creating embeddings.

• Addressed incorrect assumptions in the ChatMessage class that raised errors in valid usage scenario.

  1. ChatMessage.from_user with content_parts: Previously, at least one text part was required, even though some model providers support messages with only image parts. This restriction has been removed. If a provider has such a limitation, it should now be enforced in the provider's implementation.

  2. ChatMessage.to_openai_dict_format: Messages containing multiple text parts weren't supported, despite this being allowed by the OpenAI API. This has now been corrected.

• Improved validation in the ChatMessage.from_user class method. The method now raises an error if neither text nor content_parts are provided. It does not raise an error if text is an empty string.

• Ensure that the score field in SentenceTransformersSimilarityRanker is returned as a Python float instead of numpy.float32. This prevents potential serialization issues in downstream integrations.

• Raise a RuntimeError when AsyncPipeline.run is called from within an async context, indicating that run_async should be used instead.

• Prevented in-place mutation of input Document objects in all Extractor and Classifier components by creating copies with dataclasses.replace before processing.

• Prevented in-place mutation of input Document objects in all DocumentEmbedder components by creating copies with dataclasses.replace before processing.

• FileTypeRouter has a new parameter raise_on_failure with default value to False. When set to True, FileNotFoundError is always raised for non-existent files. Previously, this exception was raised only when processing a non-existent file and the meta parameter was provided to run().

• Return a more informative error message when attempting to connect two components and the sender component does not have any OutputSockets defined.

• Fix tracing context not propagated to tools when running via ToolInvoker.run_async

• Ensure consistent behavior in SentenceTransformersDiversityRanker. Like other rankers, it now returns all documents instead of raising an error when top_k exceeds the number of available documents.

💙 Big thank you to everyone who contributed to this release!

@abdokaseb @Amnah199 @anakin87 @bilgeyucel @ChinmayBansal @datbth @davidsbatista @dfokina @LastRemote
@mpangrazzi @RafaelJohn9 @rolshoven @SaraCalla @SaurabhLingam @sjrl

v2.17.0-rc2

v2.17.0-rc2

v2.17.0-rc1

v2.17.0-rc1

v2.16.1

Release Notes

v2.16.1

Bug Fixes

• Improved validation in the ChatMessage.from_user class method. The method now raises an error if neither text nor content_parts are provided. It does not raise an error if text is an empty string.

v2.16.1-rc1

Release Notes

v2.16.1-rc1

Bug Fixes

• Improved validation in the ChatMessage.from_user class method. The method now raises an error if neither text nor content_parts are provided. It does not raise an error if text is an empty string.

v2.16.0

⭐️ Highlights

🧠 Agent Breakpoints

This release introduces Agent Breakpoints, a powerful new feature that enhances debugging and observability when working with Haystack Agents. You can pause execution mid-run by inserting breakpoints in the Agent or its tools to inspect internal state and resume execution seamlessly. This brings fine-grained control to agent development and significantly improves traceability during complex interactions.

from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint
from haystack.dataclasses import ChatMessage

chat_generator_breakpoint = Breakpoint(
    component_name="chat_generator", 
    visit_count=0, 
    snapshot_file_path="debug_snapshots"
)
agent_breakpoint = AgentBreakpoint(break_point=chat_generator_breakpoint, agent_name='calculator_agent')

response = agent.run(
    messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")],
    break_point=agent_breakpoint
)

🖼️ Multimodal Pipelines and Agents

You can now blend text and image capabilities across generation, indexing, and retrieval in Haystack.

• New ImageContent Dataclass: A dedicated structure to store image data along with base64_image, mime_type, detail, and metadata.

• Image-Aware Chat Generators: Image inputs are now supported in OpenAIChatGenerator

from haystack.dataclasses import ImageContent, ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator

image_url = "https://cdn.britannica.com/79/191679-050-C7114D2B/Adult-capybara.jpg"
image_content = ImageContent.from_url(image_url)

message = ChatMessage.from_user(
    content_parts=["Describe the image in short.", image_content]
)

llm = OpenAIChatGenerator(model="gpt-4o-mini")
print(llm.run([message])["replies"][0].text)

• Powerful Multimodal Components:

  • PDFToImageContent, ImageFileToImageContent, DocumentToImageContent: Convert PDFs, image files, and Documents into ImageContent objects.
  • LLMDocumentContentExtractor: Extract text from images using a vision-enabled LLM.
  • SentenceTransformersDocumentImageEmbedder: Generate embeddings from image-based documents using models like CLIP.
  • DocumentLengthRouter: Route documents based on textual content length—ideal for distinguishing scanned PDFs from text-based ones.
  • DocumentTypeRouter: Route documents automatically based on MIME type metadata.

• Prompt Building with Image Support: The ChatPromptBuilder now supports templates with embedded images, enabling dynamic multimodal prompt creation.

With these additions, you can now build multimodal agents and RAG pipelines that reason over both text and visual content, unlocking richer interactions and retrieval capabilities.

👉 Learn more about multimodality in our Introduction to Multimodal Text Generation.

🚀 New Features

• Add to_dict and from_dict to ByteStream so it is consistent with our other dataclasses in having serialization and deserialization methods.

• Add to_dict and from_dict to classes StreamingChunk, ToolCallResult, ToolCall, ComponentInfo, and ToolCallDelta to make it consistent with our other dataclasses in having serialization and deserialization methods.

• Added the tool_invoker_kwargs param to Agent so additional kwargs can be passed to the ToolInvoker like max_workers and enable_streaming_callback_passthrough.

• ChatPromptBuilder now supports special string templates in addition to a list of ChatMessage objects. This new format is more flexible and allows structured parts like images to be included in the templatized ChatMessage.

  from haystack.components.builders import ChatPromptBuilder 
  from haystack.dataclasses.chat_message import ImageContent
  
  template = """
  {% message role="user" %}
  Hello! I am {{user_name}}. 
  What's the difference between the following images? 
  {% for image in images %}
  {{ image | templatize_part }}
  {% endfor %}
  {% endmessage %}
  """ 
  
  images=[
      ImageContent.from_file_path("apple-fruit.jpg"),
      ImageContent.from_file_path("apple-logo.jpg")
  ]
  
  builder = ChatPromptBuilder(template=template) builder.run(user_name="John", images=images)

• Added convenience class methods to the ImageContent dataclass to create ImageContent objects from file paths and URLs.

• Added multiple converters to help convert image data between different formats:

• DocumentToImageContent: Converts documents sourced from PDF and image files into ImageContents.

• ImageFileToImageContent: Converts image files to ImageContent objects.

• ImageFileToDocument: Converts image file references into empty Document objects with associated metadata.

• PDFToImageContent: Converts PDF files to ImageContent objects.

• Chat Messages with the user role can now include images using the new ImageContent dataclass. We've added image support to OpenAIChatGenerator, and plan to support more model providers over time.

• Raise a warning when a pipeline can no longer proceed because all remaining components are blocked from running and no expected pipeline outputs have been produced. This scenario can occur legitimately. For example, in pipelines with mutually exclusive branches where some components are intentionally blocked. To help avoid false positives, the check ensures that none of the expected outputs (as defined by Pipeline().outputs()) have been generated during the current run.

• Added source_id_meta_field and split_id_meta_field to SentenceWindowRetriever for customizable metadata field names. Added raise_on_missing_meta_fields to control whether a ValueError is raised if any of the documents at runtime are missing the required meta fields (set to True by default). If False, then the documents missing the meta field will be skipped when retrieving their windows, but the original document will still be included in the results.

• Add a ComponentInfo dataclass to the haystack.dataclasses module. This dataclass is used to store information about the component. We pass it to StreamingChunk so we can tell from which component a stream is coming from.

• Pass the component_info to the StreamingChunk in the OpenAIChatGenerator, AzureOpenAIChatGenerator, HuggingFaceAPIChatGenerator and HuggingFaceLocalChatGenerator.

• Added the enable_streaming_callback_passthrough to the ToolInvoker init, run and run_async methods. If set to True the ToolInvoker will try and pass the streaming_callback function to a tool's invoke method only if the tool's invoke method has streaming_callback in its signature.

• Added new HuggingFaceTEIRanker component to enable reranking with Text Embeddings Inference (TEI) API. This component supports both self-hosted Text Embeddings Inference services and Hugging Face Inference Endpoints.

• Added a raise_on_failure boolean parameter to OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder. If set to True then the component will raise an exception when there is an error with the API request. It is set to False by default to so the previous behavior of logging an exception and continuing is still the default.

• ToolInvoker now executes tool_calls in parallel for both sync and async mode.

• Add AsyncHFTokenStreamingHandler for async streaming support in HuggingFaceLocalChatGenerator

• Updated StreamingChunk to add the fields tool_calls, tool_call_result, index, and start to make it easier to format the stream in a streaming callback.

⬆️ Upgrade Notes

• HuggingFaceAPIGenerator might no longer work with the Hugging Face Inference API. As of July 2025, the Hugging Face Inference API no longer offers generative models that support the text_generation endpoint. Generative models are now only available through providers that support the chat_completion endpoint. As a result, the HuggingFaceAPIGenerator component might not work with the Hugging Face Inference API. It still works with Hugging Face Inference Endpoints and self-hosted TGI instances. To use generative models via Hugging Face Inference API, please use the HuggingFaceAPIChatGenerator component, which supports the chat_completion endpoint.

• All parameters of the Pipeline.draw() and Pipeline.show() methods must now be specified as keyword arguments. Example:

pipeline.draw(
    path="output.png",
    server_url="https://custom-server.com",
    params=None,
    timeout=30,
    super_component_expansion=False
)

• The deprecated async_executor parameter has been removed from the ToolInvoker class. Please use the max_workers parameter instead and a ThreadPoolExecutor with these workers will be created automatically for parallel tool invocations.

• The deprecated State class has been removed from the haystack.dataclasses module. The State class is now part of the haystack.components.agents module.

• Remove the deserialize_value_with_schema_legacy function from the base_serialization module. This function was used to deserialize State objects created with Haystack 2.14.0 or older. Support for the old serialization format is removed in Haystack 2.16.0.

⚡️ Enhancement Notes

• Add guess_mime_type parameter to Bytestream.from_file_path()

• Add the init parameter skip_empty_documents to the DocumentSplitter component. The default value is True. Setting it to False can be useful when downstream components in the Pipeline (like LLMDocumentContentExtractor) can extract text from non-textual documents.

• Test that our type validation and connection validation works with builtin python types introduced in 3.9. We found that these types were already s...

v2.16.0-rc1

v2.16.0-rc1

v2.15.2

Enhancement Notes

• We’ve relaxed the requirements for the ToolCallDelta dataclass (introduced in Haystack 2.15). Previously, creating a ToolCallDelta instance required either the parameters argument or the name to be set. This constraint has now been removed to align more closely with OpenAI's SDK behavior.
  The change was necessary as the stricter requirement was causing errors in certain hosted versions of open-source models that adhere to the OpenAI SDK specification.

Bug Fixes

• Fixed a bug in the print_streaming_chunk utility function that prevented ToolCall name from being printed.

v2.15.2-rc1

v2.15.2-rc1

v2.15.1

Bug Fixes

• Fix _convert_streaming_chunks_to_chat_message which is used to convert Haystack StreamingChunks into a Haystack ChatMessage. This fixes the scenario where one StreamingChunk contains two ToolCallDetlas in StreamingChunk.tool_calls. With this fix this correctly saves both ToolCallDeltas whereas before they were overwriting each other. This only occurs with some LLM providers like Mistral (and not OpenAI) due to how the provider returns tool calls.