Skip to content

Commit e7220f0

Browse files
committed
feat(py/genkit): add multipart tool support (tool.v2)
Add support for multipart tools that can return both structured output and rich content parts, matching the JS SDK defineTool with multipart: true. Changes: - Add TOOL_V2 enum value to ActionKind - Add multipart kwarg to @ai.tool() decorator - When multipart=True, register as tool.v2 with metadata type=tool.v2 and tool.multipart=True - When multipart=False (default), register both tool and tool.v2 (v2 wraps output in {output: result}) for discoverability - Update resolve_tool() and resolve_parameters() to look up both tool and tool.v2 kinds Includes 6 new tests covering registration, metadata, execution, and v2 wrapper behavior.
1 parent db54cf5 commit e7220f0

File tree

12 files changed

+739
-25
lines changed

12 files changed

+739
-25
lines changed

py/PARITY_AUDIT.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,7 @@ All samples except `provider-checks-hello` had `LICENSE` ✅ (now fixed).
220220
| `defineStreamingFlow` | ✅ (via options) |`DefineStreamingFlow` | ✅ (via streaming param) ||
221221
| `defineTool` ||`DefineTool` |`.tool()` decorator ||
222222
| `defineToolWithInputSchema` ||`DefineToolWithInputSchema` || Go-only |
223-
| `defineTool({multipart: true})` ||`DefineMultipartTool` | | ❌ Python missing (G18) |
223+
| `defineTool({multipart: true})` ||`DefineMultipartTool` | `.tool(multipart=True)` | ✅ (PR #4513) |
224224
| `defineModel` ||`DefineModel` |`define_model` ||
225225
| `defineBackgroundModel` ||`DefineBackgroundModel` |`define_background_model` ||
226226
| `definePrompt` ||`DefinePrompt` |`define_prompt` ||
@@ -330,7 +330,7 @@ Python users typically use `httpx` or `requests` directly.
330330
| Feature | JS | Go | Python | Gap Owner | Priority |
331331
|---------|:--:|:--:|:------:|-----------|:--------:|
332332
| `runFlow` / `streamFlow` client | ✅ (beta/client) ||| Go + Python | P2 |
333-
| `defineTool({multipart: true})` ||| | Python | P1 |
333+
| `defineTool({multipart: true})` ||| | | ✅ Done (PR #4513) |
334334
| Model API V2 (`apiVersion: 'v2'`) |||| Go + Python | P1 |
335335
| `defineDynamicActionProvider` |||| Go | P2 |
336336
| `defineIndexer` |||| Go | P2 |
@@ -490,11 +490,11 @@ Full plugin list from the repository README (10 plugins, 33 contributors, 54 rel
490490
| G14 | Python | Implement `validate_support` middleware | §8f ||
491491
| G15 | Python | Implement `download_request_media` middleware | §8f ||
492492
| G16 | Python | Implement `simulate_system_prompt` middleware | §8f ||
493-
| G18 | Python | Add multipart tool support (`defineTool({multipart: true})`) | §8h | |
493+
| G18 | Python | Add multipart tool support (`defineTool({multipart: true})`) | §8h | ✅ PR #4513 |
494494
| G19 | Python | Add Model API V2 (`defineModel({apiVersion: 'v2'})`) | §8i ||
495-
| G20 | Python | Add `context` parameter to `Genkit()` constructor | §8j | |
496-
| G21 | Python | Add `clientHeader` parameter to `Genkit()` constructor | §8j | |
497-
| G22 | Python | Add `name` parameter to `Genkit()` constructor | §8j | |
495+
| G20 | Python | Add `context` parameter to `Genkit()` constructor | §8j | ✅ PR #4512 |
496+
| G21 | Python | Add `clientHeader` parameter to `Genkit()` constructor | §8j | ✅ PR #4512 |
497+
| G22 | Python | Add `name` parameter to `Genkit()` constructor | §8j | ✅ PR #4512 |
498498
| G4 | Python | Move `augment_with_context` to define-model time | §8b.2 ||
499499
| G9 | Python | Add Pinecone vector store plugin | §5g ||
500500
| G10 | Python | Add ChromaDB vector store plugin | §5g ||
@@ -908,10 +908,10 @@ export function apiKey(
908908
909909
| Feature | JS | Python | Gap |
910910
|---------|:--:|:------:|:---:|
911-
| `defineTool({multipart: true})` | ✅ Supported. Creates a `MultipartToolAction` of type `tool.v2`. | ❌ Not supported. `define_tool` has no `multipart` parameter. | **G18** |
912-
| `MultipartToolAction` type | ✅ `tool.ts:107-122` — Action with `tool.v2` type, returns `{output?, content?}`. | ❌ Does not exist. | **G18** |
913-
| `MultipartToolResponse` type | ✅ `parts.ts` — Schema with `output` and `content` fields. | ⚠️ Type exists in `typing.py:933` but unused in tool definition. | Partial |
914-
| Auto-registration of `tool.v2` | ✅ Non-multipart tools are also registered as `tool.v2` with wrapped output. | ❌ No dual registration. | **G18** |
911+
| `defineTool({multipart: true})` | ✅ Supported. Creates a `MultipartToolAction` of type `tool.v2`. | `.tool(multipart=True)` registers as `tool.v2` with metadata `tool.multipart=True`. | ✅ PR #4513 |
912+
| `MultipartToolAction` type | ✅ `tool.ts:107-122` — Action with `tool.v2` type, returns `{output?, content?}`. | ✅ Registered under `ActionKind.TOOL_V2` with appropriate metadata. | ✅ PR #4513 |
913+
| `MultipartToolResponse` type | ✅ `parts.ts` — Schema with `output` and `content` fields. | ✅ Multipart tool functions return `{output?, content?}` dict. | ✅ PR #4513 |
914+
| Auto-registration of `tool.v2` | ✅ Non-multipart tools are also registered as `tool.v2` with wrapped output. | ✅ Non-multipart tools register both `tool` and `tool.v2` (v2 wraps output in `{output: result}`). | ✅ PR #4513 |
915915
916916
**JS** (`js/ai/src/tool.ts:306-335`):
917917
```ts
@@ -1040,11 +1040,11 @@ export interface GenkitOptions {
10401040
| G15 | Python | `download_request_media` middleware missing | P2 | `py/packages/genkit/src/genkit/blocks/middleware.py` | URL media transformed to data URI |
10411041
| G16 | Python | `simulate_system_prompt` missing | P2 | `py/packages/genkit/src/genkit/blocks/middleware.py` | system message rewritten for unsupported model |
10421042
| G17 | Python | `api_key()` context provider missing | P3 | `py/packages/genkit/src/genkit/core/context.py` | auth header extraction + policy callback tests |
1043-
| G18 | Python | multipart tool (`tool.v2`) missing | P1 | `py/packages/genkit/src/genkit/blocks/tools.py`, `.../blocks/generate.py` | tool call returns `output` + `content` parity |
1043+
| G18 | Python | ~~multipart tool (`tool.v2`) missing~~ | P1 | `ai/_registry.py`, `core/action/types.py`, `blocks/generate.py` | ✅ **Done** (PR #4513) |
10441044
| G19 | Python | Model API V2 runner interface missing | P1 | `py/packages/genkit/src/genkit/ai/_registry.py`, `.../blocks/model.py` | v2 model receives unified options struct |
1045-
| G20 | Python | `Genkit(context=...)` missing | P2 | `py/packages/genkit/src/genkit/ai/_aio.py` | context propagates to action executions |
1046-
| G21 | Python | `Genkit(clientHeader=...)` missing | P2 | `py/packages/genkit/src/genkit/ai/_aio.py`, `.../core/http_client.py` | outbound header includes custom token |
1047-
| G22 | Python | `Genkit(name=...)` missing | P2 | `py/packages/genkit/src/genkit/ai/_aio.py`, `.../core/reflection.py` | Dev UI/reflection shows custom name |
1045+
| G20 | Python | ~~`Genkit(context=...)` missing~~ | P2 | `ai/_aio.py`, `core/registry.py` | ✅ **Done** (PR #4512) |
1046+
| G21 | Python | ~~`Genkit(clientHeader=...)` missing~~ | P2 | `ai/_aio.py`, `core/constants.py` | ✅ **Done** (PR #4512) |
1047+
| G22 | Python | ~~`Genkit(name=...)` missing~~ | P2 | `ai/_aio.py`, `ai/_runtime.py`, `core/registry.py` | ✅ **Done** (PR #4512) |
10481048
| G23 | Go | `defineDynamicActionProvider` parity missing | P2 | `go/genkit/genkit.go`, `go/core/registry.go` | DAP action discovery + resolve test |
10491049
| G24 | Go | `defineIndexer` parity missing | P2 | `go/genkit/genkit.go`, `go/ai` indexing action | indexer registration + invoke test |
10501050
| G25 | Go | `defineReranker` + `rerank` runtime missing | P1 | `go/genkit/genkit.go`, `go/ai` reranker block | reranker registration + scoring call |
@@ -1198,9 +1198,9 @@ Reverse topological sort of the gap DAG yields the following dependency levels.
11981198
|----|-----|-----------|----------------|:------:|----------|
11991199
| **P1.1** | **G2** | Add `middleware` storage to `Action` class; implement `action_with_middleware()` wrapper that chains model-level middleware around `action.run()` | `core/action/_action.py` | L | G1, G12, G13, G15, G19 |
12001200
| **P1.2** | **G6** | Update `on_trace_start` callback signature to `(trace_id: str, span_id: str)` throughout action system | `core/action/_action.py`, `core/reflection.py`, `core/trace/` | S | G5 |
1201-
| **P1.3** | **G18** | Add multipart tool support: `define_tool(multipart=True)`, `MultipartToolAction` type `tool.v2`, dual registration for non-multipart tools | `blocks/tools.py`, `blocks/generate.py` | M | |
1202-
| **P1.4** | **G20** | Add `context` parameter to `Genkit()` that sets `registry.context` for default action context | `ai/_aio.py` | XS | |
1203-
| **P1.5** | **G21** | Add `clientHeader` parameter to `Genkit()` that appends to `GENKIT_CLIENT_HEADER` via `set_client_header()` | `ai/_aio.py`, `core/http_client.py` | XS | G8 |
1201+
| **P1.3** | **G18** | ~~Add multipart tool support: `define_tool(multipart=True)`, `MultipartToolAction` type `tool.v2`, dual registration for non-multipart tools~~ | `ai/_registry.py`, `core/action/types.py`, `blocks/generate.py` | M | ✅ **Done** (PR #4513) |
1202+
| **P1.4** | **G20** | ~~Add `context` parameter to `Genkit()` that sets `registry.context` for default action context~~ | `ai/_aio.py`, `core/registry.py` | XS | ✅ **Done** (PR #4512) |
1203+
| **P1.5** | **G21** | ~~Add `clientHeader` parameter to `Genkit()` that appends to `GENKIT_CLIENT_HEADER` via `set_client_header()`~~ | `ai/_aio.py`, `core/constants.py` | XS | ✅ **Done** (PR #4512) |
12041204
12051205
**Exit criteria**: All unit tests green for action middleware dispatch, span_id propagation, tool.v2 registration, and constructor parameter propagation.
12061206
@@ -1439,8 +1439,8 @@ Milestone ▲ P1 infra ▲ Middleware ▲ Full P1 ▲ Client
14391439
|----|:-----:|------|----------|:----------:|
14401440
| **PR-1a** | Core | G2 | Add `middleware` list to `Action.__init__()`, implement `action_with_middleware()` dispatch wrapper, unit tests for middleware chaining | — |
14411441
| **PR-1b** | Core | G6 | Update `on_trace_start` callback signature to `(trace_id, span_id)` across action system + tracing, update all call sites | — |
1442-
| **PR-1c** | Core | G18 | Multipart tool support: `define_tool(multipart=True)`, `tool.v2` action type, dual registration for non-multipart tools, unit tests | |
1443-
| **PR-1d** | Core | G20, G21 | `Genkit(context=..., client_header=...)` constructor params — small additive changes, can combine in one PR | — |
1442+
| **PR-1c** | Core | G18 | ~~Multipart tool support: `define_tool(multipart=True)`, `tool.v2` action type, dual registration for non-multipart tools, unit tests~~ | ✅ PR #4513 |
1443+
| **PR-1d** | Core | G20, G21, G22 | ~~`Genkit(context=..., client_header=..., name=...)` constructor params~~ | ✅ PR #4512 |
14441444
14451445
*PR-1a is the critical-path item. Land it first to unblock Phase 2.*
14461446

py/packages/genkit/src/genkit/ai/_registry.py

Lines changed: 35 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -522,15 +522,21 @@ async def get_tools():
522522
return define_dap_block(self.registry, config, fn)
523523

524524
def tool(
525-
self, name: str | None = None, description: str | None = None
525+
self, name: str | None = None, description: str | None = None, *, multipart: bool = False
526526
) -> Callable[[Callable[P, T]], Callable[P, T]]:
527527
"""Decorator to register a function as a tool.
528528
529529
Args:
530-
name: Optional name for the flow. If not provided, uses the function
530+
name: Optional name for the tool. If not provided, uses the function
531531
name.
532532
description: Description for the tool to be passed to the model;
533533
if not provided, uses the function docstring.
534+
multipart: If True, the tool is registered as a multipart tool
535+
(``tool.v2``). The function should return a dict with optional
536+
``output`` and ``content`` keys. If False (default), both a
537+
``tool`` and a ``tool.v2`` wrapper action are registered so that
538+
the tool is discoverable under both kinds. Mirrors JS SDK's
539+
``defineTool({ multipart: true })``.
534540
535541
Returns:
536542
A decorator function that registers the tool.
@@ -564,14 +570,40 @@ def tool_fn_wrapper(*args: Any) -> Any: # noqa: ANN401
564570
case _:
565571
raise ValueError('tool must have 0-2 args...')
566572

573+
tool_kind = cast(ActionKind, ActionKind.TOOL_V2 if multipart else ActionKind.TOOL)
574+
tool_metadata: dict[str, object] = {'type': 'tool.v2' if multipart else 'tool'}
575+
if multipart:
576+
tool_metadata['tool'] = {'multipart': True}
577+
567578
action = self.registry.register_action(
568579
name=tool_name,
569-
kind=cast(ActionKind, ActionKind.TOOL),
580+
kind=tool_kind,
570581
description=tool_description,
571582
fn=tool_fn_wrapper,
572583
metadata_fn=func,
584+
metadata=tool_metadata,
573585
)
574586

587+
# For non-multipart tools, also register a tool.v2 wrapper that
588+
# wraps the output in {output: result} so all tools are
589+
# discoverable under tool.v2, matching JS SDK behavior.
590+
if not multipart:
591+
592+
async def v2_wrapper_fn(*args: Any) -> dict[str, object]: # noqa: ANN401
593+
result = tool_fn_wrapper(*args)
594+
if asyncio.iscoroutine(result):
595+
result = await result
596+
return {'output': result}
597+
598+
self.registry.register_action(
599+
name=tool_name,
600+
kind=cast(ActionKind, ActionKind.TOOL_V2),
601+
description=tool_description,
602+
fn=v2_wrapper_fn,
603+
metadata_fn=func,
604+
metadata={'type': 'tool.v2'},
605+
)
606+
575607
@wraps(func)
576608
async def async_wrapper(*args: P.args, **kwargs: P.kwargs) -> Any: # noqa: ANN401
577609
"""Asynchronous wrapper for the tool function.

py/packages/genkit/src/genkit/blocks/generate.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -626,9 +626,7 @@ async def resolve_parameters(
626626
tools: list[Action[Any, Any, Any]] = []
627627
if request.tools:
628628
for tool_name in request.tools:
629-
tool_action = await registry.resolve_action(cast(ActionKind, ActionKind.TOOL), tool_name)
630-
if tool_action is None:
631-
raise Exception(f'Unable to resolve tool {tool_name}')
629+
tool_action = await resolve_tool(registry, tool_name)
632630
tools.append(tool_action)
633631

634632
format_def: FormatDef | None = None
@@ -842,6 +840,9 @@ async def _resolve_tool_request(tool: Action, tool_request_part: ToolRequestPart
842840
async def resolve_tool(registry: Registry, tool_name: str) -> Action:
843841
"""Resolve a tool by name from the registry.
844842
843+
Looks up the tool under both ``tool`` and ``tool.v2`` action kinds,
844+
matching the JS SDK's ``lookupToolByName`` behavior.
845+
845846
Args:
846847
registry: The registry to resolve the tool from.
847848
tool_name: The name of the tool to resolve.
@@ -853,6 +854,8 @@ async def resolve_tool(registry: Registry, tool_name: str) -> Action:
853854
ValueError: If the tool could not be resolved.
854855
"""
855856
tool = await registry.resolve_action(kind=cast(ActionKind, ActionKind.TOOL), name=tool_name)
857+
if tool is None:
858+
tool = await registry.resolve_action(kind=cast(ActionKind, ActionKind.TOOL_V2), name=tool_name)
856859
if tool is None:
857860
raise ValueError(f'Unable to resolve tool {tool_name}')
858861
return tool

py/packages/genkit/src/genkit/core/action/types.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ class ActionKind(StrEnum):
5656
RESOURCE = 'resource'
5757
RETRIEVER = 'retriever'
5858
TOOL = 'tool'
59+
TOOL_V2 = 'tool.v2'
5960
UTIL = 'util'
6061

6162

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
#!/usr/bin/env python3
2+
#
3+
# Copyright 2025 Google LLC
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
"""Tests for multipart tool support (tool.v2 action kind)."""
7+
8+
from typing import cast
9+
10+
import pytest
11+
12+
from genkit.ai import Genkit
13+
from genkit.core.action.types import ActionKind
14+
15+
16+
@pytest.mark.asyncio
17+
async def test_regular_tool_registers_both_kinds() -> None:
18+
"""A non-multipart tool registers under both 'tool' and 'tool.v2'."""
19+
ai = Genkit()
20+
21+
@ai.tool()
22+
def add(x: int) -> int:
23+
"""Add one."""
24+
return x + 1
25+
26+
tool_action = await ai.registry.resolve_action(ActionKind.TOOL, 'add')
27+
if tool_action is None:
28+
raise AssertionError('Expected tool action registered under ActionKind.TOOL')
29+
30+
v2_action = await ai.registry.resolve_action(ActionKind.TOOL_V2, 'add')
31+
if v2_action is None:
32+
raise AssertionError('Expected tool.v2 wrapper action registered under ActionKind.TOOL_V2')
33+
34+
35+
@pytest.mark.asyncio
36+
async def test_regular_tool_v2_wrapper_wraps_output() -> None:
37+
"""The tool.v2 wrapper for a regular tool wraps output in {output: result}."""
38+
ai = Genkit()
39+
40+
@ai.tool()
41+
def double(x: int) -> int:
42+
"""Double."""
43+
return x * 2
44+
45+
v2_action = await ai.registry.resolve_action(ActionKind.TOOL_V2, 'double')
46+
if v2_action is None:
47+
raise AssertionError('Expected tool.v2 wrapper')
48+
49+
result = await v2_action.arun(5)
50+
if not isinstance(result.response, dict):
51+
raise AssertionError(f'Expected dict response, got {type(result.response).__name__}')
52+
if result.response.get('output') != 10:
53+
raise AssertionError(f'Expected output=10, got {result.response}')
54+
55+
56+
@pytest.mark.asyncio
57+
async def test_multipart_tool_registers_as_tool_v2() -> None:
58+
"""A multipart tool is registered under 'tool.v2' only."""
59+
ai = Genkit()
60+
61+
@ai.tool(multipart=True)
62+
def rich_tool(query: str) -> dict:
63+
"""Return rich content."""
64+
return {'output': f'result for {query}', 'content': [{'text': 'extra'}]}
65+
66+
# Should be under tool.v2
67+
v2_action = await ai.registry.resolve_action(ActionKind.TOOL_V2, 'rich_tool')
68+
if v2_action is None:
69+
raise AssertionError('Expected multipart tool registered under ActionKind.TOOL_V2')
70+
71+
# Should NOT be under tool
72+
tool_action = await ai.registry.resolve_action(ActionKind.TOOL, 'rich_tool')
73+
if tool_action is not None:
74+
raise AssertionError('Multipart tool should NOT be registered under ActionKind.TOOL')
75+
76+
77+
@pytest.mark.asyncio
78+
async def test_multipart_tool_metadata() -> None:
79+
"""Multipart tool has correct metadata: type='tool.v2' and tool.multipart=True."""
80+
ai = Genkit()
81+
82+
@ai.tool(multipart=True)
83+
def my_multipart(x: int) -> dict:
84+
"""Multipart."""
85+
return {'output': x}
86+
87+
v2_action = await ai.registry.resolve_action(ActionKind.TOOL_V2, 'my_multipart')
88+
if v2_action is None:
89+
raise AssertionError('Expected multipart tool')
90+
91+
if v2_action.metadata.get('type') != 'tool.v2':
92+
raise AssertionError(f'Expected type="tool.v2", got {v2_action.metadata.get("type")!r}')
93+
94+
tool_meta = v2_action.metadata.get('tool')
95+
if not isinstance(tool_meta, dict):
96+
raise AssertionError(f'Expected dict for tool metadata, got {type(tool_meta).__name__}')
97+
tool_meta_dict = cast(dict[str, object], tool_meta)
98+
if tool_meta_dict.get('multipart') is not True:
99+
raise AssertionError(f'Expected tool.multipart=True, got {tool_meta!r}')
100+
101+
102+
@pytest.mark.asyncio
103+
async def test_multipart_tool_execution() -> None:
104+
"""Multipart tool can be executed and returns the function result directly."""
105+
ai = Genkit()
106+
107+
@ai.tool(multipart=True)
108+
def search(query: str) -> dict:
109+
"""Search."""
110+
return {'output': 'found it', 'content': [{'text': f'Details for {query}'}]}
111+
112+
v2_action = await ai.registry.resolve_action(ActionKind.TOOL_V2, 'search')
113+
if v2_action is None:
114+
raise AssertionError('Expected multipart tool')
115+
116+
result = await v2_action.arun('test query')
117+
response = result.response
118+
if not isinstance(response, dict):
119+
raise AssertionError(f'Expected dict, got {type(response).__name__}')
120+
if response.get('output') != 'found it':
121+
raise AssertionError(f'Expected output="found it", got {response.get("output")!r}')
122+
123+
124+
@pytest.mark.asyncio
125+
async def test_regular_tool_metadata_type() -> None:
126+
"""Regular (non-multipart) tool has metadata type='tool'."""
127+
ai = Genkit()
128+
129+
@ai.tool()
130+
def simple(x: int) -> int:
131+
"""Simple."""
132+
return x
133+
134+
tool_action = await ai.registry.resolve_action(ActionKind.TOOL, 'simple')
135+
if tool_action is None:
136+
raise AssertionError('Expected tool action')
137+
138+
if tool_action.metadata.get('type') != 'tool':
139+
raise AssertionError(f'Expected type="tool", got {tool_action.metadata.get("type")!r}')

py/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,7 @@ framework-dynamic-tools-demo = { workspace = true }
160160
framework-evaluator-demo = { workspace = true }
161161
framework-format-demo = { workspace = true }
162162
framework-middleware-demo = { workspace = true }
163+
framework-multipart-tools = { workspace = true }
163164
framework-prompt-demo = { workspace = true }
164165
framework-realtime-tracing-demo = { workspace = true }
165166
framework-restaurant-demo = { workspace = true }

0 commit comments

Comments
 (0)