Document streamed tool call arguments and runtime events#6438
Document streamed tool call arguments and runtime events#6438lorenzejay wants to merge 4 commits into
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b6a4af5. Configure here.
| "streamed_arguments": False, | ||
| }, | ||
| ) | ||
| call_data["id"] = call_data.get("id") or getattr(item, "call_id", None) |
There was a problem hiding this comment.
Wrong tool id in stream
Medium Severity
Responses streaming can emit tool_call.id from the output item id when argument deltas create state before output_item.added. Later merging keeps that id because _process_responses_function_call_started only sets id when missing, so frames and StreamChunk.tool_id may not match the provider call_id the docs and tests expect.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit b6a4af5. Configure here.
📝 WalkthroughWalkthroughAdds streaming of incremental tool-call argument deltas during LLM tool calls in the OpenAI Responses API completion handler, propagated via a new BaseLLM helper and event listener branching, with console formatter changes to pause Live rendering for tool events, plus tests and documentation. ChangesTool call streaming feature
Sequence Diagram(s)sequenceDiagram
participant OpenAIAPI
participant OpenAICompletion
participant BaseLLM
participant EventBus
OpenAIAPI->>OpenAICompletion: response.output_item.added (function_call)
OpenAICompletion->>OpenAICompletion: _process_responses_function_call_started
OpenAICompletion->>BaseLLM: _emit_tool_call_stream_chunk_event
BaseLLM->>EventBus: LLMStreamChunkEvent(call_type=TOOL_CALL)
OpenAIAPI->>OpenAICompletion: response.function_call_arguments.delta
OpenAICompletion->>OpenAICompletion: _process_responses_function_call_delta
OpenAICompletion->>BaseLLM: _emit_tool_call_stream_chunk_event
BaseLLM->>EventBus: LLMStreamChunkEvent(accumulated arguments)
OpenAIAPI->>OpenAICompletion: response.output_item.done
OpenAICompletion->>OpenAICompletion: _process_responses_function_call_done
OpenAICompletion->>OpenAICompletion: merge into function_calls list
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
lib/crewai/tests/llms/openai/test_openai.py (1)
1893-1947: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider extracting shared
stream_eventsfixture.The sync and async tests construct nearly identical
stream_eventslists. A shared helper/fixture would reduce duplication if these tests evolve.Also applies to: 1980-2034
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/tests/llms/openai/test_openai.py` around lines 1893 - 1947, The sync and async OpenAI streaming tests are duplicating the same `stream_events` setup, so extract that repeated event list into a shared fixture or helper used by both test cases. Update the tests around the existing `stream_events` construction in the OpenAI test module so the common response/function-call event sequence is defined once and reused, keeping the sync and async assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/tests/llms/openai/test_openai.py`:
- Around line 2036-2057: The async streaming test is mocking `responses.create`
with a synchronous lambda, but `_ahandle_streaming_responses` awaits
`self._get_async_client().responses.create(...)`, so the mock must be awaitable.
Update the test setup around `MockAsyncStream` and `fake_client` to use an async
factory or `AsyncMock` for `responses.create`, so the await succeeds before the
stream is iterated.
In `@scripts/stream_tool_call_runner.py`:
- Around line 1-6: The module docstring in stream_tool_call_runner references
“Fill in OPENAI_API_KEY below,” but the script actually reads the key from the
environment via load_dotenv() and os.getenv("OPENAI_API_KEY") in the main flow.
Update the docstring to instruct users to set OPENAI_API_KEY as an environment
variable or in a .env file instead, and keep the wording aligned with the
behavior in the runner entrypoint.
---
Nitpick comments:
In `@lib/crewai/tests/llms/openai/test_openai.py`:
- Around line 1893-1947: The sync and async OpenAI streaming tests are
duplicating the same `stream_events` setup, so extract that repeated event list
into a shared fixture or helper used by both test cases. Update the tests around
the existing `stream_events` construction in the OpenAI test module so the
common response/function-call event sequence is defined once and reused, keeping
the sync and async assertions unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: fba17fea-4e13-40f4-bd10-dd257439964d
📒 Files selected for processing (11)
docs/edge/en/concepts/streaming.mdxdocs/edge/en/learn/consuming-streams.mdxdocs/edge/en/learn/streaming-crew-execution.mdxdocs/edge/en/learn/streaming-runtime-contract.mdxlib/crewai/src/crewai/events/event_listener.pylib/crewai/src/crewai/events/utils/console_formatter.pylib/crewai/src/crewai/llms/base_llm.pylib/crewai/src/crewai/llms/providers/openai/completion.pylib/crewai/tests/llms/openai/test_openai.pylib/crewai/tests/utilities/test_console_formatter_pause_resume.pyscripts/stream_tool_call_runner.py
| """Real OpenAI Responses API runner for streamed tool-call deltas. | ||
|
|
||
| Fill in ``OPENAI_API_KEY`` below, then run from the repository root: | ||
|
|
||
| uv run python scripts/stream_tool_call_runner.py | ||
| """ |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Docstring instructs "Fill in OPENAI_API_KEY below" but the script never has a place to fill it in.
The script only reads the key via os.getenv("OPENAI_API_KEY") after load_dotenv() (Lines 17, 32) — there's no literal placeholder to edit. Update the docstring to say to set the OPENAI_API_KEY environment variable (or .env entry) instead of "fill in ... below".
📝 Proposed docstring fix
"""Real OpenAI Responses API runner for streamed tool-call deltas.
-Fill in ``OPENAI_API_KEY`` below, then run from the repository root:
+Set the ``OPENAI_API_KEY`` environment variable (e.g. in a ``.env`` file),
+then run from the repository root:
uv run python scripts/stream_tool_call_runner.py
"""Also applies to: 29-37
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/stream_tool_call_runner.py` around lines 1 - 6, The module docstring
in stream_tool_call_runner references “Fill in OPENAI_API_KEY below,” but the
script actually reads the key from the environment via load_dotenv() and
os.getenv("OPENAI_API_KEY") in the main flow. Update the docstring to instruct
users to set OPENAI_API_KEY as an environment variable or in a .env file
instead, and keep the wording aligned with the behavior in the runner
entrypoint.
1 participant
Summary
llmandtoolschannelsTesting
Note
Medium Risk
Touches core LLM streaming and OpenAI Responses event handling; behavior changes for verbose console and stream consumers, though covered by new unit tests.
Overview
Adds streamed tool-call argument support for the OpenAI Responses API (sync and async): partial
function_call_argumentsdeltas are accumulated and emitted asLLMStreamChunkEventwithcall_type=TOOL_CALL, via a shared_emit_tool_call_stream_chunk_eventhelper onBaseLLM.Event and verbose console behavior is adjusted so tool-call construction is not mixed with text streaming: the event listener forwards accumulated arguments for
TOOL_CALLchunks without writing to the text buffer; the console formatter stops the live “final answer” panel for tool-call chunks and pauses live updates when tool usage panels appear.Documentation explains the split between
llm(llm_stream_chunkwhile the model builds the call:chunkvstool_call.function.arguments) andtools(actual execution), with examples for frame streaming, crewTOOL_CALLchunks, and the runtime contract.Reviewed by Cursor Bugbot for commit 0b8df05. Bugbot is set up for automated code reviews on this repo. Configure here.