-
Notifications
You must be signed in to change notification settings - Fork 2.2k
feat: Update streaming chunk #9424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…date HuggingFaceAPIChatGenerator to start following new StreamingChunk
tool_call: Optional[ToolCallDelta] = None | ||
tool_call_result: Optional[ToolCallResult] = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about making
content = Union[str, ToolCallDelta, ToolCallResult]
but this would be a breaking change b/c users expect content to always be a string. And this breaks StreamingChunk to ChatMessage implementations (mostly in private methods).
Pull Request Test Coverage Report for Build 15191185377Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
Looks very good to me already. 👍 |
Related Issues
StreamingChunk
to havetool_calls
,tool_results
,finish_reason
,usage
#9358Proposed Changes:
Updated StreamingChunk to add the fields
tool_call
,tool_call_result
,index
, andstart
to make it easier to format the stream in a streaming callback.StreamingChunk.tool_call
field to reflect that the arguments are a string delta rather than a Dictprint_streaming_chunk
utility method to use these new fields. This especially improves the formatting when using this with Agent.OpenAIChatGenerator
andHuggingFaceAPIChatGenerator
to follow the new dataclass.ToolInvoker
to follow the new format and also it now returns a final StreamingChunk that containsfinish_reason="tool_call_result"
in its metadata.How did you test it?
TODO add more unit tests
Tested Agent streaming locally with
OpenAIChatGenerator
Before Changes

After Changes
[ASSISTANT]
headerNotes for the reviewer
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
and added!
in case the PR includes breaking changes.