You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since interrupts occur during the tool loop, and the tool loop happens after generation is complete, if a developer is constructing message history only from reading streaming data they will not receive the interrupt metadata that is expected to resume. We have a few options:
Relax the constraint that a resume needs to correspond to a toolRequest with interrupt metadata - any toolRequest is fine.
Proactively begin tool execution the moment a tool request is generated and pause streaming until the tool response is ready.
Add a mechanism for the chunk wire protocol to signal a patch to an existing chunk.
Ignore the problem and document that you are expected to treat repsonse.messages as canonical and streaming is just a way to show progress.
I can see pros and cons to all of these. (3) is probably the most robust but it makes chunk handling even more complex than it already is. Maybe that's fine though? If it was something like:
{index: 1,role: "tool",content: [],// send an empty content arrray because we're not appendingreplaceContent: [{index: 2,replaceWith: {toolRequest: {...},metadata: {...}},}]}
Anyway, definitely not going to solve this before 1.0 so I think (4) is gonna have to be the short-term solution.
The text was updated successfully, but these errors were encountered:
Since interrupts occur during the tool loop, and the tool loop happens after generation is complete, if a developer is constructing message history only from reading streaming data they will not receive the interrupt metadata that is expected to resume. We have a few options:
repsonse.messages
as canonical and streaming is just a way to show progress.I can see pros and cons to all of these. (3) is probably the most robust but it makes chunk handling even more complex than it already is. Maybe that's fine though? If it was something like:
Anyway, definitely not going to solve this before 1.0 so I think (4) is gonna have to be the short-term solution.
The text was updated successfully, but these errors were encountered: