Repository navigation
fix: keep the turn alive when a provider stream has no finish reason - #1450
ramantehlan wants to merge 1 commit into
Conversation
| declare module '../theme/SlotsProvider.js' { | ||
| interface AtomSlots { | ||
| IncompleteResponseBanner: typeof IncompleteResponseBanner; | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 Low: Code Quality Violation
ES2015 module syntax is preferred over namespaces. (...read more)
Namespaces should be avoided as an outdated feature of TypeScript. Use module syntax instead.
Helpful? 👍/👎
🚩 Report as false positive. Flags the rule for review to improve detection accuracy.
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.
| toolCallStates.size > 0; | ||
| if (InvalidResponseDataError.isInstance(raw) && hasOutput) { | ||
| break; | ||
| } |
There was a problem hiding this comment.
Empty start events bypass failure guard
Medium Severity
The empty-output guard treats a reasoning-start or tool-input-start as usable output, even when no text, reasoning, or tool arguments arrived. A stream that dies right after those start events is kept as incomplete instead of failing, so the user can get a blank answer plus warning, or AgentThread can continue and invoke a tool with empty or truncated arguments.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.
| toolCallStates.size > 0; | ||
| if (InvalidResponseDataError.isInstance(raw) && hasOutput) { | ||
| break; | ||
| } |
There was a problem hiding this comment.
All invalid-response errors are swallowed
Medium Severity
Any InvalidResponseDataError is tolerated once some output exists, not only the openai-compatible “stream ended without a finish reason” case. Other invalid-payload failures from the AI SDK (malformed chunks, failed field validation, non-string tool-call ids) are then turned into a successful incomplete turn instead of failing.
Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.
`@ai-sdk/openai-compatible` reports a stream that closed without a `finish_reason` by enqueuing an error part, then still emits a normal `finish`. Our stream mapper threw on any error part, so the turn died and discarded everything the model had already sent. A 15 minute session came back as "[Model Error] Response stream ended without a finish reason." with nothing of the answer kept in context. Make a missing finish reason a first-class outcome instead of a failure. `FinishReason` gains `incomplete` beside the five OpenAI values, the AI SDK `error` reason maps to it, and `mapStreamToChunks` starts there so a stream that ends before any `finish-step` is also honest. The error part is only tolerated once the stream produced output; an empty response still fails, so a confusing error is not traded for a blank answer. `incomplete` rather than `null` because the UI runtime uses `finishReason != null` to tell a finished model message from the opening shell. A null would make it re-append deltas and truncate tool arguments on replay; a non-null value leaves that invariant alone. The chat shows an inline warning under the partial answer, following the context-compaction `data` part pattern with a slot-overridable `IncompleteResponseBanner`. `computeAgentStepsSplit` looks past that notice so a trailing part does not push the answer into the Agent steps accordion. `AgentThread` logs a warning with the thread, model and output tokens. Refs AGE-2862, AGE-2857 Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
94ef0c8 to
be54391
Compare


Summary
@ai-sdk/openai-compatiblereports a stream that closed without afinish_reasonby enqueuing an error part, then still emits a normalfinish. Our stream mapper threw on any error part, so the turn died and discarded everything the model had already sent. A 15 minute session came back as[Model Error] Response stream ended without a finish reason.with none of the answer kept in context.This makes a missing finish reason a first-class outcome instead of a failure. The provider (
nuroutein the reported session) sent every chunk withfinish_reason: nulland then closed the body cleanly; the gateway side is already handled by truefoundry/tfy-llm-gateway#4710.Refs AGE-2862, AGE-2857
Changes
FinishReasongainsincompletebeside the five OpenAI values. The AI SDKerrorreason maps to it, andmapStreamToChunksstarts there so a stream that ends before anyfinish-stepis also honest.RawAssistantMessageWithUsage.finish_reasonis now non-null, and the dead| nullonfinishReasonErrorMessageis gone.datapart pattern with a slot-overridableIncompleteResponseBanner.computeAgentStepsSplitlooks past that notice so a trailing part does not push the answer into the Agent steps accordion.AgentThreadlogs a warning with the thread, model and output tokens.Why
incompleteand notnullThe schema already documents
finish_reason: nullas "the provider omitted it", but the UI runtime usesfinishReason != nullto tell a finished model message from the opening shell thatAgentThreademits first. The same predicate guardsmergeModelMessageDelta,mergeStreamEventDelta,isFinishedModelMessageandearliestOpenModelMessageId. Emitting null would make the UI treat a completed message as still open: re-appending deltas on replay (duplicated text), replaying the shell over the finished message (truncated tool-call arguments), and resuming the cursor from a message that already ended. A non-null value leaves that invariant alone.How was this tested?
New tests cover the tolerated stream, tool calls surviving it, the empty-output guard, a stream with no
finish-stepat all, the turn finishing withstatus: done, the content part, the agent-steps split, and the banner.Full suite passes locally: 608 trueforge-core, 303 assistant-ui-runtime, 2123 trueforge-ui, 930 trueforge, 18 frontend.
Checklist
pnpm build,pnpm test,pnpm typecheck,pnpm lint:ci, andpnpm format:checkpass locallypackages/trueforge-sdk,python/trueforge_sdk,.github/fern/openapi/openapi.json,docs/openapi.json) — theFinishReasonenum change is picked up when maintainers regenerate after merge.env.exampleupdated if configuration or behavior changed — no configuration changeNote
Medium Risk
Changes core LLM stream error handling and finish-reason semantics on every provider stream; behavior is guarded by tests but affects turn success vs failure at runtime.
Overview
When a model stream closes without a finish reason, the harness no longer throws away partial output. It introduces
incompleteas a first-classFinishReason, keeps accumulated text/tool calls when@ai-sdkreportsInvalidResponseDataErrorafter content arrived, and still fails turns that produced no output.Core/runtime:
mapStreamToChunkstolerates the missing-finish-reason error path; streams with nofinish-stepdefault toincomplete.finishReasonErrorMessagedoes not treatincompleteas a turn failure.AgentThreadlogs a warning when this happens.UI/runtime: Assistant content gains a trailing
model.incomplete_responsedata part, rendered as anIncompleteResponseBannerunder the answer.computeAgentStepsSplittreats that notice like trailing metadata so the partial reply stays in the final section, not buried in agent steps.Reviewed by Cursor Bugbot for commit be54391. Bugbot is set up for automated code reviews on this repo. Configure here.