Skip to content

fix: keep the turn alive when a provider stream has no finish reason - #1450

Open
ramantehlan wants to merge 1 commit into
mainfrom
sd/finish-reason-not-found
Open

ramantehlan wants to merge 1 commit into
mainfrom
sd/finish-reason-not-found

Conversation

@ramantehlan

@ramantehlan ramantehlan commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

@ai-sdk/openai-compatible reports a stream that closed without a finish_reason by enqueuing an error part, then still emits a normal finish. Our stream mapper threw on any error part, so the turn died and discarded everything the model had already sent. A 15 minute session came back as [Model Error] Response stream ended without a finish reason. with none of the answer kept in context.

This makes a missing finish reason a first-class outcome instead of a failure. The provider (nuroute in the reported session) sent every chunk with finish_reason: null and then closed the body cleanly; the gateway side is already handled by truefoundry/tfy-llm-gateway#4710.

Refs AGE-2862, AGE-2857

Changes

  • FinishReason gains incomplete beside the five OpenAI values. The AI SDK error reason maps to it, and mapStreamToChunks starts there so a stream that ends before any finish-step is also honest.
  • The adapter's error part is tolerated only once the stream produced output. An empty response still fails, so a confusing error is not traded for a blank answer.
  • RawAssistantMessageWithUsage.finish_reason is now non-null, and the dead | null on finishReasonErrorMessage is gone.
  • The chat shows an inline warning under the partial answer, following the context-compaction data part pattern with a slot-overridable IncompleteResponseBanner.
  • computeAgentStepsSplit looks past that notice so a trailing part does not push the answer into the Agent steps accordion.
  • AgentThread logs a warning with the thread, model and output tokens.

Why incomplete and not null

The schema already documents finish_reason: null as "the provider omitted it", but the UI runtime uses finishReason != null to tell a finished model message from the opening shell that AgentThread emits first. The same predicate guards mergeModelMessageDelta, mergeStreamEventDelta, isFinishedModelMessage and earliestOpenModelMessageId. Emitting null would make the UI treat a completed message as still open: re-appending deltas on replay (duplicated text), replaying the shell over the finished message (truncated tool-call arguments), and resuming the cursor from a message that already ended. A non-null value leaves that invariant alone.

How was this tested?

New tests cover the tolerated stream, tool calls surviving it, the empty-output guard, a stream with no finish-step at all, the turn finishing with status: done, the content part, the agent-steps split, and the banner.

Full suite passes locally: 608 trueforge-core, 303 assistant-ui-runtime, 2123 trueforge-ui, 930 trueforge, 18 frontend.

Checklist

  • I have read the contributing guidelines
  • pnpm build, pnpm test, pnpm typecheck, pnpm lint:ci, and pnpm format:check pass locally
  • Tests added/updated where it makes sense
  • No hand-edits to generated code (packages/trueforge-sdk, python/trueforge_sdk, .github/fern/openapi/openapi.json, docs/openapi.json) — the FinishReason enum change is picked up when maintainers regenerate after merge
  • Docs / .env.example updated if configuration or behavior changed — no configuration change

Note

Medium Risk
Changes core LLM stream error handling and finish-reason semantics on every provider stream; behavior is guarded by tests but affects turn success vs failure at runtime.

Overview
When a model stream closes without a finish reason, the harness no longer throws away partial output. It introduces incomplete as a first-class FinishReason, keeps accumulated text/tool calls when @ai-sdk reports InvalidResponseDataError after content arrived, and still fails turns that produced no output.

Core/runtime: mapStreamToChunks tolerates the missing-finish-reason error path; streams with no finish-step default to incomplete. finishReasonErrorMessage does not treat incomplete as a turn failure. AgentThread logs a warning when this happens.

UI/runtime: Assistant content gains a trailing model.incomplete_response data part, rendered as an IncompleteResponseBanner under the answer. computeAgentStepsSplit treats that notice like trailing metadata so the partial reply stays in the final section, not buried in agent steps.

Reviewed by Cursor Bugbot for commit be54391. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment on lines +26 to +30
declare module '../theme/SlotsProvider.js' {
interface AtomSlots {
IncompleteResponseBanner: typeof IncompleteResponseBanner;
}
}

@datadog-official datadog-official Bot Nov 20, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Low: Code Quality Violation

ES2015 module syntax is preferred over namespaces. (...read more)

Namespaces should be avoided as an outdated feature of TypeScript. Use module syntax instead.

Helpful? 👍/👎 View in Datadog  Leave us feedback  Documentation
🚩 Report as false positive. Flags the rule for review to improve detection accuracy.

@datadog-official

datadog-official Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Code Quality  Code Vulnerabilities  IaC  Library Vulnerabilities  Secrets

⚠️ Warnings

🛠️ 1 Code quality issue detected

Low: typescript-best-practices/no-namespace Avoid TypeScript namespaces View rule
packages/trueforge-ui/src/atoms/IncompleteResponseBanner.tsx:26-30

ℹ️ Info

No other issues found (see more)

🛡️ No new code vulnerabilities
📚 No new vulnerable libraries detected
🧱 No new Infrastructure as Code (IaC) issues
🔑 No new secrets detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: be54391 | Docs | Give us feedback!

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.

toolCallStates.size > 0;
if (InvalidResponseDataError.isInstance(raw) && hasOutput) {
break;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty start events bypass failure guard

Medium Severity

The empty-output guard treats a reasoning-start or tool-input-start as usable output, even when no text, reasoning, or tool arguments arrived. A stream that dies right after those start events is kept as incomplete instead of failing, so the user can get a blank answer plus warning, or AgentThread can continue and invoke a tool with empty or truncated arguments.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.

toolCallStates.size > 0;
if (InvalidResponseDataError.isInstance(raw) && hasOutput) {
break;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All invalid-response errors are swallowed

Medium Severity

Any InvalidResponseDataError is tolerated once some output exists, not only the openai-compatible “stream ended without a finish reason” case. Other invalid-payload failures from the AI SDK (malformed chunks, failed field validation, non-string tool-call ids) are then turned into a successful incomplete turn instead of failing.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.

`@ai-sdk/openai-compatible` reports a stream that closed without a
`finish_reason` by enqueuing an error part, then still emits a normal
`finish`. Our stream mapper threw on any error part, so the turn died and
discarded everything the model had already sent. A 15 minute session came
back as "[Model Error] Response stream ended without a finish reason." with
nothing of the answer kept in context.

Make a missing finish reason a first-class outcome instead of a failure.
`FinishReason` gains `incomplete` beside the five OpenAI values, the AI SDK
`error` reason maps to it, and `mapStreamToChunks` starts there so a stream
that ends before any `finish-step` is also honest. The error part is only
tolerated once the stream produced output; an empty response still fails,
so a confusing error is not traded for a blank answer.

`incomplete` rather than `null` because the UI runtime uses
`finishReason != null` to tell a finished model message from the opening
shell. A null would make it re-append deltas and truncate tool arguments
on replay; a non-null value leaves that invariant alone.

The chat shows an inline warning under the partial answer, following the
context-compaction `data` part pattern with a slot-overridable
`IncompleteResponseBanner`. `computeAgentStepsSplit` looks past that notice
so a trailing part does not push the answer into the Agent steps accordion.
`AgentThread` logs a warning with the thread, model and output tokens.

Refs AGE-2862, AGE-2857

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
@ramantehlan
ramantehlan force-pushed the sd/finish-reason-not-found branch from 94ef0c8 to be54391 Compare November 20, 2026 10:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant