Repository navigation
fix: keep the turn alive when a provider stream has no finish reason #1450
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| --- | ||
| "@truefoundry/trueforge-assistant-ui-runtime": patch | ||
| "@truefoundry/trueforge-core": patch | ||
| "@truefoundry/trueforge-ui": patch | ||
| --- | ||
|
|
||
| Keep a turn alive when the provider closes the model stream without a finish reason. The response | ||
| it did send is now kept and reported with the new `incomplete` finish reason, and the chat shows a | ||
| warning under the answer instead of failing the turn with "Response stream ended without a finish | ||
| reason". A stream that ends with no output at all still fails. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| /** | ||
| * assistant-ui `data` part name for a model response the provider cut short. Emitted when a | ||
| * model message finishes as `incomplete`, so the reader knows the answer above it may be partial | ||
| * even though the turn itself succeeded. | ||
| */ | ||
| export const INCOMPLETE_RESPONSE_DATA_NAME = 'model.incomplete_response'; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,6 +8,7 @@ import { createOpenAI } from '@ai-sdk/openai'; | |
| import type { OpenAICompatibleProviderOptions } from '@ai-sdk/openai-compatible'; | ||
| import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; | ||
| import type { JSONObject } from '@ai-sdk/provider'; | ||
| import { InvalidResponseDataError } from '@ai-sdk/provider'; | ||
| import type { ProviderOptions, ReasoningPart } from '@ai-sdk/provider-utils'; | ||
| import type { | ||
| AssistantContent, | ||
|
|
@@ -1163,8 +1164,10 @@ export function mapFinishReason(reason: FinishReason): RawAssistantMessageWithUs | |
| return 'tool_calls'; | ||
| case 'content-filter': | ||
| return 'content_filter'; | ||
| // No harness equivalent for these — fall back to 'stop'. | ||
| // Reached only once a stream error was tolerated, since any other one is thrown before this. | ||
| case 'error': | ||
| return 'incomplete'; | ||
| // The provider did report an ending, just not one the harness models. | ||
| case 'other': | ||
| return 'stop'; | ||
| default: { | ||
|
|
@@ -1313,7 +1316,8 @@ export async function* mapStreamToChunks({ | |
| return block; | ||
| }; | ||
| let finalUsage: CompletionUsage = getEmptyUsage(); | ||
| let finalFinishReason: RawAssistantMessageWithUsage['finish_reason'] = 'stop'; | ||
| // Only a `finish-step` can promote this: a stream that stops before one really is incomplete. | ||
| let finalFinishReason: RawAssistantMessageWithUsage['finish_reason'] = 'incomplete'; | ||
|
|
||
| for await (const part of stream) { | ||
| switch (part.type) { | ||
|
|
@@ -1453,6 +1457,17 @@ export async function* mapStreamToChunks({ | |
|
|
||
| case 'error': { | ||
| const raw = part.error; | ||
| // A stream that closes without a finish reason is reported as an error part, after which | ||
| // the adapter still emits `finish`. What already arrived is a usable answer, so keep it and | ||
| // let `finish-step` mark it `incomplete`. A response with nothing in it is still a failure. | ||
| const hasOutput = | ||
| accumulatedText.length > 0 || | ||
| accumulatedReasoning.length > 0 || | ||
| accumulatedThinking.length > 0 || | ||
| toolCallStates.size > 0; | ||
| if (InvalidResponseDataError.isInstance(raw) && hasOutput) { | ||
| break; | ||
| } | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All invalid-response errors are swallowedMedium Severity Any Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here. |
||
| const message = describeStreamError(raw); | ||
| // Preserve the original stream error as cause; toast/turn use .message only. | ||
| const cause = raw instanceof Error ? raw : new Error(message); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| import { Icon } from '../icons/Icon.js'; | ||
| import { cn } from './lib/cn.js'; | ||
|
|
||
| export type IncompleteResponseBannerProps = { | ||
| className?: string; | ||
| }; | ||
|
|
||
| /** Orange warning banner for a response the provider cut short, shown under the partial answer. */ | ||
| export function IncompleteResponseBanner({ className }: IncompleteResponseBannerProps) { | ||
| return ( | ||
| <div | ||
| role="status" | ||
| className={cn( | ||
| 'aui-incomplete-response-root border-warning-bg bg-warning-bg/10 text-warning-bg mt-2 inline-flex w-fit max-w-full items-center gap-3 rounded-xl border px-3 py-1.5 text-sm font-medium leading-[1.4]', | ||
| className, | ||
| )} | ||
| > | ||
| <Icon name="circle-exclamation" size="1rem" className="text-warning-bg shrink-0" aria-hidden /> | ||
| <span className="aui-incomplete-response-message min-w-0 break-words"> | ||
| The model provider ended this response early, so it may be incomplete. | ||
| </span> | ||
| </div> | ||
| ); | ||
| } | ||
|
|
||
| declare module '../theme/SlotsProvider.js' { | ||
| interface AtomSlots { | ||
| IncompleteResponseBanner: typeof IncompleteResponseBanner; | ||
| } | ||
| } | ||
|
Comment on lines
+26
to
+30
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Low: Code Quality ViolationES2015 module syntax is preferred over namespaces. (...read more)Namespaces should be avoided as an outdated feature of TypeScript. Use module syntax instead. Helpful? 👍/👎 |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| 'use client'; | ||
|
|
||
| import { useAuiState } from '@assistant-ui/react'; | ||
| import { INCOMPLETE_RESPONSE_DATA_NAME } from '@truefoundry/trueforge-assistant-ui-runtime'; | ||
|
|
||
| import { useSlot } from '../theme/SlotsProvider.js'; | ||
|
|
||
| export function IncompleteResponseContainer() { | ||
| const IncompleteResponseBanner = useSlot('IncompleteResponseBanner'); | ||
| const part = useAuiState(s => s.part); | ||
|
|
||
| if (part.type !== 'data' || part.name !== INCOMPLETE_RESPONSE_DATA_NAME) { | ||
| return null; | ||
| } | ||
|
|
||
| return <IncompleteResponseBanner />; | ||
| } |


There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Empty start events bypass failure guard
Medium Severity
The empty-output guard treats a
reasoning-startortool-input-startas usable output, even when no text, reasoning, or tool arguments arrived. A stream that dies right after those start events is kept asincompleteinstead of failing, so the user can get a blank answer plus warning, orAgentThreadcan continue and invoke a tool with empty or truncated arguments.Additional Locations (1)
packages/trueforge-core/src/core/runtime/AgentThread.ts#L1388-L1390Reviewed by Cursor Bugbot for commit 94ef0c8. Configure here.