Repository navigation
Read chat recipes, images included, on the prompt_logprobs rung - #12
Merged
Merged
Conversation
TensorFold's native server refuses reply-token logprobs for GLM-5.3-Flash and serves prompt_logprobs only on /v1/completions, so the rung read raw prompts only, and a raw prompt carries no image. That server's /v1/completions also takes a chat `messages` list, image parts included, and serves prompt_logprobs on it. In messages mode the rung now sends the readout there, with each answer spelling as the assistant's reply. The template appends its generation prompt after it, so the probe measures the answer's row (two answers, rows must differ in exactly one position holding each) and records the rows after it as probe.prompt_rows_after_answer. Every readout checks that row holds the appended answer and names a re-probe when it does not. When the strongest proven rung reads prompt rows, the warm request and the probe's cache, fan-out and image checks take the same route, so the image is proven where questions send it and the chat route never receives the reply-logprobs request the server refuses. ADR-0002 amended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
TensorFold's native server refuses reply-token logprobs for GLM-5.3-Flash and serves
prompt_logprobsonly on/v1/completions. So theprompt_logprobsrung (0.2.0) read raw prompts only, and a raw prompt can't carry an image: GLM was text-only in jevify. TensorFold's/v1/completionsalso takes a chatmessageslist, image parts included (zig/src/server/openai.zig:140), and servesprompt_logprobson it. GLM's prefill encodes the images and records the prompt rows in the same pass (families/glm/cuda_lanes.zig:223-231).What
/v1/completionswithmessages, and each answer spelling is appended as the assistant's reply (or to the recipe's prefill).probe.prompt_rows_after_answer. Searching from the end for the answer token was rejected, because it can hit the same word in the question.Evidence
CI selection:
uv run pytest -m "not live and not encoder"is green;ruff checkandruff format --checkare clean. That includes 6 new tests against the fake server (now able to render TensorFold's completions-with-messages route), and the request-fields guard now drives that lane too.Live run, by hand (not a scored suite):
f350cd3e+ MiaAI GLM-5.3-Flash-DGX-Sparks-1104 @02dac38(108 engine patches), Ablit weights @57edefd, two DGX Sparks,VISION=1, thinking off.prompt_logprobs, proved theimagemodality on/v1/completions, and recordedprompt_rows_after_answer: 3(<|assistant|><think></think>, as the server's source predicts).jevify askwith a 256×256 solid-colour PNG:prompt_logprobs, so the image is re-encoded for every question.Not proven
No evaluation suite has been scored through this path. Servers other than TensorFold's native one are untested in messages mode; vLLM takes no
messageson/v1/completions, so the rung stays unproven there for a chat recipe, as before.🤖 Generated with Claude Code