Skip to content

Read chat recipes, images included, on the prompt_logprobs rung - #12

Merged
fidecastro merged 1 commit into
mainfrom
feat/prompt-logprobs-messages
Nov 6, 2026
Merged

fidecastro merged 1 commit into
mainfrom
feat/prompt-logprobs-messages

Conversation

@fidecastro

Copy link
Copy Markdown
Owner

Why

TensorFold's native server refuses reply-token logprobs for GLM-5.3-Flash and serves prompt_logprobs only on /v1/completions. So the prompt_logprobs rung (0.2.0) read raw prompts only, and a raw prompt can't carry an image: GLM was text-only in jevify. TensorFold's /v1/completions also takes a chat messages list, image parts included (zig/src/server/openai.zig:140), and serves prompt_logprobs on it. GLM's prefill encodes the images and records the prompt rows in the same pass (families/glm/cuda_lanes.zig:223-231).

What

  • Rung, messages mode: the readout goes to /v1/completions with messages, and each answer spelling is appended as the assistant's reply (or to the recipe's prefill).
  • Answer row: the template adds its generation prompt after the reply, so the answer's row isn't the last. The probe sends the reply with two different answers, requires the rows to differ in exactly one position holding each answer, and records probe.prompt_rows_after_answer. Searching from the end for the answer token was rejected, because it can hit the same word in the question.
  • Loud failure: every readout checks that the row holds the appended answer. A template that renders differently later (thinking switched) fails with "re-run the probe" instead of reading the wrong position.
  • One route: when the strongest proven rung reads prompt rows, the warm request and the probe's cache, fan-out and image checks use the same route. So the image is proven where questions send it, and the chat route never gets the reply-logprobs request the server refuses.
  • Docs: README, playbook, SKILL.md and an ADR-0002 amendment. The old test asserting that a chat recipe is refused on this rung is replaced by the new behaviour's tests.

Evidence

CI selection: uv run pytest -m "not live and not encoder" is green; ruff check and ruff format --check are clean. That includes 6 new tests against the fake server (now able to render TensorFold's completions-with-messages route), and the request-fields guard now drives that lane too.

Live run, by hand (not a scored suite):

  • Setup: TensorFold native f350cd3e + MiaAI GLM-5.3-Flash-DGX-Sparks-1104 @ 02dac38 (108 engine patches), Ablit weights @ 57edefd, two DGX Sparks, VISION=1, thinking off.
  • Probe: a chat recipe proved prompt_logprobs, proved the image modality on /v1/completions, and recorded prompt_rows_after_answer: 3 (<|assistant|><think></think>, as the server's source predicts).
  • Images: jevify ask with a 256×256 solid-colour PNG:
state P(dominant colour is red / blue) P("mostly red") prompt tokens
red image 0.9998 / 0.0001 0.9999 159
blue image 0.0006 / 0.9993 0.010 159
no image 0.12 / 0.43 0.65 57
  • Speed: about 0.35 s per question at 256×256; a 1280×720 image is 1,251 prompt tokens and 1.6 s per question. The server skips its prefix cache for prompt_logprobs, so the image is re-encoded for every question.
  • Regression: the existing raw text recipe still reads on the same server.

Not proven

No evaluation suite has been scored through this path. Servers other than TensorFold's native one are untested in messages mode; vLLM takes no messages on /v1/completions, so the rung stays unproven there for a chat recipe, as before.

🤖 Generated with Claude Code

TensorFold's native server refuses reply-token logprobs for GLM-5.3-Flash
and serves prompt_logprobs only on /v1/completions, so the rung read raw
prompts only, and a raw prompt carries no image. That server's
/v1/completions also takes a chat `messages` list, image parts included,
and serves prompt_logprobs on it.

In messages mode the rung now sends the readout there, with each answer
spelling as the assistant's reply. The template appends its generation
prompt after it, so the probe measures the answer's row (two answers, rows
must differ in exactly one position holding each) and records the rows
after it as probe.prompt_rows_after_answer. Every readout checks that row
holds the appended answer and names a re-probe when it does not.

When the strongest proven rung reads prompt rows, the warm request and the
probe's cache, fan-out and image checks take the same route, so the image
is proven where questions send it and the chat route never receives the
reply-logprobs request the server refuses. ADR-0002 amended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fidecastro
fidecastro merged commit 2ae2cd2 into main Nov 6, 2026
5 of 6 checks passed
@fidecastro fidecastro mentioned this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant