Read chat recipes, images included, on the prompt_logprobs rung

README.md

@@ -59,9 +59,11 @@ that model and writes what it found into the file. vLLM and llama.cpp get
 their dialect extras; any other server is driven through the plain OpenAI
 surface. A server that refuses logprobs on the reply token but returns
 vLLM's `prompt_logprobs`, such as TensorFold's native server with
-GLM-5.3-Flash, is read through the `prompt_logprobs` rung, which needs a
-raw-mode recipe (the template written out with the model's own control
-tokens, served on `/v1/completions`).
+GLM-5.3-Flash, is read through the `prompt_logprobs` rung on
+`/v1/completions`: with a raw-mode recipe (the template written out with the
+model's own control tokens), or with a chat-template recipe where the server
+also takes `messages` on that route, as TensorFold's does. The chat form is
+the one that carries images.
 
 `serve` binds to `127.0.0.1`. To reach it from other machines pass
 `--host 0.0.0.0`, and set `JEVIFY_API_KEY` first so that every request must
@@ -257,12 +259,12 @@ dies before the armour, so the corridor needs a policy with memory, not
 better perception); the rerank kind and the
 embedding route against a live server; a marker-slot encoder layout; the
 grammar rung on the PrismML llama.cpp fork, which accepts a grammar without
-constraining the reported probabilities; the `prompt_logprobs` rung against a
-live server (written for TensorFold's native server with GLM-5.3-Flash, run
-only against the fake server in `tests/` so far; a server that skips its
-prefix cache for such requests, as TensorFold's source shows it does, makes
-every question read the whole state again, which the probe measures and
-notes); Windows. Choice menus on the endpoint kind are bounded by the
+constraining the reported probabilities; the `prompt_logprobs` rung on any
+server but TensorFold's native one with GLM-5.3-Flash, where it has been
+probed and asked by hand (raw and chat recipes, text and images) but no
+evaluation suite has been scored on it yet (that server skips its prefix
+cache for such requests, so every question reads the whole state again,
+images included, which the probe measures and notes); Windows. Choice menus on the endpoint kind are bounded by the
 single-token identifier alphabet the probe verifies (26 on the DeepSeek
 recipe); Jev's 255-option ceiling is not reachable there.
 

SKILL.md

@@ -81,7 +81,9 @@ headers, headers_env}`, `template` and `answers`.
   own control tokens; use it when the chat template would not put the model
   in its trained regime (the Qwen3-VL reranker recipe is the example), and
   for a server that refuses reply-token logprobs (TensorFold's native server
-  with GLM-5.3-Flash): only a raw prompt can carry the `prompt_logprobs` rung.
+  with GLM-5.3-Flash) when the state is text. The `prompt_logprobs` rung also
+  reads a messages-mode recipe where the server takes `messages` on
+  `/v1/completions` (TensorFold's does); that is the form that carries images.
 - `template.questions.choice_strategy`: `identifiers` (one request, options
   labelled A, B, C) or `per_option` (one yes/no request per option; needed
   for a model that never emits letters, such as a reranker).
@@ -90,7 +92,8 @@ headers, headers_env}`, `template` and `answers`.
 - `readout.rung: auto` walks the rungs the probe proved, strongest first:
   `named_token_logprobs` (vLLM with `logprob_token_ids`), `grammar`
   (llama.cpp), `top_k`, `prompt_logprobs` (each answer spelling appended to
-  a raw prompt and read back from vLLM-style `prompt_logprobs` rows, one
+  a raw prompt, or sent as the assistant's reply in messages mode, and read
+  back from vLLM-style `prompt_logprobs` rows, one
   request more for every spelling the rows did not show; exact, off-menu mass
   visible), `equal_bias`, `top_k_floor` (degraded, missing labels floored and
   listed). A rung that cannot see every label hands the
@@ -167,9 +170,13 @@ TypeSafe's published thresholds refer to. Everything jevify adds is under
   user turn right after the state. Use `template.mode: raw` so the warm prompt
   ends at the state boundary; the Bonsai recipe is the example.
 - **Probe notes "reply-token logprobs refused"**: the server gives no
-  logprobs on a generated token (TensorFold with GLM-5.3-Flash). Write a raw
-  recipe so the probe can prove `prompt_logprobs`; with a chat recipe the
-  probe ends in "no readout is possible".
+  logprobs on a generated token (TensorFold with GLM-5.3-Flash). The probe
+  then tries `prompt_logprobs`: a raw recipe on any such server, a chat
+  recipe where the server takes `messages` on `/v1/completions`; elsewhere a
+  chat recipe ends in "no readout is possible".
+- **"is not at the prompt row the probe measured"**: in messages mode the
+  server's template renders the answer turn differently than when the probe
+  ran (thinking switched on or off, a new template); re-run the probe.
 - **"did not come back as its own prompt token"**: the appended answer fuses
   with the end of the raw template; end the template where the answer starts
   a token of its own (after a newline, or before a leading-space spelling).

docs/adr/0002-backend-port-and-adapter-kinds.md

@@ -158,6 +158,28 @@ that request twice and records when the repeat reuses no cached prefix,
 since TensorFold skips its prefix cache for any request with
 `prompt_logprobs` and each question then reads the whole state again.
 
+Amended 2026-11-06, for images on that server: a raw prompt carries no
+image, so the rung above left such a server text-only. TensorFold's
+`/v1/completions` also takes a chat `messages` list, image parts included,
+renders it through the model's template and serves `prompt_logprobs` on it
+(its GLM prefill encodes the images and records the prompt rows in one
+pass). In messages mode the rung therefore sends the readout there, with the
+answer spelling as the assistant's reply. The template always appends its
+generation prompt, so the answer's row is not the last: the probe sends the
+reply twice with two different answers, requires the rows to differ in
+exactly one position that holds each answer, and records the rows after it
+as `prompt_rows_after_answer`. A search for the answer's token from the end
+was rejected, because it can find the same word in the question. Each
+readout reads that row and refuses one that does not hold the appended
+answer, naming a re-probe: a template that renders differently later (thinking
+switched) fails loudly instead of being read at the wrong position. When the
+strongest proven rung reads prompt rows, the warm request and the probe's
+cache, fan-out and image measurements take the same route, so the image is
+proven where questions will send it and a server that refuses reply-token
+logprobs on the chat route never gets one. A server that takes no `messages`
+on `/v1/completions` (vLLM) leaves the rung unproven for a chat recipe, as
+before.
+
 ### D5 — Choice questions on decoders
 
 Options are labeled with single-token identifiers and the distribution is

docs/playbook.md

@@ -229,8 +229,10 @@ two ways:
   do; any recipe works;
 - for the prompt (vLLM's `prompt_logprobs` on `/v1/completions`), as
   TensorFold's native server does for GLM-5.3-Flash, which refuses the
-  first way; the recipe must use `template.mode: raw`, and the probe proves
-  the `prompt_logprobs` rung.
+  first way; the probe proves the `prompt_logprobs` rung. A raw-mode recipe
+  works on any such server. A chat-template recipe works where the server
+  also takes a chat `messages` list on `/v1/completions` (TensorFold's does),
+  and is the one that can send images there.
 
 Many hosted APIs and proxies do neither; for those the probe stops with
 `no readout is possible`.
@@ -288,7 +290,7 @@ calibrated.
 | You see | Cause | Do this |
 |---|---|---|
 | `missing required header 'x-client-id'` | The server needs a header the recipe does not send. | Add `endpoint.headers: {x-client-id: <your product id>}`. |
-| `no readout is possible on this backend: it returned no logprobs` | The server or proxy returns no log-probabilities for the reply token, and the recipe could not read them through `prompt_logprobs`. | If the probe notes `reply-token logprobs refused`, write a raw-mode recipe so it can prove `prompt_logprobs` (step 6). Otherwise use a server that returns them (vLLM, llama.cpp). |
+| `no readout is possible on this backend: it returned no logprobs` | The server or proxy returns no log-probabilities for the reply token, and the recipe could not read them through `prompt_logprobs`. | If the probe notes `reply-token logprobs refused`, write a raw-mode recipe so it can prove `prompt_logprobs` (step 6); a chat-template recipe proves it only where the server takes `messages` on `/v1/completions`. Otherwise use a server that returns them (vLLM, llama.cpp). |
 | `endpoint.headers names X-Api-Key, which looks like a credential` | A secret would be written into the recipe. | Move it to `endpoint.headers_env: {X-Api-Key: <VARIABLE>}` and export the variable. |
 | `endpoint.headers_env sends X-Api-Key from $GATEWAY_KEY, which is not set` | The variable the recipe names is not exported in this shell. | `export GATEWAY_KEY=...`, then run again. |
 | `degraded: true`, options listed in `missing` | The options were not among the model's most likely tokens. Often the model starts to reason instead of answering. | Turn reasoning off (step 6, item 5), then probe again. |

jevify/adapters/endpoint/adapter.py

@@ -25,7 +25,7 @@
     Rung,
     StateHandle,
 )
-from jevify.recipes.render import render_prefix, render_question
+from jevify.recipes.render import RenderedPart, render_prefix, render_question
 from jevify.recipes.schema import Recipe
 
 Clock = Callable[[], float]
@@ -92,8 +92,12 @@ async def warm(self, state: State) -> StateHandle:
         slots = tuple(range(min(self.concurrency, self.slots))) if self.slots else ()
         started = self.clock()
         cached: int | None = None
+        # the route the questions will take, so a server that refuses one never sees it
+        prompt_rows = self._reads_prompt_rows()
         for slot in slots or (None,):
-            path, body = base_request(self.recipe.model.name, prefix, "", self.dialect)
+            path, body = base_request(
+                self.recipe.model.name, prefix, "", self.dialect, prompt_rows=prompt_rows
+            )
             if slot is not None:
                 body["id_slot"] = slot
             post = self.client.post_root if path == "completion" else self.client.post_v1
@@ -130,6 +134,35 @@ def _ladder(self) -> list[Rung]:
         proven = Capabilities.from_dict(self.recipe.probe).rungs if self.recipe.probe else set()
         return [r for r in RUNG_RANK if r in proven and r in RUNGS]
 
+    def _reads_prompt_rows(self) -> bool:
+        """Whether the strongest rung reads prompt rows; False before a probe on auto."""
+        if self.recipe.readout.rung == "auto" and not self.recipe.probe:
+            return False
+        return RUNGS[self._resolve_rung()].reads_prompt_rows
+
+    def _readout_request(
+        self, handle: StateHandle, part: RenderedPart, impl: Any, slot: int | None, tally: _Tally
+    ) -> tuple[str, dict[str, Any], Send]:
+        """The one way a question's readout request is built: the rung's route, and the
+        sender that reads the answer's row where the rung reads prompt rows."""
+        path, body = base_request(
+            self.recipe.model.name,
+            handle.prefix,
+            part.text,
+            self.dialect,
+            prompt_rows=impl.reads_prompt_rows,
+        )
+        answer_row = -1
+        if impl.reads_prompt_rows and handle.prefix.mode == "messages":
+            after = self.capabilities.prompt_rows_after_answer if self.capabilities else None
+            if after is None:
+                raise BackendError(
+                    "the prompt_logprobs rung in messages mode reads the row the probe "
+                    "measured (probe.prompt_rows_after_answer); run `jevify probe <recipe>`"
+                )
+            answer_row = -1 - after
+        return path, body, self._sender(path, slot, tally, answer_row)
+
     def _resolve_rung(self) -> Rung:
         pinned = self.recipe.readout.rung
         proven = (
@@ -189,11 +222,9 @@ async def _permuted(
         raw_parts = []
         for order in orders:
             part = render_question(self.recipe, question, order=order).parts[0]
-            path, body = base_request(
-                self.recipe.model.name, handle.prefix, part.text, self.dialect
-            )
+            path, body, send = self._readout_request(handle, part, impl, slot, tally)
             parsed = await impl.collect(
-                self._sender(path, slot, tally),
+                send,
                 body,
                 part,
                 self.recipe.readout.top_k,
@@ -232,7 +263,7 @@ async def _permuted(
             raw={"parts": raw_parts},
         )
 
-    def _sender(self, path: str, slot: int | None, tally: _Tally) -> Send:
+    def _sender(self, path: str, slot: int | None, tally: _Tally, answer_row: int) -> Send:
         """The one way a readout request goes out: pinned to its slot, timed, counted."""
         post = self.client.post_root if path == "completion" else self.client.post_v1
 
@@ -243,7 +274,7 @@ async def send(body: dict[str, Any]) -> Parsed:
             response = await post(path, body)
             tally.latency_ms += (self.clock() - started) * 1000.0
             tally.calls += 1
-            return parse_response(path, response, self.dialect)
+            return parse_response(path, response, self.dialect, answer_row=answer_row)
 
         return send
 
@@ -263,11 +294,9 @@ async def _answer(
         cached_tokens: int | None = None
         raw_parts = []
         for part in rendered.parts:
-            path, body = base_request(
-                self.recipe.model.name, handle.prefix, part.text, self.dialect
-            )
+            path, body, send = self._readout_request(handle, part, impl, slot, tally)
             parsed = await impl.collect(
-                self._sender(path, slot, tally),
+                send,
                 body,
                 part,
                 self.recipe.readout.top_k,

jevify/adapters/endpoint/dialects.py

@@ -34,12 +34,16 @@ class Parsed:
 
 
 def base_request(
-    model: str, prefix: RenderedPrefix, suffix: str, dialect: Dialect
+    model: str, prefix: RenderedPrefix, suffix: str, dialect: Dialect, *, prompt_rows: bool = False
 ) -> tuple[str, dict[str, Any]]:
     """The readout request every rung starts from: one token, no sampling, logprobs on.
 
     Returns the route key and the body: "chat/completions" and "completions" live under
     /v1; "completion" is llama.cpp's native root route, used for raw multimodal input.
+
+    `prompt_rows`: the readout reads prompt rows, not the reply (the prompt_logprobs rung).
+    A messages-mode body then goes to /v1/completions, where TensorFold's native server takes
+    a chat `messages` list and serves `prompt_logprobs`, and asks for no reply logprobs.
     """
     if prefix.mode == "raw":
         if prefix.images:
@@ -87,13 +91,17 @@ def base_request(
         "stream": False,
         "logprobs": True,
     }
+    if prompt_rows:
+        del body["logprobs"]
     if prefix.prefill:
+        # on prompt rows the answer is appended to this message (the rung's shape)
         messages.append({"role": "assistant", "content": prefix.prefill})
-        body["continue_final_message"] = True
-        body["add_generation_prompt"] = False
+        if not prompt_rows:
+            body["continue_final_message"] = True
+            body["add_generation_prompt"] = False
     if prefix.kwargs and dialect is not Dialect.GENERIC:
         body["chat_template_kwargs"] = dict(prefix.kwargs)
-    return "chat/completions", body
+    return ("completions" if prompt_rows else "chat/completions"), body
 
 
 def _base64_payload(data_uri: str) -> str:
@@ -113,7 +121,11 @@ def grammar_for(texts: list[str]) -> str:
     return f"root ::= {escaped}"
 
 
-def parse_response(path: str, response: dict[str, Any], dialect: Dialect) -> Parsed:
+def parse_response(
+    path: str, response: dict[str, Any], dialect: Dialect, *, answer_row: int = -1
+) -> Parsed:
+    """`answer_row`: which prompt_logprobs row is the answer's, counted from the end (-1: the
+    last, a raw prompt; earlier where the template adds rows after an assistant reply)."""
     entries: list[Entry] = []
     if path == "completion":
         # llama.cpp native route: completion_probabilities plus top-level counters.
@@ -131,16 +143,10 @@ def parse_response(path: str, response: dict[str, Any], dialect: Dialect) -> Par
     usage = response.get("usage") or {}
     details = usage.get("prompt_tokens_details") or {}
     rows = choice.get("prompt_logprobs")
-    if isinstance(rows, list) and rows and isinstance(rows[-1], dict):
+    if isinstance(rows, list) and len(rows) >= -answer_row and isinstance(rows[answer_row], dict):
         # vLLM's prompt_logprobs (TensorFold writes the same): null, then one {id: {logprob,
-        # rank, decoded_token}} row a prompt token, the prompt's own token listed first. The
-        # last row is the position of the last prompt token.
-        row = [
-            Entry(
-                text=str(item.get("decoded_token", "")), logprob=float(item["logprob"]), id=int(key)
-            )
-            for key, item in rows[-1].items()
-        ]
+        # rank, decoded_token}} row a prompt token, the prompt's own token listed first.
+        row = _row_entries(rows[answer_row])
         return Parsed(
             entries=tuple(row),
             prompt_tokens=usage.get("prompt_tokens"),
@@ -170,6 +176,26 @@ def parse_response(path: str, response: dict[str, Any], dialect: Dialect) -> Par
     )
 
 
+def _row_entries(row: dict[str, Any]) -> list[Entry]:
+    return [
+        Entry(text=str(item.get("decoded_token", "")), logprob=float(item["logprob"]), id=int(key))
+        for key, item in row.items()
+    ]
+
+
+def prompt_tokens_held(response: dict[str, Any]) -> tuple[Entry | None, ...]:
+    """Each prompt_logprobs row's own token (its first entry), None for the first row: the
+    prompt as the server tokenized it. Empty when the response lists no prompt rows."""
+    rows = response["choices"][0].get("prompt_logprobs")
+    if not isinstance(rows, list):
+        return ()
+    held: list[Entry | None] = []
+    for row in rows:
+        entries = _row_entries(row) if isinstance(row, dict) and row else []
+        held.append(entries[0] if entries else None)
+    return tuple(held)
+
+
 def _entry_from(item: dict[str, Any], dialect: Dialect) -> Entry:
     text = str(item.get("token", ""))
     ident = item.get("id")