Skip to content

fix(translation): let the agentic provider accept identity-only pending units - #1388

Merged
AnthonyTsu1984 merged 2 commits into
masterfrom
fix/agentic-unchanged-seeded-draft
Nov 19, 2026
Merged

AnthonyTsu1984 merged 2 commits into
masterfrom
fix/agentic-unchanged-seeded-draft

Conversation

@AnthonyTsu1984

Copy link
Copy Markdown
Collaborator

Problem

Translate runs 44526346422 and 44534975380 both translated — and failed — the exact same file in both locales:

content/en/reference/api/nodejs/nodejs/Authentication/Authentication-listPrivilegeGroups.md

Root cause chain (reproduced locally against the CI seed semantics):

  1. The en delta between the dev translation baseline a347ffbc and the Fetch checkpoint bd90c871 is a one-line link cleanup: **privileges** (*[PrivilegeEntity](PrivilegeEntity.md)[]*) - → **privileges** (*PrivilegeEntity[]*) -.
  2. Semantic seeds cover 39/40 units. The changed line is the only pending unit, classified new (English changed), so it is ineligible for the verified-current fast path and is handed to the agent.
  3. The pending unit is a pure identifier line whose correct zh/ja text is byte-identical to the English — the previously published translations of the old unit were themselves byte-identical English. The agent correctly leaves the seeded draft unchanged.
  4. runAgenticFile treats any unchanged seeded draft as a hard failure before running the deterministic gate, so the file fails.
  5. A failed file never publishes, its recorded source hash never advances, and the next hourly Fetch re-selects it — an endless loop (reason=current_delta each round, four paid no-op agent turns per cycle).

Five other files hit the same failure earlier the same day (Collections-addCollectionField(s), createCollection, describeCollection, getRefreshExternalCollectionProgress) and only escaped because upstream changed them again, downgrading them to stale_source full retranslations.

Fix

Keep the invariant that new pending units always reach the agent, but give the agent a controlled way to express "nothing to translate":

  • buildSeededTaskPrompt states an explicit identity-translation rule (identifier/type/code-only units must stay byte-identical) and the exact reply marker NO_TRANSLATION_NEEDED.
  • runAgenticFile accepts an unchanged seeded draft only when the agent replies with the marker and the unchanged bytes pass the same deterministic gate (validateWithRuntimeChecks) as any translated draft. Gate failures still flow into the bounded repair loop.
  • An unchanged draft without the marker keeps failing hard exactly as before, so a lazy no-op cannot slip untranslated prose through. Residual risk (a degraded model invoking the marker on translatable prose) is bounded by the two independent locale runs plus the deterministic gate and is documented in the code.

With the marker path, the loop closes mechanically: the file translates (unchangedDraftAccepted), publishes, the recorded hash advances, and the next selection no longer sees current_delta.

Tests

  • runAgenticFile accepts an unchanged seeded draft when the agent reports no translation needed — marker + unchanged draft → translated, published bytes equal the seeded draft, prompt states the rule and marker.
  • an unchanged seeded draft with the marker still goes through repair when the gate fails — marker + gate violation → bounded repair loop runs (attempts: ['translate', 'repair1']), no unchangedDraftAccepted.
  • The existing rejects a turn that leaves the seeded draft unchanged test is untouched and still green (no marker → throw).

Selector union for the changed paths (pnpm test:for-change -- scripts/translation/agenticProvider.js scripts/translation/agenticProvider.test.js):

  • pnpm test:translation — 92/92 green
  • focused trio (agenticProvider + validate-translation-file + surgicalUpdate) — 63/63 green
  • pnpm test:workflow-matrix, pnpm test:workflow-policy — green
  • pnpm test:replay:translation — 15/16 green; the single failure (inspect rejects the wrong child workflow identity before any retained download) is pre-existing on pristine master (verified by stashing this change): the retained fixture commits f3582b00/9fd45e90 are unreachable from origin/dev and are cruft-packed locally, so git 2.50's protocol-based clone cannot transfer them — environmental, not introduced here. CI is authoritative.
  • git diff --check — clean

promptContractSha256 only hashes .github/prompts/* and locale-contract files, so the inline prompt change does not affect recovery identity contracts.

replay-seed added 2 commits November 19, 2026 22:40
…ng units

A seeded draft can legitimately stay byte-identical after the translate
turn: the only pending 'new' unit may be a pure identifier line (for
example a parameter list row like **privileges** (*PrivilegeEntity[]*) -
whose correct zh/ja text is the English itself). The unchanged-draft
guard turned that correct no-op into a permanent failure, and since a
failed file keeps its old recorded source hash, every hourly Fetch
re-selected the same file and paid for the same no-op agent turn again
(observed on Authentication-listPrivilegeGroups across 44526346422 and
44534975380; five more files escaped only because upstream changed them
again to stale_source full retranslations).

The seeded prompt now states the identity-translation rule and lets the
agent claim it with a NO_TRANSLATION_NEEDED reply. The provider accepts
an unchanged draft only with that marker and only after the same
deterministic gate any translated draft must pass; gate failures still
flow into the bounded repair loop, and an unchanged draft without the
marker keeps failing exactly as before.
Review findings on #1388, all addressed:

- P2-1: the marker check used substring containment, so a reply that
  merely quoted the rule in prose qualified. Matching is now line-anchored
  (the marker must be a reply line of its own), and a new test pins that
  a prose mention of the marker still fails an unchanged draft.
- P2-2: the accept test's pending unit was the frontmatter title, which
  genuinely needs translation - the test proved the trust gap instead of
  the intended scenario. It now uses an identity fixture mirroring the
  production loop (identifier-only privileges row pending, translated
  title/heading seeded) and asserts the pending unit's source.
- P3-1: the batch summary line and the CI step summary now report an
  unchanged-drafts counter so the trust-based route stays observable.
@AnthonyTsu1984
AnthonyTsu1984 merged commit 1db72ac into master Nov 19, 2026
16 checks passed
@AnthonyTsu1984
AnthonyTsu1984 deleted the fix/agentic-unchanged-seeded-draft branch November 19, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant