Skip to content

docs(evidence): #400 dictionary-ref PoC — KILL, archived evidence - #413

Open
PhysShell wants to merge 1 commit into
mainfrom
research/dictionary-ref-poc
Open

PhysShell wants to merge 1 commit into
mainfrom
research/dictionary-ref-poc

Conversation

@PhysShell

Copy link
Copy Markdown
Owner

Summary

This PR archives the bounded Roslyn PoC for the #400 dictionary-ref pilot. Its result is
KILL-DICTIONARY-REF-PILOT. Owner ruling: ACCEPT KILL / ARCHIVE AS SEPARATE RESEARCH PR, outcome
STOP-PRODUCTION / KEEP-EVIDENCE.

The pilot was authorised by the decision GO-400-DICTIONARY-PILOT. That decision and the research report are in
#412.

The PR adds evidence only, under docs/evidence/dictionary-ref-poc-2026-10/. It does not change production code,
OwnIR, CI or goldens, and no implementation PR follows from it.

Result

  • Targets. Both buggy revisions warn at the right lines:

    • Meziantou DictionaryExtensions.cs:31 and :56;
    • MAUI BindableObject.cs:793, through CreateContext → defaultValueCreator(this).

    Both fixed revisions are clean. For MAUI this is trivial, because the fix removes the API from that method.

  • Controls. The mandatory negative controls are silent: no mutation, a different dictionary, a safe factory,
    and a ref used only before the mutation.

  • Precision gate (≥ 80 %): FAIL, as measured, with no retroactive adjustment:

    Round Probe Warnings TP / FP / UNCLEAR Precision Conservative
    1 v1 18 9 / 5 / 4 64 % 50 %
    2 v2 14 8 / 3 / 3 73 % 57 %
  • Severity of the 17 sample TPs. 0 were proven at runtime. 3 are contract violations without observable effect;
    14 are plausible but not demonstrated. Runtime-proven defects exist only for the two targets (docs(evidence): next real-bug class selection — #400 dictionary-ref pilot (#382/#406 NO-GO) #412).

  • Third-party analyzers: NOT_TESTED. The package download was not authorised; this is not a MISS.

  • Not authorised. The post-hoc ref = factory(…) observation is recorded as not evidence. No third round is
    authorised.

Contents

  • REPORT.md, which covers the gates, precision, the effect split, the limitations and how to reproduce.
  • PRECISION-PREREG.md, verbatim.
  • Frozen probes probe-v1/ and probe-v2/, checked by SHA256SUMS.
  • controls/.
  • data/:
    • sample lists pinned by blob SHA;
    • every warning with its label and reason;
    • outputs for the targets and the controls.
  • scripts/ for search, index derivation, fetch, blob check and the full re-run.

No third-party source is committed: samples/ and targets/ are fetched by SHA and gitignored.

Verification

  • scripts/verify_blobs.sh: r1 429/429 and r2 252/252 files match their blob SHA.
  • scripts/make_index.py --check: both sample indexes are re-derived byte-identically.
  • scripts/run.sh regenerates data/ and prints the two tally lines above.
  • ruff check, ruff format --check and mypy --strict pass on scripts/.

🤖 Generated with Claude Code

Owner ruling: ACCEPT KILL / ARCHIVE AS SEPARATE RESEARCH PR
(STOP-PRODUCTION / KEEP-EVIDENCE). The bounded Roslyn PoC found both
real stale-ref defects (Meziantou, MAUI), kept both fixed revisions and
the mandatory controls silent, but failed the pre-registered >= 80 %
precision gate on two independent samples: v1 9/5/4 (64 %), v2 8/3/3
(73 %).

Evidence only: frozen probes v1/v2, controls, sample lists pinned by
blob SHA, every warning with its label, target/control outputs and the
scripts to re-run them. No third-party source, no production, OwnIR,
CI or golden change. A third round is not authorised.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant