SOL ExecBench ROCm 4.0

SOL ExecBench ROCm evaluates GPU kernel candidates and derives formal Speed-of-Light bounds for the AMD Radeon RX 9060 XT. The distribution contains two sibling packages with a deliberately narrow boundary:

• sol_execbench owns problem/workload contracts, corpus generation, input generation, candidate correctness and timing, local score formulae, aggregation policy, security, and the CLI.
• solar owns only the three stages described in SOL-ExecBench §4.2: operator graph extraction, validated extended-einsum conversion, and SOL analysis.

The paper is the source of truth when imported implementations disagree: SOL-ExecBench.

Install

uv sync --all-groups

Only Linux x86-64 with Python 3.14 is supported. The locked stack is ROCm 10.1.0, PyTorch 2.14.0, torchvision 0.29.0a0, and ROCm Triton 3.8.0. Python 3.14.1 is excluded by the official torchvision wheel. Native installations select gfx1200 device wheels; Docker inherits the official GPU packages without reinstalling them.

Public problem corpus

The hardware-independent, text-backbone corpus is frozen at problems/LLM_CORE/releases/LLM_CORE_V2/manifest.yaml. It contains 36 production-derived Definitions and 36 frozen workload-generation rules across dense, attention, MoE, KV cache, long-context, and quantization profiles. It does not freeze concrete Workloads. Validate it or generate a target view from an isolated measurement of currently allocatable GPU memory:

uv run sol-execbench dataset corpus validate
uv run sol-execbench dataset corpus generate \
  --target-template gfx1200 \
  --device cuda:0 \
  --profile core \
  --output problems/local/LLM_CORE/gfx1200-measured

The bundled gfx1200 and gfx942 files declare capability, dtype, quantization, tensor, and IPC limits. A bounded allocation probe measures usable quota, which is floored to a frozen capacity class. Each Definition then generates exactly nine slots at one common scale or is skipped atomically. Exact free bytes and other audit-only hardware fields cannot change workloads inside one cohort. Leaderboard submission, ranking, and hosted scoring remain out of scope.

The legacy AKA-reviewed selection remains tracked at problems/AMD_AKA/manifest.yaml. Problems are derived from AMD AgentKernelArena (AKA, Apache-2.0) under the SOL-ExecBench paper §3 construction methodology: each problem's PyTorch reference is the AKA task's own correctness oracle (module_fn), lifted into a standalone def run(...). The manifest pins the exact AKA commit and records per-task file checksums so the generated problems are bound to that source revision. The current seed set covers matmul, softmax, norm, elementwise, and conv operations across FP32 / BF16 / FP16.

Authored definitions and workloads are committed under problems/AMD_AKA/<suite>/<name>/. Materialization detects the selected ROCm device, filters incompatible workloads with static target rules plus a trusted reference/harness probe, and writes an auditable target tree under problems/local/AMD_AKA/<gfx-target>/:

bash scripts/fetch_aka_source.sh
uv run sol-execbench dataset materialize
uv run sol-execbench dataset audit problems/local/AMD_AKA/gfx1200

dataset audit also verifies the local AKA clone is pinned to the manifest revision and that every problem's per-task checksums match (the problems-bound-to-commit binding). The public Definition contract requires a non-empty op_type; the AKA importer records it per problem.

Evaluate and analyze

Candidate evaluation remains an outer-project operation:

./scripts/run_docker.sh -- sol-execbench evaluate \
  problems/local/AMD_AKA/gfx1200/torch2hip/l1n1_square_matmul \
  --solution /sol-execbench/path/to/solution.json \
  --trace-output /outputs/matmul.trace.jsonl

The intended formal SOLAR target is RX 9060 XT gfx1200. This port's formal publication policy requires the pinned Orojenesis toolchain; that is an explicit ROCm release constraint, not a claim that the paper mandates this tool for every SOLAR use. The architecture audit is packaged and content-addressed, with its throttled resource measurements explicitly limited to instruction/runtime corroboration. The reproducibly built mapper digest is reviewed and present in the release allowlist, so the formal producer reports ready. The corpus scoring policy is available and the gfx1200 release bundle is published under RELEASE/. An executable whose digest is only self-declared by its local provenance manifest is still rejected:

uv run sol-execbench solar analyze \
  problems/local/AMD_AKA/gfx1200/torch2hip/l1n36_rmsnorm \
  --workload aka-l1n36_rmsnorm-w0 \
  --orojenesis-home /path/to/pinned/orojenesis \
  --output out/solar/norm_forward_bf16

The isolated worker publishes an atomic directory whose canonical top-level files are operator_graph.yaml, the selected IR graph, conversion-attestation.yaml, solar-analysis.yaml, and manifest.yaml. When tile-aware analysis runs, the directory also contains the recursively content-addressed Orojenesis evidence referenced by the manifest; successful mapper logs are not published. The worker never receives candidate runtimes or computes scores.

Before a full SOLAR release, run the mandatory uniform qualification chain:

uv run sol-execbench solar qualify-static out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification
uv run sol-execbench solar qualify-canary out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification
uv run sol-execbench solar qualify-full out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification

Full qualification derives its 43-problem / 163-workload denominator from the pinned manifest and writes a content-addressed readiness matrix. Formal release-build refuses to start without the complete content-bound gate chain. See large batch GPU qualification.

SOLAR exposes exactly two fixed paths through --backend: torchview_extended_einsum (the default) and make_fx_aten. Extraction and IR selection cannot be combined independently, a run never falls back to the other path, and a corpus audit or release build uses one path for its complete denominator. Both paths preserve exact source-input, output, tensor-metadata, and effect provenance; unsupported tracing, conversion, replay, or resource accounting fails closed.

Extended schema v6 is a native semantic IR, not serialized ATen. Its operations carry ordered operands, public attributes, effects, and bounded dynamic shapes; the default executor calls public PyTorch APIs independently of the ATen path.

Official score

The schema v7 corpus pins the rx9060xt-gfx1200-reference-v2 baseline identity, and its v2 release evidence is published. The published gfx1200 official SOL score is 0.497818327 over 43 problems and 163 workloads. Its candidate is the eager PyTorch reference scored against itself, so the score validates the scoring pipeline rather than any optimized kernel; see RELEASE/README.md. The fail-closed scorer accepts a publisher-authored release bundle whose SHA-256 references bind the corpus, baseline, candidate execution, and pinned SOLAR manifests:

uv run sol-execbench score official RELEASE/release-bundle.json

sol-execbench score status reports the repository-pinned policy and baseline. Raw caller-authored measurements, baselines, or SOLAR JSON are not scorer inputs. Bundle hashes provide artifact integrity; publisher authenticity comes from the repository or release channel that distributes the bundle.

The release workflow includes baseline release-build, baseline candidate-build, the three baseline qualify-* commands, baseline release-run, the three solar qualify-* commands, solar release-build, score build-statement, and score assemble-bundle. It has one baseline run and one candidate run; it does not require role signatures or an independent rerun. See the release and official-score workflow.

Correct candidates must satisfy T_b > T_SOL and T_k >= T_SOL; workloads are averaged within each problem and then across problems with equal weight.

Development

uv run --with ruff ruff check .
uv run ty check
uv run pytest tests/

GPU tests declare their ROCm and architecture prerequisites. Build the optional container with ./scripts/run_docker.sh --build.

GPU entry points require explicit GPU_LOCK_DIR and the full AMD-SMI GPU_LOCK_UUID. They use gpu-<UUID>.L1-ownership.lock for cross-project ownership and gpu-<UUID>.L2-execution.lock for serialized device work. The native owner retains L1 through child/container cleanup, clock restoration and its final receipt. Container workers borrow that owner and contend for L2; borrowing never transfers restoration responsibility. Old index-based lock variables are rejected. Switching versions requires a coordinated idle maintenance window; old and UUID locks must not be used concurrently.

Evaluation, marked GPU tests, ExecBench's SOLAR analysis/release bridge, standalone profiler collectors and the documented calibration scripts enter this native lifecycle. CPU analysis does not inspect GPU configuration or hardware. An optional solar analyze --device-stage-lock adds serialization; it does not replace either shared lock. Nested stages reuse their current thread's scope without relocking the same flock inode. Direct SOLAR library GPU requests must supply AnalysisExecutionPolicy.device_stage_context; controlled low-level execution helpers require an already active native scope.

SIGTERM/SIGINT drain supervised child process groups before unlocking. Device synchronization and cleanup remain inside the scope on exceptions. Unverified cleanup or restoration leaves native recovery evidence blocking a later owner. These guarantees do not make a scheduler mandatory or automatically recover from SIGKILL. See GPU configuration for private history, Docker mounts and coordinated migration.

See SOLAR boundary and scoring contract for the normative architecture. The recorded GPU engineering evidence is limited to RX 9060 XT gfx1200 on the locked ROCm 7.2 stack; see RDNA4 validation scope for the exact test, toolchain, self-hosted-runner, and publisher-release boundaries.

License

Apache-2.0. Imported SOLAR files retain their original SPDX attribution.