Skip to content

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

llama.cpp Vulkan - AMD RDNA4 16GB Tunning & Benchmarking

Vulkan GPU VRAM OS llama.cpp Qwen

Practical presets, receipts and hybrid recipes for 16GB AMD RDNA4 on Windows Vulkan.

Seed verified 2026-09-01: RX 9070 (gfx1201) - Qwen3.5-9B-UD-Q4_K_XL 131k q8 KV - b512/ub256 - 2205 tok/s prompt - 86 tok/s gen - 8.8–10GB VRAM


Why RDNA4?

16GB controls fit, bandwidth + backend controls speed. RX 9070 XT (640 GB/s) fits the same 1x-RDNA16 profiles as RX 6900 XT (512 GB/s) but needs its own lane (gfx1201 vs gfx1030). This repo is that lane: Windows Vulkan, 70–80 tok/s sustained.

Seed Baseline — RX 9070

Field Value
GPU AMD Radeon RX 9070 16GB, RDNA4 gfx1201
VRAM 16GB GDDR6, 256-bit, 640 GB/s
Host Ryzen 7 7800X3D 8-Core, 64GB DDR4
OS Windows 11 - Vulkan (RADV proprietary)
llama.cpp b10712 daef7b687 Vulkan ggml-vulkan.dll
Model Qwen3.5-9B-UD-Q4_K_XL.gguf 5.55GB, 8.95B, 32L, 262k train
Runtime 131072 ctx q8_0 KV, 512/ub256, lash_attn on,
easoning on, parallel=1
MCP 22 tools (fetch 6, filesystem 14, websearch 2) local-first
VRAM 8.8GB @65k q4 -> 10.0GB @131k q8
Perf 2205 tok/s prompt (2048), 86 tok/s gen (128) - 14.5s for 25k prompt @1764 tok/s

Full baseline: docs/rx-9070-baseline.md

Benchmark — Don't Guess, Sweep

at llama-bench -m "gguf\Qwen3.5-9B\Qwen3.5-9B-UD-Q4_K_XL.gguf" -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0 ^ -p 2048 -n 128 -b 512,1024,2048,4096 -ub 64,128,256,512 --output json > bench/receipts/q8-sweep.`json

llama-bench -m "..." -p 8192 -n 512 --output json > bench/receipts/q8-long.json `

Short sweep (2048, q8, Vulkan) — RX 9070:

batch / ubatch prompt tok/s gen tok/s Verdict
512 / 512 2205.5 86.6 fastest prompt, short-only
1024 / 512 2204.3 86.7 tied
512 / 256 2153.4 86.7 recommended for 131k
1024 / 256 2152.6 86.8 alt

Public recommendation needs non-empty/long receipt. q8-long.`json (8192/512) passed — no OOM, 5350B receipt. Don't promote 512/512 for 131k long-prefill based on short sweep alone.

Protocol: docs/benchmark-protocol.md - Raw receipts: ench/receipts/

Presets — Copy & Run

65k daily (8.8GB): at run-qwen3.5-9b-server-opencode.bat --ctx-size 65536 --cache-type-k q4_0 --cache-type-v q4_0 --batch-size 2048 --flash-attn on

131k heavy (10GB, hybrid): at run-qwen3.5-9b-server-opencode-131k-q8.bat --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --batch-size 512 --ubatch-size 256 --flash-attn on

Both: --n-gpu-layers all --jinja --reasoning on --tools all --mcp-servers-config mcp-servers.`json

Hybrid Workflow — Muse Spark + Qwen Local

Private, free, fast. No cloud billing for mechanical work.

User -> Muse Spark (gentle-orchestrator) -> SDD propose/spec/design -> Qwen 3.5-9B local (explore/tasks/apply) @ 70 tok/s -> Muse Spark verify/review -> archive

Opencode opencode.json: ``json "provider": { "llamacpp": { "baseURL": "http://localhost:8080/v1", "headers": {"Authorization": "Bearer anything"} }}, "agent": { "explore": "llamacpp/qwen3.5-9b", "general": "llamacpp/qwen3.5-9b", "sdd-explore": "llamacpp/qwen3.5-9b", "sdd-tasks": "llamacpp/qwen3.5-9b", "sdd-apply": "llamacpp/qwen3.5-9b" }

Heavy validation 2026-09-01: heavy-15min 62/62 tests in 6.24s, kvstore 17/17 in 2.18s, 49KB app, 15min @100% Compute_0.

Quick Start

at git clone https://github.com/albertgracia/llama.cpp-Vulkan-AMD-RDNA4-Tunning-Benchmarking.git cd llama.cpp-Vulkan-AMD-RDNA4-Tunning-Benchmarking REM put Qwen3.5-9B gguf in G:\llama.cpp\gguf\Qwen3.5-9B\ run-qwen3.5-9b-server-opencode-131k-q8.bat REM in another terminal cd bench && bench-q8-sweep.bat

Hardware Lanes

Lane Meaning
1x-rdna16 One 16GB Radeon (6900 XT gfx1030 or 9070 gfx1201) — filter by gpu_model
2x-rdna16 Two 16GB cards (not assumed)
other-radeon-16gb Outside first target set

RX 9070 is 1x-rdna16 + gfx1201. Don't collapse with gfx1030 until measured.

Contributing

PRs need a receipt: ench/receipts/.`json + docs/rx-9070-baseline.md update. Keep q8_0 KV as default for 16GB. Validate: python -m py_compile bench/.py.

License

MIT — see LICENSE


Basado en el repositorio de: https://github.com/5p00kyy/club-rdna16

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages