VulRunWG measurement notes — published September 24, 2026; updated October 4, 2026 Hardware: NVIDIA RTX 5080, 16 GB VRAM, 32 GB PC memory. Windows, Vulkan/WGSL through wgpu. These figures summarize separate development experiments. They are not a universal performance claim, an independent audit, or an apples-to-apples competitor comparison. September 26 Perplexity against llama.cpp: 15 GGUF files from 11 architectures, llama.cpp b11146 llama-perplexity and VulRunWG on the same file and the same text, 512-token windows, about 1,500 scored tokens per model. All within 0.8%. Perplexity agreement shows the arithmetic matches closely on that text; it does not show identical outputs or equal task accuracy. Granite rows are scored on llama.cpp's token sequence. Each supported quantization type was also compared on distance from the model's Q8_0 build: within 3.5% of llama.cpp. Evidence: results/perf_20260926/ppl_vs_llama/*.json, format_vs_ref/*.json Mixture-of-experts decode, single conversation, RTX 5080, 32 GB PC memory: Qwen3 30B-A3B Q3_K_S, every expert on the card (context limited to about 256 tokens by the 90% video-memory rule): 142.7 tokens/s (llama.cpp b11146 Vulkan, same file: 193). Same model, automatic expert cache (107 of 128 expert slots per layer on the card, about 10,500 tokens of context): 24.3 tokens/s. Experts not on the card are copied from the memory-mapped model file in PC memory when a token needs them (measured upload 5.4-7.1 GB/s); 96% of expert lookups hit the card cache (hit rate measured with the earlier kernels; routing is unchanged). This is a memory setting on the 5080, not a measurement of a smaller card. Granite 3.1 3B-A800M Q4_K_M, all experts on the card: 333 tokens/s. Long prompts on expert models: about 10 s for 4,000 tokens (Qwen3 30B-A3B). Evidence: commit 911905f; docs/designs/MOE_OFFLOAD_PLAN_20260926.md Speech-to-text: 73 recorded clips, 1,150 words, word error rate after Whisper's English text normaliser; base.en 8.4%, small.en 5.1%, small 6.1%, large-v3-turbo 3.3%. Evidence: results/perf_20260926/whisper_wer/wer_i8.json Text-to-speech, Kokoro-82M (NumPy on the processor, 4 threads): 8 sentences x 3 voices, Whisper small.en transcribed 22 of 24 back word for word; about 2.1x real time. Evidence: results/kokoro_20260926/roundtrip.json Against llama.cpp, re-measured September 28 (RTX 5080, driver 616.56; the same Qwen2.5 7B Q4_K_M file and the same prompt ids as the September 23 comparison). VulRunWG: fast_coop, 128 greedy tokens, median of three after a warm-up, first four token times excluded. llama.cpp: build 11146 Vulkan, llama-bench -p 55,728 -n 128 -ngl 99 -r 3 (no server, which favours it). llama.cpp VulRunWG decode, 55-token prompt 159.0 tok/s 150.4 tok/s decode, 728-token prompt (same run) 141.6 tok/s prompt, 55 tokens 18.6 ms 86.2 ms prompt, 728 tokens 114.4 ms 262.6 ms Decode is within 6-11%. Prompt reading is 2.3x slower; the September 23 figure was 7.9x. VulRunWG's GPU code runs through WebGPU (wgpu), which does not expose the cards' matrix units; Vulkan offers them through an extension (VK_KHR_cooperative_matrix). The two sets of figures come from different harnesses (llama-bench against VulRunWG's own generator); a single harness driving both would be tighter. Evidence: results/perf_20260928/vs_llamacpp/README.md Linux, September 28-29. One laptop (MSI Katana A15 AI), Linux Mint 22.3, kernel 7.0.0-34, Mesa 25.2.8: an RTX 4070 Laptop, 8 GB, on the open NVK driver (NVIDIA's own driver not installed) and a Radeon 780M, 4 GB, on RADV. Code at 884879f plus three fixes made for this run. GPU chosen by Vulkan ICD. Nothing here was measured on Windows and nothing here says anything about Windows. Decode, profile auto (resolves to fast_coop on both), prompt "The capital of France is", 64 greedy tokens after a 4-token warm-up, three fresh processes a row, median: 4070 Laptop (NVK) 780M (RADV) Qwen2.5 0.5B Q4_K_M 92.7 tok/s 61.8 tok/s Qwen2.5 1.5B Q4_K_M 79.1 8.8 (27.6 after the September 28 fix) Qwen2.5 3B Q4_K_M 47.3 3.0 (13.9 after the September 28 fix) Qwen2.5 7B Q4_K_M 24.0 refused (needs ~5,871 MiB of 4,096) Every run of each model produced the same 64 tokens on both GPUs. 0 GPU faults. The 780M's Q4_K figures above are from before the fix of September 28. Cause: 84% of decode time went to two one-thread feed-forward kernels. The router picks a cooperative kernel on a 32-wide-subgroup GPU; the 780M is 64-wide, so it fell back to the older ones. The runtime now measures both routes on the loaded model and caches the faster per GPU, token-identical in every A/B run. 1.5B 9.0 -> 27.6 tok/s on the script that matches the Windows run; 3B 3.0 -> 13.9. The 0.5B was never affected: its feed-forward weights are Q5_0, not Q4_K. Fixed and verified on Linux; the Windows 780M is expected to pick the same route, not yet checked. Windows 11 against Linux on the same laptop, same script (tools/verify_20260924_on_this_card.py), same models, 4070 Laptop, after installing NVIDIA's own Linux driver 595.91.07. Qwen2.5 3B Q4_K_M: Windows 11 Linux (NVIDIA 595.91.07) prompt, 55 tokens 180.5 ms 203.8 ms prompt, 728 tokens 778.5 ms 805.8 ms prompt, 2,146 tokens 2,552.2 ms 2,432.3 ms decode, 128 tokens 60.1 tok/s 84.2 tok/s 4 streams 162.8 tok/s 233.9 tok/s 16 streams 499.9 tok/s 463.7 tok/s Prompts within -5% to +13%, decode +40%, 4 streams +44%, 16 streams -7%. On NVK instead (the open driver, no NVIDIA driver installed) the same 3B decodes at 46.8 tok/s and reads prompts 8-10x slower; NVK also needs the DP4A kernel disabled, and it could not be re-measured after the NVIDIA driver was installed. The decode figures above are with fix 4 (a cached-skip prefill bug found on this run); the runs taken before it are kept in the evidence folder. Surfaces, through the prototype shell on the 4070 with NVIDIA 595.91.07: chat (four models), embeddings, text-to-speech (SpeechT5 and Kokoro), speech-to-text, SD 1.5 and SDXL pictures. 12 of 12. Single first calls including model load, not benchmarks. Radeon 780M, the website's own settings: SD 1.5, 512 px, 20 steps, int8. September 29, after turning on the shared-memory convolution and shared attention for AMD (both were on for NVIDIA only, and both are byte-identical on this chip): 78.4 s, against a 368.7 s same-day baseline with them off, 4.7x. The RTX 5080 takes 8.3 s. Timed renders sync every dispatch and are an upper bound. The laptop ran about 35% slower on September 29 than on September 28 for reasons not found, which is why the comparison uses a same-day baseline rather than the 272.5 s figure measured on September 28 with both switches off. Speech to text, whisper-tiny.en: 64.4 s of speech in 3.0 s, 1.4% word errors. Read-aloud, Kokoro on the processor: 1.7-2.0x real time; SpeechT5 on the 780M is slower than it plays, 0.4-0.8x. Evidence: results/linux_20260928/ (README.md, app_smoke, amd780m_images, amd780m_voice) Earlier notes follow. Correction, September 25 evening: the generator rounded each token's time to 1 ms, so the earlier September 25 decode figures further below (96.3, 142.9, 76.9 ...) are rounded. Per-token times are now kept to the microsecond; the figures in this section supersede them. Single-conversation decode, September 25 evening: Qwen2.5 7B/14B Q4_K_M, fast_coop profile (int8 activations, cooperative attention from the first token, attention reading four cached keys per step). 16 varied prompts, 128 greedy tokens each, four fresh interleaved processes: 7B 153.9 tokens/s, 14B 77.5. Both pass the conformance gate in this profile. The 60-question assistant evaluation below covered the int8 change only. Decode as the conversation grows, 7B, 128 greedy tokens after N cached tokens of one document: 128: 153.5 2,048: 137.5 4,096: 134.9 8,192: 122.9 tokens/s (74.0 before the attention change; the 14B at 8,192 is refused by the context guard on 16 GB). Other families, same 16x128 method, before the attention change: Llama 3.1 8B 129.8, Mistral 7B 148.0, Qwen3 8B 130.6, Qwen3 4B 205.1, Phi-3 mini 208.5, Phi-3.5 mini 198.7, Gemma 3 4B 180.5, Gemma 2 2B 258.4, SmolLM2 1.7B 387.1, Llama 3.2 1B 505.4 tokens/s. Evidence: results/perf_20260925/EVENING_RESULTS.md, speed_precise/, long_ctx/, attn_coop4/ Earlier on September 25 (rounded figures): fast_coop with int8 activations and cooperative attention from the first token. 16 varied prompts, 128 greedy tokens each, four fresh interleaved processes per change: int8 decode: 7B 87.9 -> 96.3 tokens/s, 14B 46.9 -> 51.9 (6/16 and 12/16 identical) attention from first token: 7B 96.3 -> 142.9 tokens/s, 14B 51.8 -> 76.9 (9/16 and 9/16 identical) Both models pass the conformance gate in fast_coop after each change. Evidence: results/perf_20260925/coop_from_start/ 60-question assistant run, same code, int8 off/on: 7B right answers 30/31 of 48, 14B 31/32. Evidence: results/perf_20260925/decode_int8/ September 24 (reordered kernels, before int8): 87.9 / 46.3 tokens/s. 28/32 prompt outputs token-identical to the prior arithmetic; changed reduction order can produce different wording. stable_exact retains its prior token behavior. Source: docs/work_orders/DECODE_REORDER_PROTOTYPE_20260924.md Evidence: results/perf_20260924/decode_reorder/ The subsequent 60-question assistant run retained the previous category scores; both model/workflow quality gates still fail. Faster inference is not better accuracy. Native multi-stream decode: Qwen2.5 7B/14B Q4_K_M, 16 streams combined. 417.6 / 199.8 tokens/s. Four fresh processes normal/candidate/candidate/normal; 16 real prompts, 24-step timed runs three times per process, medians of six. 8,880 checked token positions per model identical. Not single-user or HTTP speed. Source: docs/work_orders/ROWS_UNROLLED_20260924.md Evidence: results/perf_20260924/unroll_scan/ Images: SDXL base, int8 weights, 512x512, 20 steps, warm render (loading excluded). 9.10 seconds, median of process medians. Six fresh processes in the convolution tile comparison; 24 renders and latents byte-identical. Previous tile: 9.71 seconds. GPU memory reported by the work ledger: 7,088–7,110 MiB. Source: docs/work_orders/IMAGE_CONV_TILE_20260924.md Evidence: results/perf_20260924/image_conv_n64/ Parity is not an image-quality certification; the ledger records pre-existing 512px quality issues. Different checkpoints, precision and step counts differ. Conformance, September 25: 15 checkpoints in 8 families (Qwen2.5 0.5B/1.5B/7B/14B, Qwen3 4B/8B, Llama 3.2 1B / 3.1 8B, Mistral 7B, SmolLM2 1.7B, Phi-3 mini, Phi-3.5 mini, Gemma 2 2B, Gemma 3 1B/4B) pass on the RTX 5080 in both profiles: tokenizer and chat template, next-token distributions against a float64 NumPy forward of the same weights and against llama.cpp, and 48-token greedy answers. Source: conformance/README.md, LEDGER.jsonl. Hardware scope: fast-path results above are measured on RTX 5080. RTX 4070 Laptop, September 25: the September 24 speed changes measured, prompt 2.29x/2.64x/2.32x (Qwen2.5 3B, 55/728/2,146 tokens), SDXL Lightning 6.7 -> 4.5 s, tokens and image bytes identical. Source: results/verify_20260924/LAPTOP_RESULTS_20260925.md. Game scope: Rules for Thee's builds carry VulRunWG dust, stamp and verdict effects, on by default; The Far Off Up's particles, wind and flocking are experimental. VulRunWG computes effects; their game frameworks present them. Rules for Thee's cleaning duty uses CPU physics by default. No live language model is required. Shared-card model experiments do not establish stutter-free concurrent gameplay. The named evidence files are in the development repository. Request the technical write-up and underlying measurement records: lessprepinfo@gmail.com. Consolidated specification: docs/SNRT_SPEC_AND_BENCHMARKS_20260924.md Time to first token: Qwen2.5 Q4_K_M, fast_coop, f16 K/V. Request to first selected token; four fresh processes per comparison, interleaved arms, medians of six. Prompt tokens 7B 14B 55 83 ms 151 ms (7B re-measured September 28: 86.2 ms) 728 285 ms 583 ms (7B re-measured September 28: 262.6 ms) 4,255 2,174 ms 4,661 ms 6,364 not listed 8,724 ms 12,700 10,648 ms not listed Superseded by the September 28 re-measurement above; kept as recorded. Historical competitor context (NOT a new head-to-head): llama.cpp Vulkan b11146, September 23, same RTX 5080 and Qwen2.5 7B model files: about 148–152 tokens/s decode and 98 ms for a 728-token prompt; 14B about 76.5-78.1 tokens/s decode. VulRunWG, September 24-25: 7B 153.9 tokens/s and 285 ms; 14B 77.5 tokens/s. Different dates/campaigns; a fresh matched test is needed for a precise current comparison. Source: consolidated specification section 3 and results/llama_comparison_20260923. No causal attribution of the gap is established here. Compatibility: GGUF Q4_K, Q5_K, Q6_K, Q5_0 and Q8_0. MiniLM / BERT GPU embeddings are off by default. The VRAM governor uses NVIDIA NVML. AMD 780M uses the 64-wide language paths. (September 28, Linux: SDXL-Lightning at 768 px ran on the 4070 Laptop; SDXL is still not tested on AMD.) September 26, evening (RTX 5080, 16 GB; PC with 32 GB memory) - Load Qwen2.5 7B and get the first token: 5.77 s (GGUF open, tokenizer, shader compile, first 128-token prompt). - 16 conversations at once, Qwen2.5 7B, one process: 677.5 tokens/s in total (14B: 378.7). - Qwen3 30B-A3B Q3_K_S, fast profile: 4,096-token prompt 7.6 s (was 10.7 s), 720 tokens 1.2 s; same tokens as before. - First SD 1.5 picture from a cold start with the saved converted weights: 11.9 s (20.4 s without); identical pictures. - gpt-oss-20b MXFP4 (preview, SN_GPT_OSS=1): all 32 experts per layer on the card; fast profile 128-130 tokens/s, exact profile 34.5 tokens/s; exact profile gives the same first 32 tokens as the reference on 3 of 3 prompts. - Download sizes (GitHub release files, checked September 26): Ollama 0.34.4 OllamaSetup.exe 1,571 MB; ComfyUI portable 0.37.0 NVIDIA 1,925 MB; faster-whisper CUDA 12 libraries (Purfview CUDA12_v3) 849 MB. VulRunWG package zip: 55 MB (38.4 MB from October 3). September 27 (RTX 5080, 16 GB; PC with 32 GB memory) - Qwen2.5 7B, start of the program to the first reply: 4.3 s with the file cached, 4.9 s on a first run (was 7.0 / 9.6 s): two copies of every weight on its way to the card removed, the model header read once. - If the graphics device is lost (tested by destroying it), VulRunWG reloads the model: the same answer again in 0.8 s (Qwen2.5 0.5B). - gpt-oss-20b on by default. A code planted in the first sentence of a 5,746-token and an 11,052-token document is found in 5.1 s and 13.7 s (were 22.7 s and 70.6 s): prompt attention with four queries per subgroup (fast profile) and grouped expert kernels byte-identical to the previous ones. - Qwen3 30B-A3B Q3_K_S, fast profile, 4,234-token prompt: 8.0 s (9.6 s without the grouped down step), same first token; experts 107 of 128 per layer on the card. October 2 (RTX 5080, 16 GB, driver 616.56; and an RTX 4070 Laptop, 8 GB) - Prompt reading, Qwen2.5 7B Q4_K_M, fast profile, the same prompt ids as the September 28 comparison, median of three after a warm-up: 728 tokens 245 ms (was 263); 55 tokens 82 ms (was 86). llama.cpp's September 28 figures stand at 114 and 18.6 ms: the long-prompt gap is 2.1x. - Graphics memory a conversation needs, measured instead of estimated: peak card memory sampled ten times a second while prompts of 1,024 to 24,576 tokens were read, Qwen2.5 0.5B / 1.5B / 7B / 14B. The limit on how long a conversation may be is now priced from those measurements, per model shape. Qwen2.5 7B on the 5080: 32,215 tokens (was 24,104). Evidence: results/ctx_price_20261002. - Same measurement on the 4070 Laptop, Qwen2.5 7B: the price sits above every sampled reading; the model holds 4,570 tokens of conversation there. Evidence: results/context_peak_laptop_20261002. - Scratch memory on discrete cards is rounded to 12.5% steps instead of powers of two (integrated GPUs keep powers of two). Qwen2.5 7B's fixed prompt-reading memory: 1,943 MiB to 1,430 MiB; an 8,192-token prompt reads 4-6% slower. Evidence: results/prefill_scratch_20261002. - 8-bit conversation cache (K/V), on by itself when a card would otherwise hold under 16,384 tokens, off in the exact profile. Memory for the cache: 1.78x smaller. Qwen2.5 7B decode on the 5080: 0.94x of the 16-bit speed at 64, 2,048 and 8,192 tokens of context; 8,192-token prompt 4.1 s against 3.8 s. Answers: Qwen2.5 7B 2 of 4 fixed prompts and Qwen2.5 1.5B 5 of 8 identical token for token to 16-bit, the rest reworded; arithmetic, sorting and a 4,096-token continuation identical on both. Evidence: results/kv_q8_20261002. - Decode, Qwen2.5 7B, fast profile: 392 GPU dispatches a token, 62 of 64 tokens replayed from one recording. Evidence: results/decode_dispatch_20261002. October 2, later (RTX 5080, 16 GB, driver 616.56) Serving figures are client-side: streamed chat requests against a fresh local server (127.0.0.1), Qwen2.5 7B Q4_K_M, fast profile, 128-token replies, eight requests sent at the same moment, interleaved repeat runs. - Shared setting (SN_SCENARIO=shared, or --scenario shared for the helper process): continuous batching over eight conversation slots on a paged conversation cache. Eight users: 346-351 tokens/s in total, first word median 0.30 s, slowest 0.60-0.61 s; four users 256-260 tokens/s. One-person setting, same eight requests one after another: 130 tokens/s, slowest first word 4.9 s. Card memory at peak: 9.1 GB shared, 7.3 GB one-person. Evidence: results/serving_20261002 (7b_scenario, 7b_attn_split). - A ~3,100-token document arriving while three chats stream: the others' longest pause 0.13-0.14 s (was 1.17 s); the document is read a few layers at a time between their steps, with the same reply as reading it in one go. Once the document's reply is running, the others' words arrive every 14 ms (was 86 ms). Evidence: results/batch_pieces_20261002, results/batch_attn_split_20261002. - Mixed traffic, eight at once (four plain, three at temperature 0.8, one offering a tool): 2.3 s shared against 4.2 s one after another; the tool call parsed the same both ways. DeepSeek-R1-Distill-Qwen-7B, 1,024-token replies: 11.0 s against 28.3 s, no hidden reasoning in any reply. Evidence: results/batch_sampled_tools_20261002. - A lone request in the shared setting runs the one-user decode: same reply as the one-person setting. - Short prompts read on the stored weights in 32-row pieces: an 18-token prompt plus first token 70.6 -> 30.7 ms (7B), 81.4 -> 22.5 ms (3B). Evidence: results/short_prompt_20261002. - Graphics memory held after a reply, 2,048-token chat: 0.5B 1.7 -> 0.9 GB, 7B 6.1 -> 5.8 GB (the int8 prompt weight cache now holds one layer; a model whose output layer is its word table keeps one copy). Evidence: results/floor_20261002, results/i8tc_cache_20261002. October 3 (RTX 5080, 16 GB, driver 616.56; RTX 4070 Laptop, 8 GB; Radeon 780M) - Prompts of more than 48 tokens are read on the card's int8 matrix units (tensor cores): kernels written in GLSL with GL_KHR_cooperative_matrix, loaded as SPIR-V through a wgpu-native 29.0.1.1 built with two patches (build script and patch included; the DLL is byte-identical from build to build). One scale per 32 values, the rounding llama.cpp's own int8 path uses. Used only when the driver offers int8 cooperative matrices with 32-wide subgroups and a self-test product comes out exact; otherwise the previous path. Off with SN_I8_COOP=0. - Qwen2.5 7B Q4_K_M, fast profile, the September 28 prompt ids, median of three after a warm-up: 728 tokens 195 ms (242 ms with SN_I8_COOP=0); 55 tokens 88 ms (82 ms). llama.cpp's September 28 figures: 114 and 18.6 ms. The long-prompt gap is 1.7x (was 2.1x). - 1,000-token prompt: RTX 5080 334 -> 263 ms (1.27x); RTX 4070 Laptop 1.70 -> 0.99 s (1.7x). Radeon 780M: refused by the self-test (64-wide subgroups), previous path kept. - Replies against the exact profile, 6 models x 4 long prompts, 128 greedy tokens: the matrix-unit path and the previous path agree with the exact profile equally (longer agreement 4 vs 3 cases, 17 equal); 14 of 24 replies identical between the two paths. Evidence: results/i8_coop_20261003. - Package: Python 3.13, wgpu-py 0.32; zip 38.4 MB. October 4 (RTX 5080, 16 GB, driver 616.56) - The matrix-unit path now starts at 96 prompt tokens (SN_I8_COOP_MIN_ROWS): below that it was no faster than the previous path (49-80 tokens within 1 ms either way; 96 tokens 88 ms against 94 ms; 128 tokens 88 ms against 131 ms). A 55-token prompt is read the previous way, 82 ms. Evidence: results/i8_coop_20261003/crossover_5080.json, crossover_5080_after.json. - Prompt reading, Qwen2.5 7B, matrix units on: 2,000 tokens 530 ms (648 ms on the previous path). Evidence: results/i8_coop_20261003/README.md. - The engine without Python or its dependencies (kernels, code, graphics bridge, picture word list, English pronunciation dictionaries): 8.9 MB zipped, 20.9 MB unpacked; 204 compute shaders in the shipped engine files. November 14 (front page additions; figures measured earlier, listed with their own dates) - The same 30B expert model held to an 8 GB card's budget: Qwen3-30B-A3B Q3_K_S, RTX 5080 with the expert pool forced to 35 of 128 slots, 5.9 GB on the card, 18.5 tokens/s after the round-trip fix, answers correct in every row. Measured September 26. Evidence: docs/designs/MOE_OFFLOAD_PLAN_20260926.md ("Built and measured"). - Radeon 780M chat: Qwen2.5 1.5B Q4_K_M, 29.6 tokens/s decode over 128 tokens, Windows 11, AMD driver 31.0.22038.0, commit bb29100. Measured September 30. Evidence: results/windows_samecommit_20260930. - Certified model files (all five conformance levels, stable_exact and fast_coop): Qwen2.5 0.5B, 1.5B, 7B, 14B; Qwen3 4B, 8B; Llama-3.2 1B, Llama-3.1 8B; Mistral-7B-Instruct v0.3; Phi-3 mini 4k, Phi-3.5 mini; Gemma-2 2B, Gemma-3 1B and 4B; SmolLM2 1.7B. All Q4_K_M or the publisher's q4, RTX 5080, September 25. Evidence: conformance/LEDGER.jsonl, conformance/README.md. - Independence: runtime_sdk/tools/prove_independence.py (llama_cpp.Llama replaced with a function that raises; load and generate succeed and it never fires) and runtime_sdk/tools/dependency_audit.py (live import graph, no torch, onnx, onnxruntime or llama_cpp).