Local AI on NVIDIA and AMD, without CUDA.
38 MB.
A complete local AI runtime: chat, pictures, speech, document search and GPU compute from one engine, written from scratch. It runs through Vulkan, with no CUDA, ROCm or llama.cpp underneath.
The whole runtime: chat, pictures, speech and search. Unzip and run, Python included.
Ollama, ComfyUI and the CUDA speech libraries come to about 4.3 GB together.
One engine, one build. Vulkan, not CUDA or ROCm.
The same download runs on an RTX 5080 and on a laptop’s built-in Radeon.
Not a llama.cpp wrapper. About 200 GPU kernels written for it, from model loading to tensor-core prompt reading.
Qwen, Llama, Mistral, Phi, Gemma, Granite, SmolLM, DeepSeek and more, from 0.5B to 30B.
At home, at school or at work. No account, no subscription.
Putting it inside something you sell needs a commercial licence.
Measured and written up by its developer. Page updated 14 November 2026.
What it does
One engine, nine jobs.
Most local setups need a separate program for each of these, each with its own copy of CUDA. Here they share one engine and one download.
Chat
Models from 0.5B to 30B, including mixture-of-experts. Answers on your machine, offline.
Document search
Point it at a folder and ask. Answers come back with the passage they came from.
Embeddings
Turn text into search vectors for your own retrieval, locally and for free.
Vision
Image understanding, so a model can answer questions about a picture you give it.
Picture generation
SD 1.5 and SDXL, made by your own graphics card.
Speech to text
Whisper, from dictation to long recordings. A minute of audio in about three seconds.
Reading aloud
Natural voices, faster than the audio plays. English.
GPU compute for games
Particles, flocking, bouncing debris, balloon deformation, lighting and scenery rendering on the local GPU.
A local API
Code written for the OpenAI API works here. Change the address, keep the code.
In games
What VulRunWG computes during play.
In The Far Off Up, the game supplies the rules and painted artwork. VulRunWG runs its graphics and visual simulation calculations locally on the GPU, through the same engine used for its AI models.

- Particles and flocks
- Updates drifting dust, rain, sparks and flock movement, including how nearby characters and the balloon disturb the particles.
- Debris and balloon shape
- Calculates tumbling fragments and their bounces against ledges. Springs and internal pressure make Volly dent, wobble and settle after a hit.
- Lighting
- Calculates local light and occlusion from scene sources and effects, then combines the result with the painted world.
- Scenery and camera
- Scales and fades painted planets and terrain, and projects the scene for the camera and widescreen view.
Speed
Fast where it counts.
Same card, same model file as llama.cpp: text generation within 6% of it, prompt reading within 1.7×. Qwen3 4B reads a 4,200-token document, about 3,000 words, in 1.3 s.
Full tables, with the method behind each figure, are further down and in the measurement notes.
Size and setup
Unzip it and run it.
There is no installer and no toolkit to fetch first. It carries its own Python and talks to the graphics card through Vulkan, which your driver already provides.
| Download size, models not included | measured |
|---|---|
| VulRunWG: chat, pictures, speech to text, read-aloud | about 38 MB |
| Ollama 0.34.4 for Windows: chat | 1,571 MB |
| ComfyUI portable 0.37.0 for NVIDIA: pictures | 1,925 MB |
| CUDA libraries for faster-whisper: speech to text | 849 MB |
| Those three together | about 4.3 GB |
None of these includes a model. The CUDA-based ones each bring their own copy of NVIDIA’s libraries. They are full programs with their own features; this compares download size only.
Hardware
From a laptop with no graphics card upwards.
Chat, speech, document search and SD 1.5 pictures all run. A 1.5B model chats at about 30 tokens/s, and a 728-token prompt reads in about 2.2 seconds.
Chat up to 7B, speech, SD 1.5 and SDXL pictures. A 7B model holds about 4,500 tokens of conversation; smaller models hold more. A 1,000-token prompt on a 7B model reads in 1.0 s.
Chat and pictures, including SDXL-based models.
14B models, a 30B mixture-of-experts model, and every speed figure on this page.
Who it is for
Where it fits.
At home
Useful without a subscription or an account.
- Ask questions about your own documents and letters
- Dictate notes and have them read back
- Make pictures without paying per image
Schools, clinics and nonprofits
One machine, no per-seat accounts, nothing sent off site.
- Student and client information stays on the machine
- Works with the network unplugged
- No sign-in, so no age or account barrier
- Runs on a laptop with built-in graphics
For developers
A local API your existing code already knows how to talk to.
- OpenAI-compatible chat, embeddings, speech and pictures
- Python, JavaScript and C# clients included
- Runs as a helper process inside your own program
- One start-up setting serves several users or apps at once
For game developers
One runtime for the model and the graphics work.
- Run your own graphics and simulation shaders on its GPU device
- Use particles, lighting and visual physics alongside local models
- Game effects already run in The Far Off Up and Rules for Thee
For organisations with privacy rules (HIPAA, FERPA and others). VulRunWG keeps information on your own computer, so there is no outside company holding your records. It is software, not a compliance program: how that computer is secured and how information on it is handled are still up to your organisation’s policies.
For developers
Point your OpenAI code at it.
A local server that speaks the OpenAI API. Your existing code works against it unchanged: chat, streaming, embeddings, pictures, speech and transcription.
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:7331/v1",
api_key="not-needed",
)
reply = client.chat.completions.create(
model="qwen2.5-7b-instruct-q4_k_m",
messages=[{"role": "user",
"content": "Name three rivers in Europe."}],
)
print(reply.choices[0].message.content)Clients included
Python, JavaScript and C#, each able to start the runtime as a child process of your own program and shut it down with it.
No silent substitutions
A setting this build accepts but does not apply is named in the reply. A model or voice it cannot provide is refused by name, never swapped for another.
Handles several at once
Started with the shared setting, up to eight users or apps use the card together: about 350 tokens/s in total on a 7B model, each answer starting in about 0.3 s. Creative, tool and long-document requests join like any other, and the health check reports which setting is on.
How it’s built
Under the hood.
- Its own GPU engine, written from scratch: WGSL shaders on wgpu, over Vulkan. No CUDA, no ROCm, no PyTorch, no llama.cpp.
- On NVIDIA, prompts are read on the tensor cores in int8, with hand-written cooperative-matrix kernels.
- Five dependencies: wgpu, numpy, gguf, jinja2 and regex. None of them does the model maths.
- The engine itself, its GPU kernels and the code around them, is about 9 MB to download.
- That is why it is 38 MB instead of gigabytes, and why the same engine can do a model and a particle system.
- Answers are checked against exact arithmetic and against llama.cpp, model by model.
- If the graphics driver resets, it reloads the model and carries on instead of closing.
- Nothing leaves your PC except model downloads, the speed test and network name lookups, each only when you start it.
Checkable
Its own engine, and how we know.
“Written from scratch” is a claim anyone can make. These are the tests behind it.
The trap test
llama.cpp is rigged to crash the moment anything touches it. Then a model loads and answers. The answers come out right and the trap never fires.
The import audit
A check walks everything the running engine loads and must find no PyTorch, ONNX Runtime or llama.cpp. They appear only in the test tools, as answer keys.
The conformance gate
A model family is listed only after its tokens match the publisher’s tokenizer and its answers match exact float64 arithmetic from the same weights, with llama.cpp as a second, independent check. A deliberately broken model is run as a control and must fail.
| Family | Certified model files |
|---|---|
| Qwen 2.5 | 0.5B, 1.5B, 7B, 14B |
| Qwen 3 | 4B, 8B |
| Llama 3.x | 3.2 1B, 3.1 8B |
| Mistral | 7B Instruct v0.3 |
| Phi | Phi-3 mini, Phi-3.5 mini |
| Gemma | Gemma 2 2B, Gemma 3 1B and 4B |
| SmolLM2 | 1.7B |
15 model files from seven families, Q4_K_M, certified on an RTX 5080. Other models listed above run; these are the ones that have passed every level of the gate. Each run is a row in an append-only ledger.
Measurements
Every number, model by model.
Chat and prompt speed model by model, speech-to-text error rates, picture times, expert models, and perplexity against llama.cpp. Plus what it runs, what you need and which operating systems were tested.
Measurements and specificationsLimits
What it doesn’t do.
- Reading a long prompt is about 1.7× slower than llama.cpp (728 tokens: 195 ms against 114 ms). Generating is level with it.
- Tensor-core prompt reading needs an NVIDIA card whose driver offers it; other graphics read prompts the slower way.
- AMD Radeon graphics cards (RX series) are not measured yet; only the Radeon 780M built into a laptop processor is.
- Linux is measured on one laptop only. Other Linux systems, and AMD’s own Linux driver, are not tested.
- Built-in graphics is much slower than a card: a 512-pixel picture takes about 80 s on a Radeon 780M against 8 s on an RTX 5080.
- A model too big for your card or PC memory is refused with a message. Other programs using the card can still make a job fail.
- Speech to text and reading aloud are English only.
- Small models can be wrong. Answers from your documents name their source; check anything that matters before acting on it.
- Models are not included, and each keeps its publisher’s licence.
- Not public yet.
Built on it
What already uses it.
Worksheets, visual supports and clinical drafts for teachers and therapists.
An editable worksheet canvas with local art generation and background removal.
Particles, flocking, debris, balloon deformation, lighting and scenery rendering on the shared GPU engine.
Dust, stamp and verdict effects in the game builds, on by default.
Technical write-up
Want the detail?
Interface, supported models, memory controls and measurement methods. Free on request.
Measurement notes: dates, settings and methods