Skip to content

Local AI on NVIDIA and AMD, without CUDA.
38 MB.

A complete local AI runtime: chat, pictures, speech, document search and GPU compute from one engine, written from scratch. It runs through Vulkan, with no CUDA, ROCm or llama.cpp underneath.

38 MB

The whole runtime: chat, pictures, speech and search. Unzip and run, Python included.

Ollama, ComfyUI and the CUDA speech libraries come to about 4.3 GB together.

NVIDIA and AMD

One engine, one build. Vulkan, not CUDA or ROCm.

The same download runs on an RTX 5080 and on a laptop’s built-in Radeon.

Its own engine

Not a llama.cpp wrapper. About 200 GPU kernels written for it, from model loading to tensor-core prompt reading.

Qwen, Llama, Mistral, Phi, Gemma, Granite, SmolLM, DeepSeek and more, from 0.5B to 30B.

Free to use

At home, at school or at work. No account, no subscription.

Putting it inside something you sell needs a commercial licence.

Measured and written up by its developer. Page updated 14 November 2026.

What it does

One engine, nine jobs.

Most local setups need a separate program for each of these, each with its own copy of CUDA. Here they share one engine and one download.

Chat

Models from 0.5B to 30B, including mixture-of-experts. Answers on your machine, offline.

Document search

Point it at a folder and ask. Answers come back with the passage they came from.

Embeddings

Turn text into search vectors for your own retrieval, locally and for free.

Vision

Image understanding, so a model can answer questions about a picture you give it.

Picture generation

SD 1.5 and SDXL, made by your own graphics card.

Speech to text

Whisper, from dictation to long recordings. A minute of audio in about three seconds.

Reading aloud

Natural voices, faster than the audio plays. English.

GPU compute for games

Particles, flocking, bouncing debris, balloon deformation, lighting and scenery rendering on the local GPU.

A local API

Code written for the OpenAI API works here. Change the address, keep the code.

In games

What VulRunWG computes during play.

In The Far Off Up, the game supplies the rules and painted artwork. VulRunWG runs its graphics and visual simulation calculations locally on the GPU, through the same engine used for its AI models.

Current gameplay in The Far Off Up: Ruby and Volly approach Jupiter among layered stars and floating ledges
The Far Off Up, captured from the current build with VulRunWG effects enabled. See the game
Particles and flocks
Updates drifting dust, rain, sparks and flock movement, including how nearby characters and the balloon disturb the particles.
Debris and balloon shape
Calculates tumbling fragments and their bounces against ledges. Springs and internal pressure make Volly dent, wobble and settle after a hit.
Lighting
Calculates local light and occlusion from scene sources and effects, then combines the result with the painted world.
Scenery and camera
Scales and fades painted planets and terrain, and projects the scene for the camera and widescreen view.

Speed

Fast where it counts.

Same card, same model file as llama.cpp: text generation within 6% of it, prompt reading within 1.7×. Qwen3 4B reads a 4,200-token document, about 3,000 words, in 1.3 s.

Chat, 7B model
llama.cpp reaches 159 on the same card and file
150 tokens/s
Reading a long prompt
llama.cpp reads it in 114 ms on the same card and file
195 ms for 728 tokens
Answer starts after a short question
under 0.1 s
A 30B expert model on a 16 GB card
only the parts it needs sit on the card
143 tokens/s
The same 30B model held to an 8 GB card’s budget
5.9 GB on the card; the rest waits in PC memory until it is needed
about 18 tokens/s
Eight users at once, 7B model
in total, through the local server, with the shared setting
about 350 tokens/s
A minute of audio to text
about 3 s
A 512-pixel picture
SD 1.5, model already loaded
8.3 s
Reading text aloud
2× real time

Full tables, with the method behind each figure, are further down and in the measurement notes.

Size and setup

Unzip it and run it.

There is no installer and no toolkit to fetch first. It carries its own Python and talks to the graphics card through Vulkan, which your driver already provides.

Download size, models not included
Download size, models not includedmeasured
VulRunWG: chat, pictures, speech to text, read-aloudabout 38 MB
Ollama 0.34.4 for Windows: chat1,571 MB
ComfyUI portable 0.37.0 for NVIDIA: pictures1,925 MB
CUDA libraries for faster-whisper: speech to text849 MB
Those three togetherabout 4.3 GB

None of these includes a model. The CUDA-based ones each bring their own copy of NVIDIA’s libraries. They are full programs with their own features; this compares download size only.

Hardware

From a laptop with no graphics card upwards.

A laptop with no graphics card
AMD Radeon 780M, 4 GB carved out of system memory

Chat, speech, document search and SD 1.5 pictures all run. A 1.5B model chats at about 30 tokens/s, and a 728-token prompt reads in about 2.2 seconds.

A gaming laptop
NVIDIA RTX 4070 Laptop, 8 GB

Chat up to 7B, speech, SD 1.5 and SDXL pictures. A 7B model holds about 4,500 tokens of conversation; smaller models hold more. A 1,000-token prompt on a 7B model reads in 1.0 s.

A desktop card
NVIDIA RTX 5070, 12 GB

Chat and pictures, including SDXL-based models.

A big desktop card
NVIDIA RTX 5080, 16 GB

14B models, a 30B mixture-of-experts model, and every speed figure on this page.

Who it is for

Where it fits.

At home

Useful without a subscription or an account.

  • Ask questions about your own documents and letters
  • Dictate notes and have them read back
  • Make pictures without paying per image

Schools, clinics and nonprofits

One machine, no per-seat accounts, nothing sent off site.

  • Student and client information stays on the machine
  • Works with the network unplugged
  • No sign-in, so no age or account barrier
  • Runs on a laptop with built-in graphics

For developers

A local API your existing code already knows how to talk to.

  • OpenAI-compatible chat, embeddings, speech and pictures
  • Python, JavaScript and C# clients included
  • Runs as a helper process inside your own program
  • One start-up setting serves several users or apps at once

For game developers

One runtime for the model and the graphics work.

  • Run your own graphics and simulation shaders on its GPU device
  • Use particles, lighting and visual physics alongside local models
  • Game effects already run in The Far Off Up and Rules for Thee

For organisations with privacy rules (HIPAA, FERPA and others). VulRunWG keeps information on your own computer, so there is no outside company holding your records. It is software, not a compliance program: how that computer is secured and how information on it is handled are still up to your organisation’s policies.

For developers

Point your OpenAI code at it.

A local server that speaks the OpenAI API. Your existing code works against it unchanged: chat, streaming, embeddings, pictures, speech and transcription.

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:7331/v1",
    api_key="not-needed",
)

reply = client.chat.completions.create(
    model="qwen2.5-7b-instruct-q4_k_m",
    messages=[{"role": "user",
               "content": "Name three rivers in Europe."}],
)
print(reply.choices[0].message.content)

Clients included

Python, JavaScript and C#, each able to start the runtime as a child process of your own program and shut it down with it.

No silent substitutions

A setting this build accepts but does not apply is named in the reply. A model or voice it cannot provide is refused by name, never swapped for another.

Handles several at once

Started with the shared setting, up to eight users or apps use the card together: about 350 tokens/s in total on a 7B model, each answer starting in about 0.3 s. Creative, tool and long-document requests join like any other, and the health check reports which setting is on.

How it’s built

Under the hood.

  • Its own GPU engine, written from scratch: WGSL shaders on wgpu, over Vulkan. No CUDA, no ROCm, no PyTorch, no llama.cpp.
  • On NVIDIA, prompts are read on the tensor cores in int8, with hand-written cooperative-matrix kernels.
  • Five dependencies: wgpu, numpy, gguf, jinja2 and regex. None of them does the model maths.
  • The engine itself, its GPU kernels and the code around them, is about 9 MB to download.
  • That is why it is 38 MB instead of gigabytes, and why the same engine can do a model and a particle system.
  • Answers are checked against exact arithmetic and against llama.cpp, model by model.
  • If the graphics driver resets, it reloads the model and carries on instead of closing.
  • Nothing leaves your PC except model downloads, the speed test and network name lookups, each only when you start it.

Checkable

Its own engine, and how we know.

“Written from scratch” is a claim anyone can make. These are the tests behind it.

The trap test

llama.cpp is rigged to crash the moment anything touches it. Then a model loads and answers. The answers come out right and the trap never fires.

The import audit

A check walks everything the running engine loads and must find no PyTorch, ONNX Runtime or llama.cpp. They appear only in the test tools, as answer keys.

The conformance gate

A model family is listed only after its tokens match the publisher’s tokenizer and its answers match exact float64 arithmetic from the same weights, with llama.cpp as a second, independent check. A deliberately broken model is run as a control and must fail.

Certified model files
FamilyCertified model files
Qwen 2.50.5B, 1.5B, 7B, 14B
Qwen 34B, 8B
Llama 3.x3.2 1B, 3.1 8B
Mistral7B Instruct v0.3
PhiPhi-3 mini, Phi-3.5 mini
GemmaGemma 2 2B, Gemma 3 1B and 4B
SmolLM21.7B

15 model files from seven families, Q4_K_M, certified on an RTX 5080. Other models listed above run; these are the ones that have passed every level of the gate. Each run is a row in an append-only ledger.

Measurements

Every number, model by model.

Chat and prompt speed model by model, speech-to-text error rates, picture times, expert models, and perplexity against llama.cpp. Plus what it runs, what you need and which operating systems were tested.

Measurements and specifications

Limits

What it doesn’t do.

  • Reading a long prompt is about 1.7× slower than llama.cpp (728 tokens: 195 ms against 114 ms). Generating is level with it.
  • Tensor-core prompt reading needs an NVIDIA card whose driver offers it; other graphics read prompts the slower way.
  • AMD Radeon graphics cards (RX series) are not measured yet; only the Radeon 780M built into a laptop processor is.
  • Linux is measured on one laptop only. Other Linux systems, and AMD’s own Linux driver, are not tested.
  • Built-in graphics is much slower than a card: a 512-pixel picture takes about 80 s on a Radeon 780M against 8 s on an RTX 5080.
  • A model too big for your card or PC memory is refused with a message. Other programs using the card can still make a job fail.
  • Speech to text and reading aloud are English only.
  • Small models can be wrong. Answers from your documents name their source; check anything that matters before acting on it.
  • Models are not included, and each keeps its publisher’s licence.
  • Not public yet.

Technical write-up

Want the detail?

Interface, supported models, memory controls and measurement methods. Free on request.

Measurement notes: dates, settings and methods
Request the technical write-up