Do you need an NPU or GPU for local AI dictation?

Search "gpu for whisper dictation" and almost everything you find is a server deployment guide for developers hosting a transcription API — nothing for someone wondering if their own PC can do it. Meanwhile, Microsoft's Copilot+ PC marketing has quietly convinced a lot of people that local AI needs a specific chip: a Neural Processing Unit rated at 40-plus TOPS, which most machines simply don't have.

Here's the direct answer: you don't need a Copilot+ PC, and you don't need an NPU. If your machine has a graphics card — NVIDIA, AMD, or Intel — from roughly the last eight years, it can very likely run Whisper, the speech-to-text model behind most local dictation tools, in real time. The NPU in a Copilot+ PC is built for a narrow set of Windows features. It isn't the chip that does this job, and most local AI software — including the whisper.cpp engine that underlies local Whisper apps — never touches it.

What does quantized Whisper actually need from a GPU?

Whisper large-v3-turbo is a distilled version of OpenAI's large-v3 model — same encoder, a decoder cut from 32 layers down to 4 — which is why it runs roughly 8x faster than the full model while keeping close to the same accuracy. In parameter terms that's about 809 million: a small model by 2026 standards, a fraction of the size of even a modest local chat LLM.

The full-precision build whisper.cpp runs needs on the order of a few gigabytes of VRAM. Quantized — the same int8/int5/int4 compression that shrinks the full large-v3 model from roughly 3.1 GB down to under 1 GB on disk — that footprint drops further still, comfortably inside what a mid-range gaming card has shipped with for most of the last decade. That's why the real constraint for real-time dictation almost never turns out to be VRAM capacity. It's whether the GPU has a working, modern-enough compute driver — and that's a Vulkan question, not a GPU-generation question.

Why Vulkan is the real story — NVIDIA, AND AMD, AND Intel

Most of what's written about GPU-accelerated Whisper assumes CUDA, because most of it is written for developers deploying on NVIDIA data-center cards. CUDA is NVIDIA-only — if you own an AMD or Intel GPU, a CUDA-focused guide is a dead end.

whisper.cpp — the C++ engine that Inkvox and most local Whisper apps build on — doesn't have that limitation. Its own documentation lists four separate acceleration paths: CUDA for NVIDIA, ROCm/HIP for AMD, OpenVINO for some Intel deployments, and Vulkan, described as the cross-vendor option that works across NVIDIA, AMD, and Intel GPUs through a single driver-level API. Vulkan is the one that matters for a general-purpose app — it runs on whatever GPU is already in the machine, with no vendor-specific toolkit to install.

This isn't theoretical. Developers building other Vulkan-accelerated whisper.cpp tools have reported close to 8x real-time transcription on a Radeon RX 9070 XT once Vulkan support was wired in — no NVIDIA card involved. Intel integrated graphics generally work the same way, through the same Vulkan driver path, though on older or lower-power Intel iGPUs, compute feature support can be inconsistent enough that the software falls back to CPU automatically rather than accelerating.

Inkvox uses this same Vulkan path. It's the reason the app runs on NVIDIA, AMD, and Intel GPUs from a single Windows build, with no separate download per vendor and no CUDA toolkit for you to install.

Do you need an NPU for local AI dictation?

No — and this is where Copilot+ PC marketing genuinely confuses the question. Microsoft's own developer documentation defines the Copilot+ PC tier around an NPU capable of 40-plus trillion operations per second (TOPS), paired with at least 16 GB of RAM. That NPU is real hardware doing real work — but it's scoped to a specific list of first-party Windows features: Recall, live-caption translation, Windows Studio Effects, and Click to Do. Those features run through Microsoft's own ONNX models routed to vendor-specific execution providers — Qualcomm's QNN, Intel's OpenVINO — not through any general-purpose path a third-party app can casually plug into.

That distinction matters because the mainstream tools that actually run local AI models on your behalf — Ollama, llama.cpp, LM Studio, and the whisper.cpp engine dictation apps build on — don't route to the NPU at all, as of mid-2026. They target the GPU, through Vulkan, ROCm, or Metal, or the CPU. Getting a model onto the NPU instead means hand-converting it to ONNX and compiling it for a vendor SDK — a specialist path, not something whisper.cpp or Ollama do by default.

So a Copilot+ PC's NPU rating tells you nothing about how fast it will dictate. What actually decides that is the GPU sitting next to the NPU — and on a lot of thin, fanless Copilot+ ultrabooks, that GPU is a modest integrated one, not a discrete card. A five-year-old gaming laptop with a mid-range discrete GPU will typically out-transcribe a brand-new Copilot+ PC on this specific job, because the number that matters here is GPU compute and driver support, not TOPS.

Indicative fit by hardware class

None of the following are lab benchmarks — they're an honest, qualitative map built from whisper.cpp's own documented acceleration paths and the one number we've measured ourselves.

• NVIDIA GPU, GTX 10-series or newer (RTX 20/30/40/50) — Vulkan or CUDA. Real time, comfortably: Inkvox on an RTX 3070 transcribes a sentence in ~0.3–0.4s, roughly 27x real time.
• AMD GPU, RX 500-series or newer (RX 6000/7000/9000) — Vulkan (ROCm/HIP also available). Real time on modern cards — developers have logged close to 8x real time on an RX 9070 XT with Vulkan-enabled whisper.cpp.
• Intel integrated graphics (recent Iris Xe / Arc) — Vulkan. Usable real time on most; a minority of older, lower-power Intel iGPUs lack full compute driver support and fall back to CPU.
• Copilot+ PC NPU alone, no capable discrete GPU — not used by whisper.cpp at all. Falls back to the machine's integrated GPU or CPU — depends on that chip, not the NPU's TOPS rating.
• No compatible GPU / older hardware — CPU fallback. Works. Slower than real time on long audio; fine for short dictation bursts.

No compatible GPU? The CPU fallback

Every whisper.cpp-based app, Inkvox included, falls back to a CPU path automatically when no usable GPU is present — nothing breaks, dictation just runs slower. It's the safety net, not the target experience: the entire reason Vulkan support exists is to skip the CPU path for the vast majority of machines that don't need it.

The question isn't "do I need a Copilot+ PC to dictate locally." It's "does my GPU support Vulkan" — and for almost anyone reading this on a machine bought in the last decade, the answer is yes.

If you're comparing offline dictation options on Windows more broadly, here's the fuller picture on local voice typing. If you're weighing Inkvox specifically against another local-first tool, see how it stacks up as a SuperWhisper alternative on Windows. And if you just want to see the speed for yourself, the numbers are on the homepage — or join the waitlist to try it on your own GPU, whatever's inside it.

Frequently asked questions

Can Whisper run on AMD GPUs?

Yes. whisper.cpp, the engine behind most local Whisper apps, supports AMD GPUs through Vulkan (cross-vendor, works out of the box) and through ROCm/HIP for deeper AMD-specific acceleration. Developers have reported close to 8x real-time transcription on a Radeon RX 9070 XT using Vulkan-enabled whisper.cpp. Inkvox uses the Vulkan path, so AMD GPUs are supported the same way NVIDIA GPUs are.

Do I need an NPU?

No. The NPU in a Copilot+ PC accelerates a specific set of Microsoft's own Windows features (Recall, live-caption translation, Windows Studio Effects) through ONNX models and vendor-specific execution providers. Local Whisper transcription runs through whisper.cpp on the GPU via Vulkan — a separate path that mainstream local-AI tools don't route to the NPU at all. A machine with no NPU dictates just as well as a 40-plus TOPS Copilot+ PC, provided it has a Vulkan-capable GPU.

What GPU do I need for real-time Whisper transcription?

Any Vulkan 1.3-capable GPU from NVIDIA, AMD, or Intel — which covers most graphics cards and integrated GPUs shipped in the last decade. Whisper's quantized large-v3-turbo model is small enough that VRAM capacity is rarely the limiting factor; a working, modern-enough Vulkan driver is what actually decides it.

Does a Copilot+ PC dictate faster than a regular gaming PC?

Usually not, for this specific task. Copilot+ PCs are built around battery life and NPU-only Windows features, and many ship with a modest integrated GPU rather than a discrete one. Since Whisper transcription runs on the GPU, not the NPU, a gaming PC or laptop with a mid-range discrete GPU will typically transcribe faster, regardless of Copilot+ certification.