ENGINEERING TOOLS

Local LLM Hardware Selector

Estimate model weight memory and understand the overhead that a real local AI workload needs.

Planning model, not a compatibility test. Defaults describe an illustrative standard decoder architecture. Enter your model's actual KV dimensions. Hybrid, compressed-cache and sliding-window architectures can differ.

Open saved scenario ↑

Save the inputs to a file and reopen them later. Nothing is stored automatically.

01 / Machine & weight payload

Unified RAM or GPU VRAM budget (GiB)

Total parameters (billions, including all experts)

Weight bits per parameter

Weight metadata / runtime allowance (%)User assumption; actual overhead must be measured.

OS, apps & other runtime reserve (GiB)

02 / KV cache geometry

Attention layers

KV heads per layer

Head dimension

Tokens reserved per request

Concurrent requests

KV precision (bits)

Memory formulas and limits

Raw weights = total parameters × bits / 8, converted to GiB. KV cache = 2 × attention layers × KV heads × head dimension × context tokens × parallel requests × cache bytes. The factor 2 represents keys and values. Values are binary GiB; a vendor's decimal GB display may differ.

The allowance is a user assumption, not a universal overhead percentage. This does not estimate speed or guarantee that a runtime supports the chosen settings. A MoE model still needs a budget for resident weights beyond the active parameter count.

WHEN THE TOOL ISN’T ENOUGH

Bring the problem. We’ll scope the next step.

You need a private model deployment that fits your hardware and actual workload.

Send the current setup, what failed, and the result you need. I can help with software integrations, local AI infrastructure, agent permissions, failure recovery and deployment.

Describe the problem →Inspect my public work ↗

Useful first details: system or runtime, expected workload, data restrictions and one redacted example. Don’t send credentials or private customer data.