Local LLM Benchmark Analyzer
Compare repeated inference runs with medians, tail latency and downloadable summaries.
p95 uses nearest rank: sorted observation at ceil(0.95 × count). With few runs it is usually the maximum; it is not a reliable production tail estimate. Retain failed runs separately rather than entering zero-speed successes.
Capacity holds the model. Bandwidth feeds the GPU.
Apple-rated unified memory bandwidth in decimal GB/s. 1.2 TB/s = 1,200 GB/s in decimal units. Reference chips are not additional tested machines.
What changes token speed?
Single-stream dense-model decoding can be limited by how fast weights and cache data reach the GPU. More bandwidth can help when memory traffic is the bottleneck; extra RAM alone does not guarantee faster tokens.
Prompt processing uses a different computation pattern. Batching, GPU compute, context, kernel efficiency and quantization can shift the bottleneck. MoE models do not read every expert weight for every token, so dividing bandwidth by the entire model file is not a valid speed prediction for this Qwen result.
Read the bandwidth and context guide →| Machine | Known RAM | Nominal bandwidth | Measurement status |
|---|---|---|---|
| M1-generation MacBook Pro | 16GB, owner brief | Unconfirmed — exact chip required | No recorded inference runs |
| M2 Max MacBook Pro | 64GB, owner confirmed | 400 GB/s | Four owner-provided oMLX result rows |
| M3 Ultra Mac Studio | 256GB, owner confirmed | 819 GB/s | Four owner-provided oMLX result rows |
| M5 Ultra reference | Not part of our measured bench | 1,200 GB/s (1.2 TB/s) | Specification only — no inference runs supplied |
Apple sources: M5 Ultra specifications, M3 Ultra specifications, M2 Max, M1 Pro / Max. The name “MacBook Pro” does not establish whether the chip is base M1, M1 Pro or M1 Max.
Bring the problem. We’ll scope the next step.
You need a repeatable hardware or inference-runtime test before choosing a deployment.
Send the current setup, what failed, and the result you need. I can help with software integrations, local AI infrastructure, agent permissions, failure recovery and deployment.
Useful first details: system or runtime, expected workload, data restrictions and one redacted example. Don’t send credentials or private customer data.