M2 Max 64GB: 81 tok/s solo, 213 tok/s with 4× batching

The result: more combined output, longer waits

On our M2 Max with 64GB unified memory, the supplied oMLX v0.7.0 screenshot records 81.0 generated tok/s for one request and 213.1 generated tok/s aggregate at 4× batching. Both use 1,024 input tokens and 128 output tokens, with Code (Python) selected. The application reports 2.63× throughput speedup. That is a combined rate across the batch, not 213.1 tok/s for each agent.

The trade-off is visible: average first-token time rises from 1,076.3 ms to 3,871.2 ms, and displayed end-to-end latency rises from 2.662 to 8.526 seconds. For this observed workload, batching increases total output capacity while making responses take longer. This is one screenshot-derived observation per condition, not a repeat-run average.

The visible model label begins Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Opti…; its full artifact, quantization and advanced settings remain unconfirmed. The M3 Ultra Mac Studio with 256GB RAM in our lab used a different artifact and oMLX v0.5.4, so we do not use these captures to declare a hardware winner. Inspect the original screenshots and both datasets.

One tokens-per-second number is incomplete

A user waiting for an interactive answer cares about when output begins and when the task finishes. A service operator handling many requests also cares about total work completed. These perspectives can move in different directions. A batching configuration may process more combined tokens each second while making each user wait longer before the first token.

Label the workload before interpreting a result: input length, output length, concurrency, model artifact, quantization and cache state. A short warm prompt and a long uncached document are not the same experiment. Even a precise result can be misleading when the condition that produced it is omitted.

Separate the main measurements

Time to first token measures the delay until output begins under the benchmark’s timing definition. Generation throughput describes output production over a stated interval. Prompt-processing throughput describes input handling. End-to-end latency includes more of the request lifetime. Check the benchmark implementation rather than assuming all tools choose identical interval boundaries.

Artificial Analysis publishes a performance methodology that distinguishes request latency, speed and throughput at different concurrency levels. The principle to adopt is explicit measurement context. This article does not copy its leaderboard rankings or infer that a local result is directly comparable to its test environment.

Read the supplied M3 Ultra results carefully

The owner-provided oMLX pp1024/tg128 result shows 73.5 generation tok/s and 520.1 ms TTFT. The 4× batching result shows 217.8 aggregate generation tok/s and 2,458.5 ms average TTFT. These observations show a throughput-and-latency tradeoff within the displayed run; they do not establish an independently reproduced performance range.

The 4× number is not an individual user’s token speed. The dashboard session average is another scope and should not be mixed into the benchmark table. Visit the hardware lab record for the original screenshots, configuration and missing-metadata notes.

Repeat conditions and retain the distribution

Run each configuration repeatedly after defining whether loading and warmup belong in the test. Save every observation, including failures, rather than entering only the best run. Compare medians for a compact summary and keep the raw distribution for investigation. With three runs, a nearest-rank p95 is usually the maximum; it is not a well-established production tail estimate.

A run label should identify the exact configuration, not just the machine nickname. If acceleration, context or cache precision changes, create another condition. Otherwise, an apparent hardware improvement may actually be a settings difference.

Tie the benchmark to a useful outcome

After measuring synthetic throughput, run a representative task with a known acceptance criterion. A coding result should pass the intended tests; an extraction result should match the schema and expected facts. Speed without correctness is not useful throughput. Keep failed tasks visible in the outcome dataset even if they emitted tokens quickly.

Use the benchmark analyzer to summarize repeated rows and the bandwidth guide for hardware context. Nominal bandwidth belongs beside the chip specification, while measured inference belongs beside the complete test condition. They support interpretation but should never be silently substituted for one another.

Implementation map

Sources and limits

Artificial Analysis performance methodology; oMLX benchmark implementation. Owner values are transcribed from the supplied screenshots, not independently reproduced. Documentation checked October 14, 2026. Recommendations and worked examples are editorial methods, not claims of a completed client engagement.

Owner evidence: M2 Max 64GB