The first recorded bench results
These results come from four screenshots supplied by Fazal Shah, who identified the machine as an M3 Ultra Mac Studio with 256GB unified memory. The configuration capture names Qwen3.6-35B-A3B-8bit, with an estimated model size of 36.9 GB, the Code (Python) benchmark context, and 128 generated tokens. The dashboard shows oMLX v0.5.4. CPU/GPU core counts, macOS, model repository and run date are not visible in the evidence.
The dataset below preserves the measured values shown by the application. It is an owner-provided record, not an independently reproduced result or a claim about every M3 Ultra configuration. The four rows describe two single-request tests and two concurrent-batching tests. The one-request pp1024 result is reused as the batching baseline rather than counted as another independent trial.
Compare context and concurrency
Generation speed
Time to first token
| Input | Batch | TTFT (ms) | Prefill tok/s | Generation tok/s | E2E (s) | Peak GB reported |
|---|---|---|---|---|---|---|
| 1024 | 1 | 1076.3 | 951.4 | 81 | 2.662 | 19.89 |
| 4096 | 1 | 3956.5 | 1035.3 | 78.4 | 5.595 | 21.79 |
Owner-supplied screenshots; not independently reproduced. M2 Max RAM is owner-confirmed at 64GB; M3 Ultra RAM is owner-confirmed at 256GB; both machines’ CPU/GPU cores and macOS remain unconfirmed. Full model repositories/revisions, M2 quantization, acceleration/cache settings, measurement dates and repeat-run spread are unknown. The model artifacts and oMLX versions differ: this is not a controlled hardware ranking. Peak memory uses the application-reported GB label. Batch generation TPS is aggregate; batch TTFT is an average. Dashboard averages are separate from benchmark records.
Capacity holds the model. Bandwidth feeds the GPU.
Apple-rated unified memory bandwidth in decimal GB/s. 1.2 TB/s = 1,200 GB/s in decimal units. Reference chips are not additional tested machines.
What changes token speed?
Single-stream dense-model decoding can be limited by how fast weights and cache data reach the GPU. More bandwidth can help when memory traffic is the bottleneck; extra RAM alone does not guarantee faster tokens.
Prompt processing uses a different computation pattern. Batching, GPU compute, context, kernel efficiency and quantization can shift the bottleneck. MoE models do not read every expert weight for every token, so dividing bandwidth by the entire model file is not a valid speed prediction for this Qwen result.
Read the bandwidth and context guide →| Machine | Known RAM | Nominal bandwidth | Measurement status |
|---|---|---|---|
| M1-generation MacBook Pro | 16GB, owner brief | Unconfirmed — exact chip required | No recorded inference runs |
| M2 Max MacBook Pro | 64GB, owner confirmed | 400 GB/s | Four owner-provided oMLX result rows |
| M3 Ultra Mac Studio | 256GB, owner confirmed | 819 GB/s | Four owner-provided oMLX result rows |
| M5 Ultra reference | Not part of our measured bench | 1,200 GB/s (1.2 TB/s) | Specification only — no inference runs supplied |
Apple sources: M5 Ultra specifications, M3 Ultra specifications, M2 Max, M1 Pro / Max. The name “MacBook Pro” does not establish whether the chip is base M1, M1 Pro or M1 Max.
What the results show
Increasing input length from 1,024 to 4,096 tokens raised first-token time from 520.1 to 1,907.8 ms, while generation throughput was 73.5 and 75.2 tok/s respectively. The application-reported peak memory rose from 35.48 to 36.20 GB. These are two displayed observations; they do not establish repeat-run variability or a universal relationship between context and performance.
At pp1024/tg128, the displayed 2× batch generated 137.5 tok/s aggregate, with 1,306.6 ms average TTFT and 3.212 seconds E2E. The 4× batch generated 217.8 tok/s aggregate, with 2,458.5 ms average TTFT and 4.975 seconds E2E. The application reports speedups of 1.87× and 2.96× relative to its single-request baseline. More combined throughput comes with higher first-token delay in this displayed run. This should not be described as each user receiving 217.8 tok/s.
Inspect the source screenshots
Each screenshot opens in a full-size gallery. Use the previous/next controls or arrow keys to compare the result and configuration captures. Escape closes the viewer. The dashboard and cache images are included as context, with captions explaining their different scope.
The benchmark output used for the four result records. Batch throughput is aggregate; single-request results use 128 generated tokens.
Qwen3.6-35B-A3B-8bit; Code (Python); pp1024, pp4096 and 2×/4× batching. Advanced settings were collapsed in the supplied capture.
oMLX v0.5.4 session averages across the selected dashboard scope. These averages are separate from the benchmark records.
Cache state shown in the supplied dashboard capture. This does not establish whether benchmark requests reused cached prompts.
What is not established yet
The dashboard session average of 65.5 generation tok/s and 164.8 prompt-processing tok/s is not substituted for the benchmark table. Its selected scope is All Models, and session traffic can include different workloads. Cache statistics show runtime state but do not prove which cache policy the benchmark used. The advanced benchmark settings were collapsed in the supplied capture.
Before a cross-machine comparison, collect exact hardware and software versions, the model repository/revision, sampling and acceleration settings, prompt-cache policy, power mode and background activity. Repeat the same configuration at least three times and retain the individual rows. Add a separate quality task with an explicit acceptance test: throughput alone does not measure code correctness or agent reliability.
Reproduce the comparison
Use pp1024 and pp4096 with tg128 and the same benchmark context. Run 1×, 2× and 4× concurrency as separate conditions. Match model artifacts and settings between machines wherever they fit. A smaller common model is needed for a fair comparison involving a 16GB machine; changing quantization or model size changes the experiment.
Export repeated results to the benchmark analyzer, use the memory planner for explicit memory assumptions, and read the measurement protocol. Source: owner-supplied screenshots; metric implementation reference: oMLX benchmark code. No external leaderboard position is claimed.
M2 Max 64GB: benchmark and all-time serving record
Measured workload, not a hardware ranking.
Owner-supplied oMLX v0.7.0 screenshots show Code (Python), 128 output tokens, and a model label beginning Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Opti… . The full artifact and quantization are not visible.
| Input / batch | Generation tok/s | TTFT ms | E2E seconds | Peak GB reported |
|---|---|---|---|---|
| 1024 / 1× | 81 | 1076.3 | 2.662 | 19.89 |
| 4096 / 1× | 78.4 | 3956.5 | 5.595 | 21.79 |
| 1024 / 2× | 133.2 | 2085.7 | 4.47 | Not shown |
| 1024 / 4× | 213.1 | 3871.2 | 8.526 | Not shown |
At 2× and 4× batching, generation is aggregate throughput, not the rate each user receives. The screenshot reports 1.64× and 2.63× speedup.
All-time serving: a separate view
The All-Time / All Models dashboard reports 215.6 tok/s prompt processing excluding cached tokens and 33.2 tok/s generation. Its date range, model mix and request count are unknown. These are mixed-use averages, not another benchmark or proof of a latency/cost saving. The empty Today history does not invalidate the all-time counters.
The M2 Max specification is 400 GB/s memory bandwidth; the M3 Ultra is 819 GB/s. The supplied tests use different artifacts and oMLX versions (v0.7.0 versus v0.5.4), so their token speeds cannot isolate bandwidth or establish which chip is faster. Inspect both datasets, bandwidth sources and original screenshots →
M2 Max original screenshots
oMLX v0.7.0: two single-request tests and 2×/4× aggregate batching. One capture per condition; repeat-run variation unknown.
Code (Python), pp1024/pp4096, tg128 and 2×/4× batches. Model label is truncated; quantization and collapsed advanced settings are unconfirmed. ANE-aligned prompts is unchecked.
All-Time / All Models: rounded token counters and mixed-workload averages. The Today usage-history section is empty and is a different scope.
More memory didn’t fix the model. Finding the right model did.
Even our M3 Ultra Mac Studio with 256GB unified memory could run models that failed to do the work we needed. Qwen3.6 35B MoE became our sweet spot for agentic AI, coding and basic operations.
- Did not meet our needs
Smaller models tried
Frequent hallucinations and insufficient reasoning for our OpenClaw workflows.
- Did not meet our needs
Other Qwen variants
The variants we tried did not sustain the agent workflows we needed.
- Did not meet our needs
“Mistral 70B” — owner’s label
Still unsuccessful in our setup, including on the 256GB M3 Ultra. Exact artifact unconfirmed.
- Our operational sweet spot
Qwen3.6 35B MoE
Worked for our agentic AI, coding and basic operations; our preferred model among those tried.
Running in parallel for months of basic operations, according to the owner’s experience with Qwen3.6 35B MoE.
Our recommendation: test the workflow before buying more RAM.
For an OpenClaw setup like ours, start by evaluating this 35B MoE model against the tasks your agents actually perform. Check reasoning, hallucinations, tool use and completed work, then measure latency and memory. A model fitting into RAM does not establish that it can run the workflow successfully.
Evidence scope and how to read this verdict
This is Fazal Shah’s retrospective account of models tried in his own setup, not a universal claim that other models fail. “Failed” means the model did not meet his operational needs; it does not necessarily mean it failed to load. Smaller models reportedly hallucinated and could not support the reasoning and work required by his OpenClaw agents.
Exact alternative model repositories, quantizations, dates, prompts, failure counts and intervention rates have not been supplied. “Mistral 70B” preserves the owner’s wording, not a verified model identifier. The 16-agent account does not establish 16 simultaneous inference requests, uninterrupted uptime, unattended operation or a measured success rate. Its exact machine allocation and deployment artifact remain unconfirmed.
The screenshot benchmarks test 1×, 2× and 4× batching with 128 output tokens, not the 16-agent operational deployment. They show Qwen3.6-35B-A3B-8bit on the M3 Ultra and a differently named, truncated artifact on the M2 Max. They cannot independently verify months of agent operation or identify the exact deployment artifact.
Next evidence to collect: a redacted OpenClaw agent roster, dated run history, exact model artifact and configuration, representative completed tasks, failed attempts and human interventions. Keep task quality and throughput as separate measurements.
Fazal Shah builds AI systems, automation and software products. Based in Pakistan, working with a global audience. About Fazal →