Run a local LLM on Mac: a workload-first setup guide

Define the task and the data boundary

Before a large model download, choose one job you can judge: answer a document question with a source, extract fields into JSON, or produce a patch that passes a test. That small check tells you more than an impressive model size. Also trace the whole workflow: a local model can still call a cloud tool.

LM Studio documents offline chat, document processing and local serving after the necessary models and runtimes are available. Model discovery, downloads and update checks require connectivity. Offline operation. Confirm the behavior of any plugins or external tools separately.

Pick a starting interface

Ollama documents ollama ps for inspecting loaded models and processor placement. MLX LM provides command-line and Python workflows. These are documented entry points, not a ranking of speed. Ollama FAQ, MLX LM.

Estimate weights, then measure the full workload

The weight calculation is a lower bound. A four-billion-parameter model at four bits has approximately 2 GB of raw weights in decimal units. Eight billion parameters at four bits gives 4 GB. Neither number includes runtime overhead, context cache, activations, the application or the operating system.

For a 16GB Mac, choose a modest starting model and a bounded task, then observe memory pressure and responsiveness while the real task runs. Avoid a blanket promise that every model under a particular parameter count will be useful. Context settings and other applications can change the result substantially.

Keep your first experiment inspectable

Save the prompt, expected output, model identifier and software versions. Run once to observe loading, then repeat warm trials. Record first-token time, task correctness and memory alongside generation speed. If the task requires structured JSON, validate the schema; fluent text does not satisfy that requirement.

Begin without an agent attached to external services. Once the local inference step works, add retrieval and tools one at a time. Confirm what each tool can read, where it sends data and which actions require review.

Act on the result

If the model completes the task correctly but feels slow, isolate loading from generation and check the runtime settings. If it is fast but wrong, improve the task definition or model choice before buying hardware. If memory pressure is sustained, reduce workload demands and measure again.

Use the hardware planning tool, benchmark template and paired comparison method. For deployment requirements, see private local AI consulting.

Check the path beyond the model

A local model connected to a remote search tool is part of a mixed deployment. Draw the complete data path from source document to retrieval, prompt construction, inference and any external action. Record which component can send network requests and which account controls it. This makes the privacy boundary inspectable without implying that every local application behaves identically.

For a first trial, use a document you own with an answer you can verify. Record the source passage, ask a precise question and check whether the answer is supported. Then ask something absent from the document. A useful system should acknowledge missing evidence rather than fill the gap with a confident guess.

Retain both outputs alongside timing and memory observations. If the application becomes unresponsive while another program is open, that is relevant workload evidence. Do not remove the observation simply because a synthetic throughput run looked fast. Hardware recommendations should describe those practical constraints and the exact setup that was tested.

Sources and review notes

Official documentation checked October 14, 2026. Memory examples are theoretical weight calculations. No machine-specific throughput, speedup or battery-life result is claimed.

Owner evidence: M2 Max 64GB