Fazal Shah.
AGENT ENGINEERING

AI agents beyond the demo.

Building an AI agent is one thing. Getting it to work reliably with real tools, real business data, and real operational constraints is another.

HOW THE PIECES FIT

Understand the workflow before adding autonomy.

Start with the request, allowed tools and approval boundary. The next step is making tool failures, retries and incomplete work visible to the operator.

Read the engineering method ↗
THE AGENT WORKBENCHWorkflow illustration
01
YOUR REQUEST

A task, with context.

02
REASON & PLANUnderstand the task.
Choose the next tool.

Keep actions inside the allowed scope.

03CALL THE RIGHT TOOL
Research
Retrieve data
Connect APIs
04 / CHECK

Validate. Get approval
when needed.

05 / DELIVER

Take the action.
Keep a record.

Tools do the work. Permissions, validation and human approval define the boundaries. Errors need a visible recovery path.

What AI agents can actually do

Agents can coordinate model decisions with tools and workflows. Their usefulness depends on the task, the permissions available, and how failures are handled. Start with a bounded workflow that can be tested before expanding autonomy.

The engineering underneath.

01 / ENGINEERING NOTE

Multi-agent architecture

Separate responsibilities only when doing so improves evaluation, permissions or workflow ownership. More agents introduce coordination and cost overhead.

02 / ENGINEERING NOTE

Tool calling & integrations

Define schemas, validate arguments and enforce permissions outside the model. Keep read and write tools distinct.

03 / ENGINEERING NOTE

Scheduling & background workflows

Give each workflow a bounded trigger, timeout, owner and recovery path. Make duplicate execution safe.

04 / ENGINEERING NOTE

Production reliability

Test partial failures, API rate limits and missing data. Use bounded retries and a visible human fallback.

05 / ENGINEERING NOTE

Security & permissions

Treat external content as untrusted, keep secrets server-side and require approval for consequential actions.

06 / ENGINEERING NOTE

Testing & monitoring

Evaluate representative tasks and track success, latency, cost, tool errors and intervention rates.

07 / ENGINEERING NOTE

Cost optimization

Measure tokens, tool calls, retries and hosting separately. A model price alone is not an operating-cost estimate.

08 / ENGINEERING NOTE

Deployment

Document configuration, incident ownership, rollback and a kill switch before expanding the workflow.

From architecture to implementation

Read Define the boundaries before building an agent, estimate spend with the cost calculator, or explore AgentMagic, a platform we're building and a project I work on.

Discuss Agent Architecture ↗
FIELD VERDICT / REPORTED BY FAZAL SHAH

More memory didn’t fix the model. Finding the right model did.

Even our M3 Ultra Mac Studio with 256GB unified memory could run models that failed to do the work we needed. Qwen3.6 35B MoE became our sweet spot for agentic AI, coding and basic operations.

What worked in our setupQualitative experience · no accuracy scores or ranked benchmark
  1. Smaller models tried

    Frequent hallucinations and insufficient reasoning for our OpenClaw workflows.

    Did not meet our needs
  2. Other Qwen variants

    The variants we tried did not sustain the agent workflows we needed.

    Did not meet our needs
  3. “Mistral 70B” — owner’s label

    Still unsuccessful in our setup, including on the 256GB M3 Ultra. Exact artifact unconfirmed.

    Did not meet our needs
  4. Qwen3.6 35B MoE

    Worked for our agentic AI, coding and basic operations; our preferred model among those tried.

    Our operational sweet spot
REPORTED OPERATIONAL OUTCOME16 OpenClaw agents

Running in parallel for months of basic operations, according to the owner’s experience with Qwen3.6 35B MoE.

Our recommendation: test the workflow before buying more RAM.

For an OpenClaw setup like ours, start by evaluating this 35B MoE model against the tasks your agents actually perform. Check reasoning, hallucinations, tool use and completed work, then measure latency and memory. A model fitting into RAM does not establish that it can run the workflow successfully.

Evidence scope and how to read this verdict

This is Fazal Shah’s retrospective account of models tried in his own setup, not a universal claim that other models fail. “Failed” means the model did not meet his operational needs; it does not necessarily mean it failed to load. Smaller models reportedly hallucinated and could not support the reasoning and work required by his OpenClaw agents.

Exact alternative model repositories, quantizations, dates, prompts, failure counts and intervention rates have not been supplied. “Mistral 70B” preserves the owner’s wording, not a verified model identifier. The 16-agent account does not establish 16 simultaneous inference requests, uninterrupted uptime, unattended operation or a measured success rate. Its exact machine allocation and deployment artifact remain unconfirmed.

The screenshot benchmarks test 1×, 2× and 4× batching with 128 output tokens, not the 16-agent operational deployment. They show Qwen3.6-35B-A3B-8bit on the M3 Ultra and a differently named, truncated artifact on the M2 Max. They cannot independently verify months of agent operation or identify the exact deployment artifact.

Next evidence to collect: a redacted OpenClaw agent roster, dated run history, exact model artifact and configuration, representative completed tasks, failed attempts and human interventions. Keep task quality and throughput as separate measurements.

BUILD IN PUBLIC

Tools, skills and field notes you can inspect.

I publish Mac utilities, agent infrastructure and reusable workflows on GitHub. Start with the code and documentation, then adapt the parts that fit your setup.

Mac utilities

Auto Pause Mac Apps

A macOS menu bar utility for pausing background apps and managing memory headroom before local model work.

Explore on GitHub ↗
Agent infrastructure

Turnstile

A concurrency-capping FIFO proxy in front of a local model endpoint, designed to keep multi-agent requests from overwhelming one GPU.

Explore on GitHub ↗
Skills & playbooks

Claude skills

Reusable skills covering living documentation, self-healing MCP services and coverage-verified RAG indexing.

Explore on GitHub ↗
FROM THE ENGINEERING NOTES

Practical guides from the Workshop

Pick a problem you are working on. Each guide includes a method, sources and a tool or example to try.

All articles →