Blog.

Explorations at the frontier of Local AI.

14 ↓

Latest

Local AI is not just about inference

A deep dive into training across slow networks: how DiLoCo works, why checkpoint averaging helps, and how OverlapSPARTA puts more compute to work.

How Exo combines DGX Spark’s compute with Mac Studio’s memory bandwidth, splitting prompt processing and token generation across the two devices.

The complete collection01—12

Real-device benchmarks for local LLMs, with a guide to how Exo splits models across consumer hardware and what affects performance.

Running a 671-billion-parameter model on eight Mac Minis, with a closer look at Apple Silicon, memory bandwidth, and mixture-of-experts models.

A local, open-source home assistant that uses function calling to control smart devices while keeping personal data on your own hardware.

Bringing Llama to a Pentium II running Windows 98, from transferring files over FTP to compiling and running a model on vintage hardware.

An introduction to training models across consumer devices and how DiLoCo reduces the communication needed between machines.

Connecting a Nokia 3310 to a local AI assistant over SMS, with access to a personal calendar and messages through a Mac.

A look back at Exo’s community, the year’s progress in local and open-source AI, and the mission to make AI accessible to everyone.

Giving local models access to current information with encrypted search that keeps queries private, using embeddings and homomorphic encryption.

A toolkit for simulating distributed training on a single machine, so researchers can test algorithms without maintaining a hardware cluster.

Exploring how to verify work from crowdsourced devices and make decentralized model training trustworthy with edge verification.

Using iPhone mirroring to give a local AI agent personal context from your apps while keeping your data on your own devices.

A distributed training method that exchanges only a small subset of model parameters, plus experiments comparing and combining it with DiLoCo.