Running a 70-billion-parameter language model on the computers already in your house is now a documented, benchmarked possibility.
Researchers released prima.cpp, a distributed inference system designed to spread model execution across a home cluster of mixed consumer hardware — laptops, desktops, phones, whatever is available — connected over Wi-Fi. The system introduces a scheduling layer called Halda that assigns workloads to each device based on its actual CPU, GPU, RAM, and VRAM constraints. A second technique, pipelined-ring parallelism, keeps disk reads from stalling computation by overlapping storage I/O with processing. On four ordinary consumer devices, the researchers ran a 70B model at 674 milliseconds per token and a 32B model at 26 tokens per second using speculative decoding — with memory pressure staying below 6%.
The gap between those numbers and existing tools is the real headline. Against llama.cpp, exo, and dllama, prima.cpp posts 5 to 17 times lower time-per-output-token. That range matters because it suggests the improvement holds across different hardware configurations, not just a cherry-picked benchmark setup. Private, offline inference at this model scale has been technically possible in theory but practically unusable for most people; closing that gap changes who can self-host a capable model.
The code is open-source, which invites scrutiny — real-world results on random consumer hardware will vary considerably from a controlled four-device test. Still, this is meaningfully further along than the usual academic demo.