A framework called E2LLM lets large language models run across scattered edge and fog devices instead of demanding one beefy server.
The system, described in a paper on arXiv, tackles a basic problem: most LLM deployment schemes assume the whole model fits on a single machine. That breaks down on edge and fog networks, where devices are cheap, uneven, and resource-constrained. E2LLM's fix is to replicate the full model across multiple device clusters rather than slicing one copy thin across everything. Each cluster gets assigned a specialized job - handling either the prefill phase (reading input) or the decode phase (generating output) - because those two phases stress hardware differently. A genetic algorithm groups devices into clusters, and dynamic programming decides how to split the model within each cluster to avoid bottlenecks. Against the Splitwise baseline, the researchers report average waiting times cut by more than 50 percent under heavy load.
That matters because edge inference is becoming less optional. Running models locally on phones, routers, and industrial sensors saves bandwidth and keeps data closer to home, but the hardware is a patchwork - nothing like the uniform GPU racks that datacenter LLM serving assumes. A framework that adapts clustering and partitioning to whatever devices happen to be available is a more realistic answer to that mess than hoping for better chips.
Still, this is an arXiv preprint with benchmark numbers from the authors, not an independently verified production deployment. Splitwise is a reasonable baseline, but it is also a research project, not the toughest competitor available. Treat the 50 percent figure as promising, not proven.