A new open-source framework lets small, locally run language models handle multi-step agent tasks without routing every call through a cloud API.
Researchers released EffGen, an agentic framework built specifically for small language models - models in the 1.5B-to-32B parameter range that can run on local hardware. Its headline technique is prompt compression: it squeezes input prompts by 70-80% without losing task meaning, which matters most at small scale where every token is a cost. The system also routes tasks by complexity before running them, breaks compound queries into parallel or sequential subtasks based on dependencies, and combines short-term, long-term, and vector memory into a single unified layer. Tested across 13 benchmarks, EffGen beat LangChain, AutoGen, and Smolagents on success rate, speed, and memory use.
The cost-and-privacy case for local agents is real: every API call to a hosted model means money out and data off your network. The scaling data buried in the paper is the most interesting part - prompt compression gives a bigger lift to smaller models (an 11.2% gain at 1.5B parameters versus 2.4% at 32B), while smarter task routing helps larger models proportionally more (3.6% at 1.5B versus 7.9% at 32B). Combined, the two techniques produce gains at every scale without separate tuning, which is a genuinely useful property for teams that want one framework to cover the range.
The Apache 2.0 license and a pip install lower the barrier to trying it; whether these benchmark numbers hold up under real production traffic is the question the paper leaves open.