A new partitioning framework promises to make computing-in-memory hardware far more useful for running ML models at the edge.
Researchers published a method for splitting machine learning workloads between traditional CPUs and Computing-in-Memory accelerators — chips that do math directly inside memory rather than shuttling data to a separate processor. The paper targets resistive RAM-based CIM hardware, which comes with real constraints: limited storage, slow write speeds, and a finite number of write cycles before the memory degrades. Prior partitioning approaches largely ignored those constraints, and ignored the CPU as a useful co-processor. The new framework uses Integer Linear Programming to find the optimal split, factoring in parallelism and low-level hardware behavior alongside those RRAM limits.
The numbers are notable: heterogeneous CPU-CIM execution hit up to 30.9x faster inference than running on an edge CPU alone, and 7.3x faster than a high-performance desktop CPU. That gap matters because edge inference — running models locally on devices rather than in the cloud — is where power budgets and memory constraints bite hardest. A framework that wrings more speed from constrained hardware without ignoring its failure modes is the kind of unglamorous work that actually ships.
CIM accelerators have been a research darling for years, but the gap between lab results and deployable systems has stayed stubbornly wide — largely because papers optimize for peak throughput and gloss over endurance or write-latency penalties. Whether this framework closes that gap in production silicon remains to be seen.