Turns out predicting how much GPU memory an AI coding agent will eat doesn't need machine learning at all.
Researchers tested memory-usage forecasting for a code-generating AI agent running on quantized models, using a LangGraph-based CUDA-kernel-synthesis agent named AgentK on a single Nvidia H100 GPU. They compared a simple closed-form formula, fed with two measured constants for loaded model weights and a fixed activation-memory overhead, against machine-learning-based prediction, across four LLM backbones and 1,920 trajectories, all running 4-bit quantization. The simple formula matched or beat the best learned model on three of four backbones, with prediction error (MAPE) of 2.2-4.4% versus 3.4-6.5% for the learned approach. Only the smallest model, Phi-4-mini, broke the pattern, because its memory use barely varied at all (a coefficient of variation of just 0.3%).
The bigger finding buried in the same dataset: whether generated code actually compiles has nothing to do with memory, and everything to do with the model. Success rates ranged from 5.7% for Phi-4-mini to 62.0% for Qwen2.5-Coder-14B, a gap driven purely by model capability, not GPU headroom. For anyone building infrastructure around agentic coding tools, that's a signal to stop over-engineering memory management and start caring more about which model actually writes working code.
Complex regression models built to forecast VRAM in these quantized setups are, per the paper, solving a problem that barely exists: the variance is too small to model. Sometimes the fancy machine learning approach just isn't worth the GPU cycles it takes to run it.