A small architectural tweak may quietly upend assumptions about how big reinforcement learning models need to be.
Researchers have found that inserting a fixed orthonormal projection layer — a mathematical constraint that forces neural network features into a low-dimensional subspace — preserves or improves agent performance on both single and multi-task benchmarks. No retraining, no auxiliary objectives, no changes to the underlying algorithm required. The key finding: once the bottleneck's dimension exceeds the "intrinsic rank" of the optimal value function (a measure of how complex the task structure actually is), the compression leaves gradient dynamics essentially unchanged. In many cases, agents compressed their value representations to extremely low dimensions with no measurable loss.
This matters because RL research has long assumed high-dimensional representations are necessary for capable agents — an assumption that drives up compute costs and complicates deployment. If task-relevant structure is genuinely low-dimensional, then much of that representational overhead is waste. The work also found that the minimum sufficient dimension tracks environment complexity, not encoder size — meaning bigger networks aren't buying what practitioners think they're buying.
The finding echoes broader trends in ML: sparse attention, low-rank adapters, and distillation research all push the same intuition that neural networks overparameterize. Orthogonal bottlenecks are a simpler intervention than most, which makes them easier to adopt — and harder to dismiss as a research artifact.