A new tuning-free method lets language models handle far longer inputs than they were trained on — without the usual accuracy trade-offs.
Most open-weight language models ship with a fixed context window, but real deployments — retrieval-augmented generation, code assistants reading entire repositories, agentic pipelines accumulating tool call logs — routinely push inputs far beyond that limit. Existing zero-shot extension methods get around this by applying a rescaling factor to the model's positional encodings, but they force an uncomfortable choice: dial the factor up and short-context accuracy degrades; dial it down and long-context performance collapses. Jet-Long sidesteps that trade-off by running two attention windows in parallel — one that preserves the model's original positional encoding behavior for nearby tokens, and one whose rescaling factor adjusts dynamically based on how long the current input actually is. An efficient kernel implementation keeps the added compute cost to under 4% overhead on standard hardware, and long-context prefill reaches up to 1.39 times the throughput of FlashAttention-2 on H100 GPUs.
The numbers are worth taking seriously. On the RULER benchmark, Jet-Long outperforms the strongest existing baseline by 4.79 percentage points at the 1.7B model size, with smaller but consistent gains at 4B and 8B. It also posts the best overall score on HELMET-RAG — a benchmark the HELMET authors themselves flag as the most efficient predictor of real downstream long-context performance — and achieves the lowest perplexity on PG-19. Those are three independent signal sources pointing in the same direction, which is less common than the preprint circuit makes it seem.
Context extension has become its own cottage industry as labs race to make open-weight checkpoints competitive with frontier closed models on long-document tasks. Jet-Long's claim to distinguish itself is that it requires no fine-tuning and no hyperparameter search — useful properties if the goal is a drop-in upgrade to existing deployment pipelines. Whether that holds across model families beyond the Qwen3 line tested here is the obvious next question.