Researchers have found a way to make AI models follow strict output formats without tanking the quality of what they write.
A team describes GrammarRL, a training method that teaches language models to work within grammar constraints - think enforced formats like JSON schemas or classification labels - using reinforcement learning instead of hand-labeled examples. The system rewards the model twice: once for how likely its constrained answer is given the prompt, and again for how well the original input can be reconstructed from that answer. Those two reward signals get combined using a group-comparison reinforcement learning technique, nudged along by a beam-search guess, and held in check by comparing against a frozen copy of the original model so it doesn't drift too far from how it normally writes. The team tested it on Llama models ranging from 1 billion to 8 billion parameters across sign language translation, hierarchical text classification, and named entity recognition.
Grammar-constrained decoding is already widely used anywhere an AI needs to output structured data instead of free text, but forcing a valid structure often pushes the model into answers it would not otherwise choose, and accuracy suffers. GrammarRL's approach recovered an average of 9.8 points of performance over constrained greedy decoding, with one task jumping 22.8 BLEU points, and it did so without the extra compute cost that beam search demands at inference time.
That's the part worth watching: a cheap decoding method is closing the gap on a more expensive one's accuracy, which is the kind of unglamorous efficiency gain that tends to quietly ship into production tooling long before anyone writes a headline about it.