A new benchmark says fine-tuning a language model still beats clever prompting when the task is planning someone's vacation.
Researchers built TripScore, a benchmark drawn from real user logs and calibrated against 1,468 pairwise judgments from 203 travel experts, because most people typing into a trip-planning bot give vague requests rather than neat checklists of constraints. The benchmark pairs a feasibility check for format and basic sense with a single reward score that blends output quality and how well a plan matches what the traveler actually wanted. The team then tested several approaches under the same base model and comparable response times: direct prompting, giving the model more "thinking" time, symbolic solvers bolted onto the LLM, code-writing agents, and reinforcement-learning fine-tuning such as GRPO. Fine-tuning won consistently across the comparison.
For consumer AI products, that is a data point against the assumption that a good enough prompt can substitute for retraining. It suggests that companies shipping travel bots, or any tool that has to handle messy, constraint-heavy requests from real users, may need to budget for fine-tuning pipelines rather than leaning entirely on prompt engineering.
One caveat: the result comes from the same team's own benchmark, calibrated on their own service's users - promising, but worth reproducing on other tasks before anyone rewrites their AI roadmap around it.