A large language model picking fine-tuning settings for neural operators listens to its own results, not just its training-time instincts.
Researchers tested an LLM agent that selects fine-tuning configurations for neural operators adapting across partial differential equation (PDE) families, under a tight trial budget. The agent beat both random search and Bayesian optimization on held-out test error in nearly every matched comparison. To figure out why, the team ran controlled interventions rather than trusting the scoreboard alone. Before the LLM ever saw a validation score, its first guess already landed near the top of the random-search pool, and swapping the PDE description shifted the base learning rate it chose, showing the bias is tied to the specific problem.
That matters because 'our AI agent got good results' is a weak claim on its own; it says nothing about whether the system is reasoning or just got lucky. By scrambling validation scores among already-tested configurations and watching the next proposal change every time, while a scrambled but value-preserving rewrite changed nothing, the researchers showed the model is genuinely reacting to feedback, not running on autopilot.
It is a narrow demonstration, confined to PDE solver tuning, but it is the kind of causal test that scientific-AI claims rarely get, and more of them should.