A new benchmark scores AI chart-explaining agents without needing a human-made answer key.
Researchers introduced Lexara-RF, a set of 13 reference-free metrics for evaluating conversational visual analytics agents - chatbots that turn plain-language questions into charts and explanations. Built on an earlier framework called Lexara, the metrics check only the original prompt, the underlying data, and the model's response, rather than comparing against a curated correct answer. The checks translate visualization design theory and Gricean cooperative principles, the norms of cooperative conversation, into computable tests for consistency, intent alignment, and design validity. Tested against a corpus of human-rated cases, Lexara-RF matched the accuracy of reference-based evaluation methods and beat baseline metrics that just measure surface text similarity.
Reference benchmarks are expensive to build by hand and can't cover every valid way an AI might answer an open-ended data question, and they don't exist at all once a system is live in production. A method that skips the answer key and can still pinpoint which part of a response failed, a bad chart choice versus a misread of the data, gives teams a way to monitor AI analytics tools continuously rather than just during a one-time benchmark run.
It's a narrow fix for a narrow category of product, but as more dashboards get a chatbot bolted on, knowing whether the chart is even right is turning into an operational problem, not just an academic one.