Kuaishou is now using an AI system to grade whether its own search results are actually good.
Researchers describe SEEK (Skill-routed Evaluation with Evolvable Knowledge), a framework that judges search results the way users actually see them: as a full page, not isolated snippets. It stores specific evaluation criteria - things like relevance, freshness, or spam signals - in an external "skill bank," picks the relevant ones for each query, and feeds them to a listwise evaluator trained to score entire result lists and flag the specific reason a result failed. A two-stage training process aligns those judgments with human raters, and a "replay-gated" version of the skill bank lets the team add new evaluation rules later without retraining the underlying model. SEEK is now running in production at Kuaishou, the short-video platform with more than 400 million daily active users.
The interesting part is the architecture choice, not the accuracy numbers. Most LLM-based evaluators bake their criteria into a single prompt or into the model's training data, so every new rule means a retrain or a prompt rewrite. SEEK splits "what to check" from "the model doing the checking," which is a more sustainable way to run evaluation at platform scale, where rules change constantly.
The paper claims SEEK "significantly improved the scale and quality" of Kuaishou's search evaluation, but it stops short of any concrete before-and-after numbers - so take the real-world impact on faith for now, benchmark tables notwithstanding.