[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-kuaishou-deploys-ai-evaluator-to-grade-its-own-search-results":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},7738,"kuaishou-deploys-ai-evaluator-to-grade-its-own-search-results","Kuaishou Deploys AI Evaluator to Grade Its Own Search Results","Kuaishou deployed a skill-routed AI evaluator that grades search quality and can absorb new evaluation rules without retraining the underlying model.","Kuaishou is now using an AI system to grade whether its own search results are actually good.\n\nResearchers describe SEEK (Skill-routed Evaluation with Evolvable Knowledge), a framework that judges search results the way users actually see them: as a full page, not isolated snippets. It stores specific evaluation criteria - things like relevance, freshness, or spam signals - in an external \"skill bank,\" picks the relevant ones for each query, and feeds them to a listwise evaluator trained to score entire result lists and flag the specific reason a result failed. A two-stage training process aligns those judgments with human raters, and a \"replay-gated\" version of the skill bank lets the team add new evaluation rules later without retraining the underlying model. SEEK is now running in production at Kuaishou, the short-video platform with more than 400 million daily active users.\n\nThe interesting part is the architecture choice, not the accuracy numbers. Most LLM-based evaluators bake their criteria into a single prompt or into the model's training data, so every new rule means a retrain or a prompt rewrite. SEEK splits \"what to check\" from \"the model doing the checking,\" which is a more sustainable way to run evaluation at platform scale, where rules change constantly.\n\nThe paper claims SEEK \"significantly improved the scale and quality\" of Kuaishou's search evaluation, but it stops short of any concrete before-and-after numbers - so take the real-world impact on faith for now, benchmark tables notwithstanding.","[\"ai\",\"search\",\"kuaishou\",\"evaluation\"]","2026-09-25T04:00:00.000Z","2026-09-25T19:00:04.020Z","2026-09-25T19:00:08.981Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The claim that 'Kuaishou's production numbers suggest that bet paid off, not just in a benchmark table' invents evidence — the source gives no production metrics, only the vague claim that SEEK is 'significantly improving the scale and quality of online search evaluation'; rewrite that line to reflect only what the source actually supports.","resolved","ai",[30,32,33,34],"search","kuaishou","evaluation",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.29803",0,{"sections":41},[42,46,51,56,61,66,71,76,81,86,91,96,101,106],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",4487,"2026-09-25T15:40:03.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Security","security",734,"2026-09-25T15:52:13.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Policy","policy",389,"2026-09-25T15:27:35.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Deals","deals",247,"2026-09-25T15:26:22.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Hardware","hardware",185,"2026-09-25T15:00:22.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Science","science",139,"2026-09-25T11:55:23.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Consumer Tech","consumer-tech",132,"2026-09-25T15:30:00.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Software","software",88,"2026-09-24T23:06:55.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Dev Tools","dev-tools",81,"2026-09-25T09:59:40.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Startups","startups",74,"2026-09-25T14:05:04.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Gaming","gaming",47,"2026-09-25T13:33:16.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"General","general",46,"2026-09-25T02:12:57.000Z",{"name":102,"slug":103,"count":104,"latest_published_at":105},"Reviews","reviews",30,"2026-09-24T20:07:31.000Z",{"name":107,"slug":108,"count":109,"latest_published_at":110},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]