[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-firstresearch-makes-ai-generated-science-questions-auditable":10,"sections":44},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":34,"tags":35,"sources":39,"feedback":43,"feedback_at":22,"cost_usd":43,"total_tokens":43},4273,"firstresearch-makes-ai-generated-science-questions-auditable","FirstResearch Makes AI-Generated Science Questions Auditable","A new framework forces LLM research agents to show their work before proposing a scientific question, scoring 4.86\u002F5 against a 4.38\u002F5 baseline.","A new open-source framework wants to fix one of the quieter problems in AI-assisted science: the research question that sounds rigorous but cannot be inspected.\n\nFirstResearch wraps each proposed research question in what its authors call a Research Question Certificate — a structured document that records assumptions, a mechanism model, a falsifiable hypothesis, a minimal decisive test, and a failure update rule. The idea is that a scientist should be able to audit the question before any downstream experiment runs. The system was tested across ten LLM-agent research topics against prompt-level baselines drawn from three existing frameworks: AI co-scientist, Agent Laboratory, and AI Scientist-v2. Under a blind DeepSeek judge, FirstResearch outperformed all three. An independent Gemini-2.5-Flash rescore of the same 40 baseline packages confirmed the ranking: FirstResearch scored 4.86\u002F5 versus 4.38\u002F5 for the strongest competing baseline, with a Pearson correlation of 0.865 between the two judges on average score.\n\nThe more striking result is the ablation: stripping out the certificate entirely drops scores below 1\u002F5 under both judges, while a certificate-only configuration scores 4.90\u002F5 under DeepSeek and 4.88\u002F5 under Gemini. That suggests the structured derivation record — not prompt engineering or model choice — is doing most of the work. Most AI science tools compete on ideation volume or literature coverage; FirstResearch is betting that auditability is the bottleneck researchers actually care about.\n\nThe authors are candid that these results are preliminary and rely on LLM judges rather than human domain experts, which limits how much weight any single benchmark score can carry — a caveat worth holding onto as the AI-for-science space gets louder.","[\"ai\",\"science\",\"research\",\"open-source\"]","2026-07-08T04:00:00.000Z","2026-07-08T04:25:27.864Z","2026-07-08T04:25:30.680Z","published",null,[24,30],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The body states the certificate-only ablation scores as 4.86\u002F4.86 (matching the full system), but the source reports 4.90 under DeepSeek and 4.88 under Gemini — the article's figures are wrong and contradict the source material.","resolved",{"id":31,"reviewer":26,"round":32,"reason":33,"status":29},"editor-r2",2,"The article states the 4.86\u002F5 Gemini rescore figure as FirstResearch's score under the Gemini judge, but per the source, 4.86\u002F5 is the Gemini rescore of FirstResearch under the Gemini judge — and the certificate-only ablation figures (4.90 under DeepSeek, 4.88 under Gemini) are still reported incorrectly in the body as 4.86\u002F4.86, so concern [editor-r1] remains unresolved.","ai",[34,36,37,38],"science","research","open-source",[40],{"name":41,"url":42},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.05682",0,{"sections":45},[46,50,55,60,65,70,75,80,84,89,94,98,103,108],{"name":47,"slug":34,"count":48,"latest_published_at":49},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":81,"slug":36,"count":82,"latest_published_at":83},"Science",66,"2026-07-10T10:29:37.000Z",{"name":85,"slug":86,"count":87,"latest_published_at":88},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":95,"slug":96,"count":92,"latest_published_at":97},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":104,"slug":105,"count":106,"latest_published_at":107},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":109,"slug":110,"count":111,"latest_published_at":112},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]