[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-a-tiny-probe-beats-gpt-35-at-rag-metadata-filtering":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},3892,"a-tiny-probe-beats-gpt-35-at-rag-metadata-filtering","A Tiny Probe Beats GPT-3.5 at RAG Metadata Filtering","Researchers replaced a GPT-3.5 metadata extractor with a lightweight probe trained on hidden states, hitting 90.9% accuracy against GPT-3.5's 80.9%.","A small, locally-run classifier outperforms GPT-3.5 on a key retrieval task - and it costs almost nothing to run.\n\nThe paper targets Multi-Meta-RAG, a retrieval system that answers multi-hop questions by filtering a vector store based on news source metadata. The original setup prompts GPT-3.5-turbo to extract that metadata from each query - a proprietary, pay-per-call step with a known failure mode: the model sometimes returns sources outside the fixed 49-source allow-list. The researchers swapped that out for a probe trained on the hidden states of a small open-source model. Tested across all 2,556 queries in the MultiHop-RAG benchmark, the probe hit 90.9% set-exact accuracy. GPT-3.5 managed 80.9%. A model-free substring baseline landed in between at 88.0%.\n\nThe gap comes almost entirely from null queries - cases where no source should be returned. GPT-3.5 never abstains on these; the probe handles them correctly. For non-null queries, all three approaches sit within about one percentage point of each other, which tells you where the real problem was all along. Beyond accuracy, the probe is bounded by design: its output space is exactly the 49-source vocabulary, so it cannot hallucinate a source that does not exist.\n\nThree choices drive the result: tapping a shallow layer rather than the final output, mean pooling the hidden states, and training with class-imbalance awareness to handle the long tail of rarely-cited sources. A 135-million-parameter model comes within roughly 1.5 points of a 1.5-billion-parameter one, which means the filter is a partial forward pass and a linear head - no API call, no inference cost. The broader implication is uncomfortable for the prompt-everything school of thought: a structured classification problem dressed up as a generation problem is still a classification problem, and a small deterministic probe often does it better.","[\"ai\",\"retrieval\",\"open-source\",\"research\"]","2026-07-07T04:00:00.000Z","2026-07-07T11:03:31.286Z","2026-07-07T11:03:34.257Z","published",null,[],"ai",[24,26,27,28],"retrieval","open-source","research",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.03929",0,{"sections":35},[36,40,45,50,55,60,65,70,75,79,84,88,93,98],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":80,"slug":81,"count":82,"latest_published_at":83},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":85,"slug":86,"count":82,"latest_published_at":87},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]