[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-new-split-method-makes-graph-neural-network-tests-more-reliable":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},6831,"new-split-method-makes-graph-neural-network-tests-more-reliable","New Split Method Makes Graph Neural Network Tests More Reliable","A topology-aware evaluation method that stratifies by node homophily, not just class balance, cuts variance in graph neural network accuracy scores.","A new benchmarking method suggests that years of graph neural network research may have been comparing models across a coin flip, not a fair test.\n\nResearchers built a technique called HP that changes how datasets get split into training, validation, and test sets before benchmarking graph neural networks (GNNs). The standard approach borrows k-fold cross-validation from ordinary machine learning, sometimes stratifying folds by class label to keep the label mix even. But graphs are not rows in a spreadsheet: nodes are connected, and how much a node's neighbors share its label, its local homophily, varies across the graph and directly shapes how message-passing models behave. HP stratifies folds by homophily first and class label second, so no single fold accidentally loads up on nodes that are easy or hard for a GNN to classify.\n\nAcross 15 datasets and 7 GNN architectures, HP produced far more consistent accuracy rankings than random k-fold splitting, and it won on 13 of the 15 datasets while still preserving class balance. That matters because published claims that one architecture beats another by a point or two could just be measuring which random split a paper happened to use, not a real algorithmic edge.\n\nIt's a small fix, but it points at a bigger problem: benchmark leaderboards move faster than the statistical hygiene needed to trust them.","[\"graph neural networks\",\"machine learning benchmarks\",\"ai research\",\"reproducibility\"]","2026-09-18T04:00:00.000Z","2026-09-18T18:38:05.582Z","2026-09-18T18:38:17.496Z","published",null,[],"ai",[26,27,28,29],"graph neural networks","machine learning benchmarks","ai research","reproducibility",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19210",0,{"sections":36},[37,40,44,49,54,58,62,67,71,76,81,86,91,96],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",4030,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",654,{"name":45,"slug":46,"count":47,"latest_published_at":48},"Policy","policy",338,"2026-09-11T04:00:00.000Z",{"name":50,"slug":51,"count":52,"latest_published_at":53},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":55,"slug":56,"count":57,"latest_published_at":18},"Hardware","hardware",155,{"name":59,"slug":60,"count":61,"latest_published_at":18},"Science","science",121,{"name":63,"slug":64,"count":65,"latest_published_at":66},"Consumer Tech","consumer-tech",99,"2026-09-09T17:27:33.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":18},"Dev Tools","dev-tools",78,{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",75,"2026-09-10T20:41:21.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Startups","startups",55,"2026-09-09T23:14:29.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Gaming","gaming",43,"2026-09-10T12:18:06.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",41,"2026-09-08T01:57:23.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]