[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-llmography-wants-to-audit-how-you-used-ai-not-just-what-it-made":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},2682,"llmography-wants-to-audit-how-you-used-ai-not-just-what-it-made","LLMography Wants to Audit How You Used AI, Not Just What It Made","A new framework scores the human-AI conversation process itself, tracking prompt quality, dependency, and traceability across seven measurable KPIs.","Auditing AI outputs is the wrong unit of analysis, argues a new paper — the conversation that produced them is what actually reveals accountability.\n\nResearchers have proposed LLMography, a framework that treats Human-AI conversation logs the way bibliography treats sources: as structured, citable records of how a piece of work came to exist. A prototype tool analyzes chat traces and generates reports across seven KPIs — Prompt Quality Score, Human Direction Score, AI Dependency Level, Auditability Score, Final Output Traceability, Privacy Risk Level, and a recommended LLMography label. In a preliminary run on 19 anonymized engineering-student audit reports, most interactions were classified as Human-AI co-produced. Average scores came in at 86.8\u002F100 for Human Direction, 81.9\u002F100 for Prompt Quality, 72.8\u002F100 for Auditability, and 77.1\u002F100 for Final Output Traceability. The paper also applies its own framework to itself, landing the classification \"human-originated, human-directed, AI-assisted co-production.\"\n\nThe practical gap this targets is real: organizations deploying AI in education or engineering workflows currently have no standard way to distinguish a human who used AI as a spell-checker from one who outsourced the entire task. A conversation-level trace would make that distinction auditable rather than assumed. The framework also introduces a Privacy Risk Level score, quietly flagging that detailed interaction logs carry their own exposure.\n\nThe proposal is academic for now — 19 student reports is a thin evidence base, and the hardest adoption problem (getting people to share unedited AI conversation histories) goes largely unaddressed. It reads more as a call to reframe the debate than a deployable standard, but that reframe is the right one.","[\"ai\",\"transparency\",\"education\",\"dev-tools\"]","2026-06-30T04:00:00.000Z","2026-06-30T10:49:16.850Z","2026-06-30T10:49:19.557Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The draft omits two KPI scores present in the source (81.9\u002F100 for Prompt Quality and 77.1\u002F100 for Final Output Traceability) while selectively citing others, creating an incomplete and potentially misleading picture of the evaluation results — include all reported scores or note explicitly that only a selection is shown.","resolved","ai",[30,32,33,34],"transparency","education","dev-tools",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.29437",0,{"sections":41},[42,46,51,56,61,66,71,76,81,85,90,94,99,104],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":82,"slug":34,"count":83,"latest_published_at":84},"Dev Tools",59,"2026-07-07T04:00:00.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":89},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":91,"slug":92,"count":88,"latest_published_at":93},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]