[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-training-method-forces-ai-translators-to-actually-use-images":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},8053,"training-method-forces-ai-translators-to-actually-use-images","Training Method Forces AI Translators to Actually Use Images","A new fine-tuning method weights translation loss toward image-relevant words, lifting accuracy over 7 points on a contrastive benchmark test.","Researchers have found a way to make AI translation models actually pay attention to the images meant to disambiguate their text, and it works significantly better than existing fixes.\n\nThe technique, called Metric-based Loss Weighting, increases the training penalty for words whose correct translation depends on an accompanying image. To find those words, the researchers compare a model's output probabilities with and without visual input, using a measure called Point-wise Cross-mutual Information, or PCXMI. They also built a second, congruency-based version of that metric, and found combining both approaches worked best. The team fine-tuned three pretrained multimodal LLMs across three language directions to test the method on image-guided translation.\n\nMultimodal translation systems have long had a habit of accepting images as input while quietly ignoring them, since text alone is often enough to produce a plausible translation. That makes benchmarks built specifically to test visual grounding, like the CoMMuTE contrastive dataset used here, more revealing than standard translation scores. On CoMMuTE, the new method beat standard fine-tuning by more than 7 percentage points in accuracy, without dragging down general translation quality.\n\nIt is a narrow fix for a narrow failure mode, but it is a useful reminder that giving a model extra data is not the same as making it use that data.","[\"multimodal-ai\",\"machine-translation\",\"llms\",\"research\"]","2026-09-28T04:00:00.000Z","2026-09-28T07:33:44.862Z","2026-09-28T07:33:56.313Z","published",null,[],"ai",[26,27,28,29],"multimodal-ai","machine-translation","llms","research",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.31169",0,{"sections":36},[37,40,44,49,54,59,63,68,73,78,83,88,92,97],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",4750,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",759,{"name":45,"slug":46,"count":47,"latest_published_at":48},"Policy","policy",399,"2026-09-27T18:39:02.000Z",{"name":50,"slug":51,"count":52,"latest_published_at":53},"Deals","deals",261,"2026-09-27T15:30:35.000Z",{"name":55,"slug":56,"count":57,"latest_published_at":58},"Hardware","hardware",188,"2026-09-27T20:46:36.000Z",{"name":60,"slug":61,"count":62,"latest_published_at":18},"Science","science",151,{"name":64,"slug":65,"count":66,"latest_published_at":67},"Consumer Tech","consumer-tech",135,"2026-09-26T14:30:00.000Z",{"name":69,"slug":70,"count":71,"latest_published_at":72},"Software","software",91,"2026-09-25T20:55:00.000Z",{"name":74,"slug":75,"count":76,"latest_published_at":77},"Dev Tools","dev-tools",84,"2026-09-26T04:20:58.000Z",{"name":79,"slug":80,"count":81,"latest_published_at":82},"Startups","startups",76,"2026-09-25T18:33:59.000Z",{"name":84,"slug":85,"count":86,"latest_published_at":87},"Gaming","gaming",48,"2026-09-25T18:35:21.000Z",{"name":89,"slug":90,"count":86,"latest_published_at":91},"General","general","2026-09-26T17:02:42.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"Reviews","reviews",30,"2026-09-24T20:07:31.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]