[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-meta-now-runs-ai-inference-on-google-tpus-via-pytorch":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},12120,"meta-now-runs-ai-inference-on-google-tpus-via-pytorch","Meta Now Runs AI Inference on Google TPUs, Via PyTorch","Google's AI chips now serve production traffic at Meta through PyTorch-native software, and Amazon made a similar move for its own chips a day earlier.","Google's tensor processing units (TPUs), its custom AI chips, are now serving AI models in production at Meta. They run through a PyTorch-native version of the SGLang inference engine, the software that answers queries once a model is trained.\n\nGoogle announced the work on its developer blog, along with a matching open-source version of vLLM, another widely used inference engine. Until now, both engines reached TPUs through Google's own JAX framework. The new versions make TPUs a native target for PyTorch, Meta's home framework, mostly without rewriting code. \"SGLang-TorchTPU is running in production at Meta today,\" said Zongwei Zhou, Meta's PyTorch-TPU lead. Jana van Greunen, a director of PyTorch engineering at Meta, said the company \"cannot be constrained by a single hardware architecture.\"\n\nThis is a second attempt. An analyst firm reported last year that Meta research teams used TPUs through an older bridge, PyTorch\u002FXLA, from 2020 to 2023 before Meta canceled those contracts. Talks last year reportedly covered TPU rentals in 2026 and purchases in 2027, and Meta later reportedly signed a multi-year rental deal. Where Meta's production TPUs actually sit is unclear.\n\nThe performance claim comes from Inferact, the startup founded by vLLM's creators that helped build the new version. It said 16 Ironwood TPUs, Google's seventh generation, served Moonshot AI's 2.8-trillion-parameter Kimi K3 at more than 700 tokens per second for a single user, using speculative decoding (a small draft model that speeds up a larger one). Published vLLM numbers for 16 Nvidia GB200s reached 452. Treat it as a vendor-adjacent comparison of a tuned setup against a published baseline.\n\nThe timing matters more than the benchmark. Amazon said a day earlier that it was open-sourcing TorchNeuron, a native PyTorch backend for its Trainium2 and Trainium3 chips. Nvidia's lead rests partly on CUDA, its software layer, and most developers write PyTorch. Two rivals now aim at that layer, and the porting cost is what they are attacking.\n\nOne production deployment at one customer is not a market shift. The next markers are Day 0 support for new open models by the first quarter of 2027, and what Google, Meta and RadixArk share at SGLang Summit in San Francisco on November 12 and 13. The open question is how much of Meta's traffic ends up on TPUs this time.","[\"tpu\",\"pytorch\",\"nvidia\",\"ai-inference\"]","2026-10-22T10:00:00.000Z","2026-10-22T10:29:39.440Z","2026-10-22T10:29:41.795Z","published",null,[],"hardware",[26,27,28,29],"tpu","pytorch","nvidia","ai-inference",[31],{"name":32,"url":33},"Tom's Hardware","https:\u002F\u002Fwww.tomshardware.com\u002Ftech-industry\u002Fartificial-intelligence\u002Fgoogles-tpus-for-ai-workloads-now-run-in-at-metas-data-centers-tpus-sold-to-external-customers-for-the-first-time-threaten-nvidias-market-dominance",0,{"sections":36},[37,42,47,52,57,61,66,71,76,81,86,91,96,101],{"name":38,"slug":39,"count":40,"latest_published_at":41},"AI","ai",7058,"2026-10-22T11:25:00.000Z",{"name":43,"slug":44,"count":45,"latest_published_at":46},"Security","security",1037,"2026-10-22T11:43:09.000Z",{"name":48,"slug":49,"count":50,"latest_published_at":51},"Policy","policy",567,"2026-10-22T10:43:13.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":56},"Deals","deals",554,"2026-10-22T09:15:13.000Z",{"name":58,"slug":24,"count":59,"latest_published_at":60},"Hardware",284,"2026-10-22T12:05:31.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Consumer Tech","consumer-tech",278,"2026-10-22T10:46:15.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Science","science",212,"2026-10-21T22:13:50.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Startups","startups",141,"2026-10-22T11:56:11.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Software","software",138,"2026-10-22T04:52:10.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Dev Tools","dev-tools",122,"2026-10-21T16:00:00.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",95,"2026-10-22T09:31:41.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Gaming","gaming",85,"2026-10-21T18:39:52.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"Reviews","reviews",38,"2026-10-20T12:00:00.000Z",{"name":102,"slug":103,"count":104,"latest_published_at":105},"How-To","how-to",8,"2026-10-05T09:00:00.000Z"]