GLM's maker decided serving its own models was worth building from scratch.
Z.ai, the company behind the GLM family of language models, published a technical post detailing why it moved off third-party inference providers and built its own serving infrastructure. The post walks through the tradeoffs the team weighed: relying on external GPU-serving vendors is simpler to start with, but it caps how much a lab can optimize for its own model architecture and traffic patterns. Z.ai says the switch let it tune scheduling, batching, and hardware utilization specifically for GLM's workloads rather than settling for whatever a generic serving stack offers. The result, per the post, is infrastructure shaped around GLM's actual inference traffic instead of a one-size-fits-all setup.
Inference is the bill that never stops arriving, unlike training, which is a one-time cost per model version. Building serving infrastructure in-house is a bet that the cost and performance gains outweigh the engineering headcount it takes to run it, a tradeoff most labs still hand off to third-party GPU-serving vendors. Z.ai's move suggests GLM's traffic has scaled to the point where owning that infrastructure now pencils out.
It's the same threshold every infrastructure-heavy AI company eventually crosses: renting compute is fine until your own scale makes owning it cheaper.