A year-long experiment wrung state-of-the-art AI inference out of GPUs that were headed for the scrap heap.
Researchers built a 128-GPU cluster, nicknamed DumpsterCluster, entirely from second-hand parts bought off the secondary market, then ran it in production for a full year. Individual V100 GPUs reportedly cost as little as $60 each, and the whole rig totaled about $22,000, versus roughly $600,000 for a modern 8-GPU B200 system. Using pipeline-parallel optimizations, the team says the setup delivered LLaMA-70B inference throughput competitive with current-generation hardware.
The economics look great until you check the power bill. Older GPUs burn far more energy per token, so the cost advantage only holds in regions with cheap electricity. Under average grid carbon intensity, the researchers calculated second-hand systems emit roughly 4x more carbon per token for 8-billion-parameter models, and more than 40x for 70-billion-parameter models, compared to new hardware.
Cheap silicon can stretch AI budgets further, but only where the grid is clean - otherwise the savings just move the pollution somewhere less visible.