A lawyer has built a private AI server from eight used Nvidia Tesla V100 GPUs, so client case files never leave the office. According to the builder's Reddit post, it runs the open DeepSeek V4.1 Flash model at up to 215 tokens per second (a token is roughly a word fragment) on code and up to 118 on legal writing.
The cards are 2018-era data center parts, each with 32GB of fast memory. They sit on two four-card boards, and the cards on each board talk to one another over Nvidia's direct NVLink connection. The two boards only connect through the slower PCIe bus, so the builder split the model's layers in half, one half per board, to keep that hop to one per step. The model is compressed to mostly two bits per weight and still totals 248.8GB. About 110GB of lookup tables sit in ordinary RAM, which the builder says barely affects speed.
The speeds come with limits. At 128K tokens of input, the first word of the answer takes 59.2 seconds, and the server handles one request at a time. Three days before the post, four cards managed only 36 to 38 tokens per second. The builder credits the second board plus custom software work for the jump.
Stability took effort. The host reset twice with a PCIe error, which stopped after the builder capped GPU clocks. Each card now runs at 150W, half its rating. The post also mentions a burn mark on one board's power connector, and memory errors that cleared when the RAM was slowed to DDR4-2666.
The builder estimates 50 to 60 percent of existing workflows can now run locally instead of on frontier models. Nothing goes out without human review.
The cheap-hardware route is closing, though. The four-card boards cost about $2,400 each, but used cards now list for about $900, against roughly $400 in the post, and prices appear to be rising. The R580 driver branch is also the last that supports the V100's Volta architecture, so the hardware is both getting pricier and approaching the end of its software support.