An independent developer trained a 3.8 billion parameter language model from scratch for $998 in compute, then published the full recipe.
Hugo Vergnes detailed the project on his personal site: architecture choices, training data, and a final CORE benchmark score of 0.384. The post walks through the budget line by line, down to the compute hours that made up the $998 total. It picked up a modest 11 points and a single comment when it circulated online. There's no accompanying paper or peer review, just a blog post and the code.
That price tag matters more than the score. Training runs at this scale used to require serious institutional backing; a $998 budget suggests the floor for "toy" pretraining experiments keeps dropping. But the post doesn't say what a competitive CORE score looks like for a model this size, so there's no way to tell from the source alone whether 0.384 is impressive or just serviceable.
Cheap and reproducible isn't the same as competitive. Until someone runs the same recipe against a same-sized baseline, the real headline here is the price, not the performance.