A research team built an influencer-matching system that beats a frontier LLM on accuracy while costing a fraction as much to run.
InfluMatch is a three-stage pipeline — retrieve, rerank, reason — built entirely from small open-weight models capped at 4 billion parameters. Dense retrieval pulls 50 candidates, a 4B reranker scores each by the log-probability of a single Yes token and trims the list to 10, and a 4B reasoner then grades that shortlist against Thai marketing criteria with a written rationale. On an 11-query benchmark with all 50 candidates human-labeled, the full cascade reached 94.1% P@5, edging out Kimi-K2.6 at 91.8% — a meaningful gap, though cautious readers will note the test set is small. The system serves a 50-candidate query in roughly 20 seconds on one A100 and emits about 35 times fewer output tokens than the frontier alternative.
The more interesting finding is what fine-tuning does and does not help. A SimPO-tuned reranker matches frontier best-pick accuracy at 78.0 EM — so pairwise fine-tuning pays off. Fine-tuning the reasoner, however, does the opposite: it improves offline scores but degrades end-to-end ranking, an inversion the authors trace to how absolute per-criterion labels were designed. The untuned base model turns out to be the stronger deployed reasoner, which is a useful reminder that offline eval metrics and production performance can diverge badly.
The broader pattern here is familiar: careful pipeline design plus targeted fine-tuning can close most of the gap to frontier models for narrow, well-defined tasks — while cutting serving costs dramatically. Whether that holds outside Thai-language influencer search is the question no one has answered yet.