Encoder models that power clone detection and code classification tools turn out to be easily fooled by code that does the exact same thing but looks different on the page.
Researchers behind a new arXiv paper tested four encoder baselines across two downstream tasks and four datasets, feeding them "invariant programs": functions that are semantically equivalent but written in different syntactic forms. The representations these models produce degrade substantially on the rewritten versions, even though the underlying program behavior hasn't changed at all. To fix this, the team built invariant pretraining (InvPT), a continued pretraining recipe that applies semantic-preserving transformations to code and combines masked language modeling with a multi-positive contrastive learning setup, treating different augmentations of the same function as matching examples. The method needs no paired natural-language data, unlike prior contrastive code encoders.
This matters because clone detection and code classification are used for things like plagiarism checks, license compliance scanning, and vulnerability triage in real codebases, where attackers or just ordinary refactoring can trivially change syntax without changing behavior. A model that loses up to 19 percentage points of accuracy on code classification just because variables got renamed or a loop got rewritten is a model that can be gamed or simply misfires in production.
The gains are real but the framing is honest: this isn't a new architecture, just a measurement of how brittle these small, cheap-to-run encoders already were, and a demonstration that a fairly simple, code-only pretraining tweak recovers most of the lost ground.