AI-powered code review is shipping faster than evidence of whether it works — so researchers went and found some.
A study of 31,073 CodeRabbit review comments across 10,191 pull requests and 239 GitHub repositories found that developers accepted 36.4% of the AI's suggestions, engaged in discussion on 7.3%, and flatly rejected 56.3%. Rejections clustered around a few recurring problems: false positives, suggestions that were redundant or out of scope, and feedback that clashed with how the team actually writes code. The researchers also found that agentic reviews leaned heavily toward functional issues — bugs, logic errors — rather than longer-term code health concerns, and that those functional comments were the ones most likely to be wrong.
That breakdown matters because agentic review is being sold as a productivity multiplier. If developers spend more time dismissing noise than acting on signal, the tool creates work rather than removing it. The study also tested lightweight machine-learning models to predict which AI comments would be rejected and hit up to 76% F1 score — meaning the patterns of unhelpful feedback are learnable, which is either encouraging or a little damning depending on your read.
CodeRabbit is one of several AI review tools competing for a spot in the pull-request workflow; GitHub's own Copilot code review is chasing the same market. A rejection rate above 50% is not disqualifying on its own — human reviewers leave plenty of noise too — but it is a baseline the vendors will need to argue against.