Researchers found that the type of explanation you attach to a prompt - not just whether you attach one - decides how much it helps a language model.
A team tested natural language explanations across six benchmarks and four instruction-tuned models, comparing human-written rationales, explanations an external LLM generated, and explanations a model wrote about its own answers. On classification tasks, adding explanations to few-shot prompts usually beat plain few-shot prompting, and externally generated LLM explanations held up about as well as human ones when both were available. Self-generated explanations were shakier: their usefulness depended heavily on which examples got selected. Math reasoning tasks did not follow a clean pattern - results varied by model and by where the explanation came from.
This is a useful gut check for anyone building prompting pipelines that lean on chain-of-thought or rationale examples: cheap, LLM-written explanations are apparently not a downgrade from human-authored ones, at least for classification work. It also complicates the popular idea of filtering self-generated explanations for faithfulness before using them - the paper found different faithfulness metrics disagree with each other, so the filter you pick can help or hurt depending on the task.
In other words, before a pipeline swaps in a model's own explanation of itself as a training aid, it is worth asking which faithfulness metric is being trusted and why - the paper suggests that answer is not settled.