Cameras can show you a garbage dump; they cannot tell you it reeks. A research framework called SCENT tries to fix that gap.
The system, described in a new paper, trains a smell encoder using signals from electronic noses — sensor arrays that measure chemical compounds in the air. The trick is using a vision-language model as a go-between: it generates text descriptors from images, noting objects, environmental context, and likely ambient odors a scene would produce. Those text descriptions then guide the smell encoder, pulling its outputs into a shared embedding space alongside visual and textual data. The team also introduced a decomposition step that separates odors tied to specific objects from broader environmental smells — think coffee versus "damp basement."
Tested on the New York Smells dataset, SCENT beat vision-only baselines on smell-to-image and smell-to-text retrieval, a task where the system matches a sensor reading to the scene that produced it. That matters because most multimodal AI research treats sight, sound, and text as the only senses worth modeling — smell has been a stubborn outlier, since the same visual scene can smell entirely different depending on weather, time of day, or what happened there an hour ago.
Practical applications — food safety, environmental monitoring, accessibility tools for people with visual impairments — are plausible but still a long way off. For now, this is a proof of concept that language, oddly enough, may be the best handle we have on a sense that resists direct digitization.