An Android app built for people with visual impairments puts six deep learning models on the phone itself — no internet required for the core features.
Researchers present VisionAId, which runs depth estimation, instance segmentation, facial recognition, and a custom banknote detector entirely on-device via ONNX Runtime. On a Samsung Galaxy S21 Ultra, INT8 quantization cuts depth-sensing latency from roughly 1,200 ms to 491 ms, and the banknote detector hits a mean average precision of 0.986 at the 50 percent IoU threshold. The standout feature is a few-shot personal object retrieval pipeline: a user photographs an item from several angles, and the app later tracks that specific object in the environment, directing the user through augmented-reality overlays, spatial audio, and distance-proportional haptics. Cloud access to Google Gemini Flash is available but optional, used only for scene narration and automatic labeling.
The offline-first design addresses a real gap. Most existing assistive apps lean on cloud inference or require specialized hardware — neither works well in areas with spotty connectivity or for users who cannot afford dedicated devices. Squeezing competitive accuracy out of a commodity smartphone changes the cost and access equation meaningfully for the 285 million people worldwide living with visual impairment.
Before VisionAId reaches that audience at scale, the researchers would need to move beyond Romanian speech synthesis to broad multilingual support, clear a path through accessibility certification on Android, and demonstrate the latency numbers hold on mid-range handsets rather than a flagship S21 Ultra.