Researchers have built a local-first visual assistant for blind and low-vision users that runs without sending footage to the cloud.
OpenGlass pairs an ESP32-based glasses unit with a nearby consumer device running a local multimodal large language model. The glasses capture images; the local device handles inference and speaks the response. Under real Wi-Fi conditions, the system hit a median latency of 993 ms with compressed images and 1,625 ms with full 1280x720 frames - 97.5% and 93.3% of trials respectively finished within two seconds. The team evaluated response quality, safety-aware abstention, and auditable logs, and released all source code, hardware instructions, and evaluation data.
Most cloud-based visual assistants require streaming first-person footage to remote servers, which raises obvious privacy concerns for a device worn all day at eye level. OpenGlass keeps raw egocentric data on user-controlled hardware by default - a meaningful distinction for a population that depends on continuous environmental awareness and may have limited ability to audit what gets uploaded. The split architecture also sidesteps the power and compute constraints that make running a large model directly on glasses impractical today.
The project positions itself explicitly as a reference platform for obstacle awareness and sign queries, not a certified navigation aid - a disclosure that most commercial wearable AI products tend to bury in fine print.