Wireless earbuds are everywhere, yet they’re still like tiny stage performers trapped in a single act,
sound. Their small size and limited power keep them tuned to audio alone, preventing them from branching out into bigger roles.
University of Washington researchers developed VueBuds, the first system to embed tiny cameras in commercial wireless earbuds, enabling users to converse with an AI model about whatever is in front of them.
VueBuds are Bluetooth-enabled earpieces that take speedy, grainy black-and-white photos and then upload them to your phone. The artwork is a tiny on-device AI model that can answer questions about the images it sees in about a second. All processing is performed locally to protect privacy. Recording activates a light, and users can delete images immediately.
Wireless system to power devices inside the body
Each VueBud is a Sony WF‑1000XM3 that’s been taught to “see.”
UW professor and senior author Shyam Gollakota said, “We haven’t seen most people adopt smart glasses or VR headsets, in part because a lot of people don’t like wearing glasses, and they often come with privacy concerns, such as recording high-resolution video and processing it in the cloud. But almost everyone wears earbuds already, so we wanted to see if we could put visual intelligence into tiny, low-power earbuds, and also address privacy concerns in the process.”
High‑res cameras and endless video drain too much power for earbuds. So the team miniaturized vision into a “grain‑of‑rice” camera that snaps quick, low‑res black‑and‑white stills, light enough for Bluetooth and battery to handle.
By angling the cameras just 5–10° outward, VueBuds capture nearly everything ahead, leaving only a small blind spot for objects closer than 20 cm, a distance people rarely use.
Keeping data fresh for wireless networks
They also learned to stitch the images of both earbuds into a single image, cutting response time to a single second and making the experience feel live.
In trials with 74 people, VueBuds’ low-resolution yet private still images performed as well as Ray-Ban Meta Glasses, powered by multiple clouds for non-private storage. If it were a translation task, users preferred VueBuds, while Ray‑Bans worked better for object counting.
In another study with 16 testers, VueBuds reached 83–84% accuracy on everyday translation and object tasks, and 93% when identifying book titles and authors.
Because they only capture grayscale images, VueBuds can’t yet handle color‑based questions. The team hopes to add color cameras, though they require more power, and train specialized AI models for tasks like translation.
“This study lets us glimpse what’s possible just using a general-purpose language model and our wireless earbuds with cameras,” Kim said. “But we’d like to study the system more rigorously for applications like reading a book, for people who have low vision or are blind, for instance, or translating text for travelers.”
Journal Reference:
- Maruchi Kim, Rasya Fawwaz, Zhi Yang Lim et al. VueBuds: Visual Intelligence with Wireless Earbuds. DOI: 10.1145/3772318.3791322
