Conventional imaging and vision systems require a direct line of sight of the scene of interest. In certain situations, obtaining a direct line of sight can be difficult or even impossible, which can make it challenging for conventional imaging and vision systems to capture the necessary data.
This can be especially true in scenarios like search-and-rescue missions, infrastructure evaluations, and archaeological expeditions, where safety concerns or terrain obstacles may make it difficult to obtain the necessary view.
John Murray-Bruce, a computer science and engineering professor at the University of South Florida (USF), has developed a technology that can reconstruct hidden scenes in 3D using just a single photo. This breakthrough has the potential to reveal what’s behind walls, doors, or cars, even when a direct line of sight is challenging or impossible.
The concept of seeing around obstacles, which has been a popular topic in science fiction for years, is slowly becoming a reality with the latest research. According to Murray-Bruce, this research is a significant step in that direction.
Murray-Bruce and his doctoral student, Robinson Czajkowski, developed an algorithm that can create highly accurate, full-color 3D reconstructions of areas behind obstacles using just one photograph.
“We’re turning ordinary surfaces into mirrors to reveal regions, objects, and rooms that are outside our line of vision,” Murray-Bruce said. “We live in a 3D world, so obtaining a complete 3D picture of a scenario can be critical in a number of situations and applications.”
The research is the first of its kind to successfully reconstruct hidden scenes in 3D using just an ordinary digital camera. Czajkowski and Murray-Bruce’s algorithm works by using information from the photo of faint shadows cast on nearby surfaces to create a high-quality reconstruction of the scene. Even though it sounds quite technical, it could have a big impact on many different fields.
Until now, scientists have only been able to create rough 2D reconstructions of small spaces using ordinary cameras. However, the successful demonstrations of 3D imaging of hidden scenes have all required specialized, expensive equipment.
“Our work achieves a similar result using far less,” Czajkowski said. “You don’t need to spend a million dollars on equipment for this anymore.”
Researchers say the technology may take around 10 to 20 years for it to be adopted by law enforcement and car manufacturers. For now, they plan to continue their research to improve the technology’s speed and accuracy further to expand its potential applications in the future, including enhancing the safety and situational awareness of self-driving cars.
“In just over a decade since the idea of seeing around corners emerged, there has been remarkable progress, and there is accelerating interest and research activity in the area,” Murray-Bruce said. “This increased activity, along with access to better, more sensitive cameras and faster computing power, form the basis for my optimism on how soon this technology will become practical for a wide range of scenarios.”
By making their research available to other researchers, they’re inviting collaboration and innovation in this field. The possibilities for this technology are endless, from enhancing the safety of self-driving cars to aiding law enforcement and military operations. While it may take some time before we see the commercial application of this breakthrough, there’s no doubt that it will greatly impact various industries in the years to come.
Journal reference:
- Robinson Czajkowski and John Murray-Bruce. Two-edge-resolved three-dimensional non-line-of-sight imaging with an ordinary camera. Nature Communications, 2024; DOI: 10.1038/s41467-024-45397-7