In recent years, thermal imaging technology has seen significant advancements, particularly in applications involving artificial intelligence (AI) integration. A groundbreaking study conducted at Purdue University has led to the development of an innovative thermal imaging system that empowers AI to perceive and navigate through pitch darkness with the same ease as in broad daylight.
This revolutionary advancement has far-reaching implications for various industries, including surveillance, security, search and rescue operations, and autonomous vehicles.
Researchers at Purdue University have developed an innovative method called HADAR (heat-assisted detection and ranging) that enhances traditional machine vision and perception in robotics and autonomy. This patent-pending technology allows automated vehicles and robot helpers to gather information about their surroundings through advanced sensors, enabling them to make decisions without human intervention.
The research, conducted by Zubin Jacob and Fanglin Bao, was featured on the cover of the peer-reviewed journal Nature and has significant implications for the future of autonomous systems. With the projected rise of automated vehicles and robot helpers, HADAR holds the potential to revolutionize how these agents interact with their environments efficiently.
Traditional active sensors like LiDAR, radar, and sonar emit and receive signals to gather 3D information about a scene. However, they face drawbacks when scaled up, including signal interference and eye safety risks.
Video cameras are advantageous in well-lit conditions but struggle in low-light scenarios like nighttime or fog. On the other hand, thermal imaging is fully passive and can sense through darkness and adverse weather conditions. However, fundamental challenges currently limit its widespread use.
Bao said, “Each agent will collect information about its surrounding scene through advanced sensors to make decisions without human intervention. However, numerous agents’ simultaneous perception of the scene is fundamentally prohibitive.”
However, fundamental challenges, such as the “ghosting effect,” hinder its use. It leads to textureless images lacking features, limiting machine perception using heat radiation.
HADAR is an innovative technology that combines thermal physics, infrared imaging, and machine learning to enable fully passive and physics-aware machine perception. It overcomes the traditional bias towards daytime perception. It shows pitch darkness contains the same amount of information as broad daylight.
HADAR can vividly recover texture from cluttered heat signals, accurately distinguish objects’ temperature, emissivity, and texture (TeX) in a scene, and perceive physical attributes beyond standard visible imaging or thermal sensing. This groundbreaking capability allows HADAR to see through pitch darkness like broad daylight, a surprising and revolutionary achievement.
In testing HADAR TeX vision with an off-road nighttime scene, the team successfully recovered textures and overcame the ghosting effect, capturing fine details like water ripples, bark wrinkles, and culverts, as well as information about the environment, such as grassy land. However, areas for improvement remain, particularly in reducing the size and weight of the hardware and increasing data collection speed, as the current sensor’s limitations hinder its application in self-driving cars and robots, which require faster processing.
Initially, HADAR TeX vision will be employed in automated vehicles and robots operating in complex human environments, but its potential reaches far beyond fields like agriculture, defense, geosciences, health care, and wildlife monitoring, showcasing the promise of this technology with further advancements.
Journal Reference:
- Bao, F., Wang, X., Sureshbabu, S.H. et al. Heat-assisted detection and ranging. Nature. DOI: 10.1038/s41586-023-06174-6.