Conventional silicon architecture has taken computer vision a long way; now, researchers at Purdue University are exploring a new path toward computer vision by taking inspiration from nature.
Purdue researchers have developed a prototype device that takes inspiration from light perception in retinal cells. The light triggers an electrochemical reaction in the prototype device, which strengthens steadily and incrementally with repeated exposure to light and fades slowly when light is removed, effectively creating a memory of the light information it has received. This memory could be used to reduce the amount of data that needs to be processed to understand a moving scene, which is a more energy-efficient and computationally efficient approach than traditional computer vision.
According to the researchers, the organic electrochemical photonic synapse they developed closely mimics the functioning of the human visual system, making it a promising foundation for human-machine interfaces. The design may also prove useful in neuromorphic computing applications that follow the architecture of the human brain.
“In a normal computer vision system, you create a signal, then you have to transfer the data from memory to processing and back to memory; it takes a lot of time and energy to do that,” Ke Chen, lead author of a paper that tested the device on facial recognition. “Our device has integrated functions of light perception, light-to-electric signal transformation, and on-site memory and data processing.”
The current robotic or autonomous devices rely on digital cameras for computer vision. The camera contains light-sensitive areas of crystal silicon called photosites. These photosites absorb photons and release electrons, converting light to an electrical signal that can be processed with sophisticated computer image recognition programs. A smartphone uses more than 10 million photosites, capturing images with far higher resolution than our own eyes can do.
However, this data analysis isn’t necessary for many of the tasks that use computer vision. In contrast, Purdue’s solution is comparable to human vision, which is well-suited for sensing movement and has relatively low resolution. The prototype device houses 18,000 transistors on a 10-centimeter square chip. It has a resolution of a few hundred microns, and the technology could be improved by lowering the resolution to about 10 microns.
“Our eye and brain aren’t as high resolution as silicon computing, but the way we process the data makes our eye better than most of the imaging systems we have right now when it comes to dealing with data,” said Jianguo Mei, the Richard and Judith Wien Professor of Chemistry in Purdue’s College of Science. “Computer vision systems deal with a humongous amount of data because the digital camera doesn’t differentiate between what is static and what is dynamic; it just captures everything.”
Mei and his team have developed a unique method to convert light into an electrical signal. Instead of directly converting light to an electrical signal, they convert light to a flow of charged atoms known as ions. This mechanism is similar to that used by retinal cells to transmit light inputs to the brain.
To achieve this, they use a small square of a light-sensitive polymer that is embedded in an electrolyte gel. Whenever light hits a particular spot on the polymer square, positively charged ions in the gel are attracted to the spot, while negatively charged ions are repelled, creating a charge imbalance in the gel.
Repeated exposure to light increases the charge imbalance in the gel, which can be utilized to differentiate between the consistent light of a static scene and the dynamic light of a changing scene. The ions remain in their charged configuration for a short period of time, even after the light is removed. This can be considered as a temporary memory of light, which gradually returns to a neutral configuration.
The positively charged spot serves as a gate on a transistor, permitting a small electric current to flow between a source and a drain when light is present. Much like a conventional photodetector, this electric current indicates light intensity and wavelength and is sent to a computer for image recognition. However, what sets Mei’s electrochemical transistor apart is that it has motion sensing and memory capabilities due to the intermediate step of converting light to the electrochemical signal.
Mei’s device is part of a new generation of optoelectronic devices that aim to integrate light perception and memory. Compared to other designs, their device exhibits superior performance as the charge imbalance increases in smooth, steady increments with repeated light exposure and decays slower.
“Computer vision systems use a huge amount of energy, and that’s a bottleneck to using them widely. Our long-term goal is to use biomimicry to tackle the challenge of dynamic imaging with less data processing,” Mei said in a statement. “By mimicking our retina in terms of light perception, our system can be potentially much less data-intensive, though there is a long way ahead to integrate hardware with software to make it become a reality.”
Future iterations of this device will be made of a flexible material, which could result in a wearable version that is even bio-compatible.
Journal reference:
- Ke Chen, Hang Hu, Inho Song, Habtom B. Gobeze, Won-June Lee, Ashkan Abtahi, Kirk S. Schanze, and Jianguo Mei. Organic optoelectronic synapse based on photon-modulated electrochemical doping. Nature Photonics, 2023; DOI: 10.1038/s41566-023-01232-x