Predictive touch: A no-touch touchscreen technology

AI-based ‘no-touch touchscreen’ could reduce the risk of pathogen spread from surfaces.


Engineers at the University of Cambridge, as part of a research collaboration with Jaguar Land Rover, have devised a technology called predictive touch, a no-touch touchscreen for use in cars. The technology uses a combination of artificial intelligence and sensor technology to predict a user’s intended target on touchscreens and other interactive displays or control panels, selecting the correct item before the user’s hand reaches the display.

In lab-based tests, driving simulators, and road-based trials, the prescient touch technology successfully reduced interaction effort and time by up to half because of its capacity to anticipate the client’s intended target with high exactness early in the pointing task.

The technology utilizes machine knowledge to decide the item the client expects to choose on the screen early in the pointing task, accelerating the interaction. It uses a gesture tracker, including vision-based or RF-based sensors, which are progressively normal in customer electronics; contextual information, for example, user profile, interface design, environmental conditions; and data available from different sensors, for example, an eye-gaze tracker, to derive the user’s intent in real-time.

As lockdown restrictions around the world continue to ease, the scientists say the technology could also be useful in a post-COVID-19 world. Eliminating the need actually to touch a touchscreen or other interactive display could reduce the risk of spreading pathogens – such as the common cold, influenza, or even coronavirus – from surfaces.

Professor Simon Godsill from Cambridge’s Department of Engineering, who led the project said, “Touchscreens and other interactive displays are something most people use multiple times per day, but they can be difficult to use while in motion, whether that’s driving a car or changing the music on your phone while you’re running. We also know that certain pathogens can be transmitted via surfaces, so this technology could help reduce the risk for that type of transmission.”

Lee Skrypchuk, Human Machine Interface Technical Specialist at Jaguar Land Rover, said, “This technology also offers us the chance to make vehicles safer by reducing the cognitive load on drivers and increasing the amount of time they can spend focused on the road ahead. This is a key part of our Destination Zero journey.”

“It could also be used for displays that do not have a physical surface such as 2D or 3D projections or holograms. Additionally, it promotes inclusive design practices and offers additional design flexibilities, since the interface functionality can be seamlessly personalized for given users and the user ability no longer constrains the display size or location to reach-touch.”

Dr. Bashar Ahmad, who led the development of the technology, said“Our technology has numerous advantages over more basic mid-air interaction techniques or conventional gesture recognition because it supports intuitive interactions with legacy interface designs and doesn’t require any learning on the part of the user. It fundamentally relies on the system to predict what the user intends and can be incorporated into both new and existing touchscreens and other interactive display technologies.”


See stories of the future in your inbox each morning.