First-person view (FPV) drone racing has become a popular televised sport where professional pilots fly high-speed drones through a 3D circuit. During a race, drone-racing pilots fly quadcopters at speeds exceeding 100 km/h, controlling them remotely while wearing headsets that give them a ‘first person’ view through a camera attached to the drone.
However, achieving the level of professional pilots with an autonomous drone is challenging, as the robot needs to fly at its physical limits while estimating its speed and location in the circuit solely from onboard sensors.
Now, researchers from the University of Zurich and Intel have set a new milestone with the first autonomous system capable of beating human champions at the world’s best in a real-world physical sport: drone racing.
An AI-controlled drone system called Swift quickly learned a tight, technical 3D racetrack and proceeded to win the race against human world champions. The system has triumphed over three human champions, including the world champions of two international leagues, in a series of intense head-to-head races.
Swift showcased its exceptional capabilities by winning races against each of the human champions and demonstrated the fastest recorded race time. Combining AI-learning algorithms, onboard sensors, and a single camera, Swift outmaneuvered human pilots in 15 of the 25 races on a track full of sweeping turns and screeching pivots designed by a professional drone-racing pilot.
Unlike its predecessors, which relied on inputs from external motion cameras, the Swift system used the same single-camera vision setup as the human pilots to see its way around the course and detect the gates along the racetrack. It also had the advantage of using real-time acceleration, speed, and orientation data from an onboard inertial measurement unit. This data is fed to a control unit based on a deep neural network that chooses the best action to finish the circuit as fast as possible.
It learned the fairly complex track covered an area of 25 by 25 meters, with seven square gates that had to be passed in the right order to complete a lap. This includes challenging maneuvers like a Split-S, an acrobatic feature that involves half-rolling the drone and executing a descending half-loop at full speed.
Overall, Swift achieved the fastest lap, with a lead of half a second over the best lap achieved by a human pilot, according to the researchers. However, human pilots demonstrated more adaptability than the autonomous drone, which failed when operating in conditions that were different from what it was trained for.
In certain applications like forest monitoring or space exploration, flying fast is important to cover large spaces in a limited time. In the film industry, fast autonomous drones could be utilized to shoot action scenes. In emergency situations, the ability to fly at high speeds could prove to be a game-changer for rescue drones sent to navigate through a burning building.
Journal reference:
- Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, Davide Scaramuzza. Champion-Level Drone Racing using Deep Reinforcement Learning. Nature, 2023; DOI: 10.1038/s41586-023-06419-4