Engineering swarm robotics

Every winter evening, hundreds of thousands of starlings swarm from across Sussex to roost on Brighton’s piers, forming dark, liquid shapes across the sky. This sight – also visible in many other places in the UK over winter – is a grounding reminder of the natural world’s beauty and power.

Murmurating starlings, as well as swarming insects and schools of fish, are fantastic examples of the inspiration for a branch of engineering called swarm robotics. This is the approach of using large numbers of robots that coordinate with each other to create a moving mass whose behaviour is more than the sum of its parts. 

How swarms turn simple behaviors into complex ones

Swarming is what’s known as an ‘emergent’ behaviour in mathematics. It’s an example of a self-organising system, where simple rules followed by a group of individuals give rise to a larger mass that behaves as one. Importantly, it does not involve any central coordination or instruction. This also means the swarm doesn’t rely on any member playing a different role from the others, such as being a leader (although you can have a replaceable leader, as with a V formation of flying geese). Swarms can therefore cope with the removal or addition of individual members without it majorly impacting the whole.

Computer models of swarms were first developed in the 1980s by Craig Reynolds, a computer programmer from Chicago. While living in Los Angeles, Craig made a computer program about swarming ‘boids’, which simulate a flock of birds. 

“As a child, I was fascinated by what I would now call natural complexity: the turbulent shape of clouds, plant shapes, ant colonies, bird flocks,” says Craig, of his early inspiration. “In college [in the 1970s], I studied programming and graphics, and started thinking about such things through the lens of software simulation models.” 

Craig was inspired by Braitenberg vehicles. These very simple machines were first proposed by (and named after) the neuroscientist, Valentino Braitenberg, in the early 1980s. They move in response to signals (usually a light source) detected by onboard sensors.

“Watching a flock from the outside seemed too complicated. But when I imagined it from a bird’s perspective it seemed much less so. It seemed clear that as a bird, I would want to make small incremental adjustments in my speed and heading,” explains Craig. “If I got too close to a nearby flockmate, I’d want to gently steer away from them. To not get too close in the future, I’d want to steer in roughly the same direction as nearby flockmates. To stay with my flock, if I were on the outside edge, I would steer gently in, toward my nearby flockmates.” As it turned out, complexity could emerge from these surprisingly simple rules.

“I carried that idea in my head for a while. I thought those three rules (what I now call separation, alignment and cohesion) were necessary but wasn’t sure if they were sufficient. In 1986, I finally tried to implement it. Fortunately for me, those three rules produce flocking.”

“The unpredictable, improvisational nature of flock motion was a pleasant surprise to me,” says Craig. “It made the simplistic simulations feel much more ‘alive’ than I expected.” But while the idea sounds relatively straightforward, implementing it had its challenges.

“A problem I did not anticipate was the difficulty of tuning the model’s parameters. It had about 10 parameters, control knobs, to adjust,” he explains. “They all interacted (nonlinearly), so adjusting one always meant adjusting others, which meant adjusting others. It took many, many tests to converge on the desired type of motion.”

While this means essentially any type of motion was possible, it was also deeply complex to do. So, researchers have been automating the process using optimisation techniques, such as algorithms and machine-learning approaches.

Simulating swarms in movies and gaming 

Craig has since spent a large part of his career refining these models, including adding complications such as obstacles for the boids to avoid, or targets for them to reach. His career has led him to work in all sorts of applied areas, from graphics for gaming and the films Tron and Batman Returns, to autonomous vehicles. One example is a school of fish designed for the 1987 short film, Stanley and Stella: Breaking the Ice.

“In feature animation, use of these techniques generally has to do with background action: crowds in cities, herds of animals out in the wild, and battle scenes,” says Craig. “In games, the difference is that everything must operate in real-time (easy [today], quite challenging a decade or two ago) and the members of the crowd (nonplayer characters) usually need to react to the player’s character.”

Craig also developed virtual simulations of cars, technology that would lay the groundwork for that used in autonomous or driverless vehicles. “I was creating the dynamic elements in urban and highway environments, to which the simulated car under test needed to react. This was primarily vehicle traffic on the roads, and to a lesser extent, pedestrians on sidewalks and crosswalks,” he says. 

While there was a degree of crossover with his previous projects, these simulations had to go further still. “The crowds were quite similar to game worlds. The vehicle agents had [very] different ‘locomotion’ styles due to their mechanics: turning radius, stopping distance, and the like, in addition to being much larger and more dangerous.” 

This illustrates one key challenge in the field of swarm robotics: how to bridge the gap between simulation in the virtual world and real robots in the physical world.

Swarm intelligence

Moving swarms from the digital world into a physical environment introduces lots of extra challenges, quickly pushing the limits of these so-called ‘classical’ algorithms. This is especially true if there are many robots involved. 

So, today’s models for swarm robotics are being informed by artificial intelligence (AI) – a field called swarm intelligence. It allows each robot to interact locally with one another (as in the early models), but also for members of the swarm to quickly learn about the surrounding environment and adapt to changes in it.

According to Amanda, researchers will instead pre-train the robot in simulation, and then fine-tune the behaviour of the actual robot in a “safe kind of sandbox-type situation”, such as in the lab. With this approach and the help of clever algorithms, robotics labs around the world are making solid progress towards addressing the many technical challenges that swarm robotics face.

That’s all well and good for individual robots, but how does it map onto swarm control? As Amanda explains, a decentralised model is most like natural swarms seen in nature. In such a scheme, there’s no overarching, ‘mastermind’-style control system. Each robot governs its own behaviour, with input only from nearby robots. This is how it works in the Prorok Lab’s research, too – each robot in a swarm is responsible for its own behaviour.

Professor Amanda Prorok at the University of Cambridge uses neural networks to train robotic swarms. She explains the key training paradigms: imitation learning and reinforcement learning. “In imitation learning, you show the robot what it’s supposed to do in given scenarios … and the robot learns to copy it. And in reinforcement learning, you reward the ideal behaviour when the robot gets something right. For example, if you want a robot to find an exit of a maze, you can tell that robot, ‘Hey, when you find that exit, I’m going to reward you.’ Over time, it learns the right behaviour.”

This ‘training’ would take too long in the physical world, so the robots are often ‘trained’ in a simulated, computer world, and then this learning is transferred to the physical robots. But this leads to another problem.

“As we go about deploying our robots in the real world, we encounter something called the simulation-to-reality gap. Because the world that the robot encountered in simulation is not the same [as] the real world,” she says. “No matter how hard you try, it’s very hard to create photo-realistic environments in simulation with the right kind of lighting and everything … so when you give the robot its sensor, it’s suddenly encountering sunlight, and cloudy skies … and it doesn’t know how to deal with those conditions.” This particular case applies to robots with camera sensors. Other types of sensors, for example sonar or infrared, can suffer from other distortion factors. 

Real-world swarms

There are many exciting real-world applications for swarm robotics. Militaries have made some of the most notable advances in the area. US government agencies, such as the Defense Advanced Research Projects Agency (DARPA) and the Navy, are investing in developing swarms of uncrewed aerial vehicles and boats. Ingenia has explored this further in 'Adding value through maintenance'. 

Civilian applications are also taking flight. Search and rescue missions often demand access to difficult-to-reach places, exploring unknown environments and solving complex geometrical problems (such as systematically searching a collapsed building). Swarms of small flying or ground-based robot systems are well-suited to such tasks. London-based startup Unmanned Life is one company developing AI software to manage drone swarms for search and rescue tasks, as well as for commercial security such as in ports. Similarly, Southampton-based Windracers is working on autonomous drone swarm technology to detect and fight wildfires. 

In a project led by Oregon State University, researchers are developing swarms to explore hard-to-reach underwater polar environments, where communications with the surface are limited. They hope that sending them to places such as the cavities underneath ice shelves will shed light on how ice melt contributes to sea level rise. 

Autonomous swarms have even been suggested for manufacturing, known as swarm 3D printing, and medicine, with microscopic swarms that could deliver pharmaceutical or surgical interventions inside the body.

However, before these exciting applications materialise, engineers must address key challenges in the field such as keeping costs down, poor battery life and miniaturising the individual robots. Researchers share the goal of keeping the individual robots as simple and cheap as possible, to maximise scalability and power efficiency. Specific challenges such as programming adjustments in flight for drones blowing each other off-course also plague researchers, but programming involving AI is helping refine these systems.

AI, too, comes with its own problems. “I think one thing that people don’t really realise is that the larger the models are, the less interpretable they are, and the less we can verify and certify their behaviour. So it’s really difficult for us to guarantee what the model’s going to do in certain situations,” says Amanda.

She draws a parallel with large language models such as ChatGPT, and the unexpected ways they can behave. “It’s the same thing with robots. But the thing with robots is, now we’re not talking about virtual harm, we’re talking physical harm, right? They’re moving in a physical world. Their actions are not words, their action is motion. So, I think we have to be a bit careful about that.”