Engineering personality into robots

With stars such as Harrison Ford and Carrie Fisher walking the red carpet at the world premiere of the most recent Star Wars film, The Force Awakens, it would have taken something very special to steal the spotlight. Yet this is exactly what happened when a two-foot-tall robot rolled out onto the red carpet, in front of the world’s media.

Named BB-8, this spherical droid, dreamed up by the film’s director JJ Abrams, was a huge hit with fans. However, most had believed it to be the work of clever computer-generated graphics. The droid’s surprise appearance at the premiere in Los Angeles stunned many by proving it was a real robot; it was science fiction turned into reality.

This little ball-shaped robot highlights how Hollywood is now at the cutting edge of robotics development. The film industry’s demands are almost unique in the robotics world – few other applications require machines to move in lifelike ways, react to the humans around them and, perhaps most importantly, have a personality. BB-8 is a perfect example of how the skills and tenacity of engineers can transform lifeless hunks of electronics and motors into something magical to capture viewers’ hearts. “Basically, we are just given drawings by the concept artists and some guidance from the director, then it is up to us to work out how to make it all work,” explains Lee Towersey, a member of the creature effects team on the new Star Wars films who built many of the droids.

Robots on screen 

Robots have been appearing in films for nearly 100 years. The first known appearance of one on the big screen was in the 1927 German silent movie, Metropolis. However, in this, like so many films that came after, the robot was merely an actor in a suit.

Even in the original Star Wars movies, hailed for their innovative special effects when they were made in the 1970s, the droids C-3PO and R2-D2 were brought to life by humans inside suits. Famously, the late actor Kenny Baker spent hours inside R2-D2 in the original films, wearing the robotic shell almost like a rucksack and controlling the movements from inside.

In The Force Awakens, the R2-D2 that appears on the screen is a real robot. Towersey and his colleague Oliver Steeples were amateur robot builders who were recruited by the production team to use their skills to create a real working R2-D2 droid.

It highlights the drive within the film industry to take advantage of the huge changes in technological capability that have taken place since the original Star Wars movies were made.

“The technology wasn’t really there then and the control mechanisms were not as precise as they are now,” recalls Towersey. “They did try using robots but, towards the end, crew members were having to pull them around on string to get them to move in a straight line.”

This lack of control generally led to robotics being used to bring onscreen monsters to life. Think about the shark bursting from the water in Jaws and the fearsome Tyrannosaurus rex in Jurassic Park. The animatronics used to create the dinosaur in Steven Spielberg’s 1993 movie were cutting edge at the time, but it could essentially do just three things: blink, move its head and open its mouth.

New technology

As servomotors (which allow for precise control of angular or linear position, velocity and acceleration), and perhaps more importantly, the microcontrollers needed to coordinate their actions have improved and grown smaller, the possibilities in robotics have opened up. Suddenly, directors have found that they can get their robotic stars to convey emotion and even personality.

“There was only so much we could do with R2-D2 to give him character,” says Towersey. “He can drive around, we can rotate his head and he can make sounds.

“However, BB-8 is much more interactive. I think a lot of kids who grew up with the prequel films, which had extensive CGI in them, expected BB-8 to be computer generated, so when they saw that it was a real robot moving about, they were blown away.” Computer-generated graphics became a major component of the Star Wars films when director George Lucas decided to make the prequels.

The decision to revert back to so-called ‘practical effects’ (what physical props are called in the special FX world) left the engineers in the creature workshop with a major challenge. Director JJ Abrams had sketched out his idea on a napkin for a ball-shaped robot and asked the team to see what they could come up with. Matt Denton, an electrical software and robotics engineer, and his long-time colleague Josh Lee were the people who ultimately built BB-8.

“We had to figure out how to make it work,” says Denton. “There has never been a round robot with a head on it, mainly because why would you do that? But having a head gave new ways of showing emotion and personality. You can move the body around slowly to point the other way, you can dip the head a little, cock it to one side. There is a huge amount of emotion that effectively comes from three axes of movement.”

There were existing examples of spherical robots on the market. One, built by researchers at Uppsala University in Sweden, features cameras on either side and has been tested by the US Military as a security droid. It works using a pendulum inside the ball that is suspended from an axis. A motor moves the pendulum forward, displacing the centre of mass and causing the ball to roll forward. Moving it from side to side allows it to steer left and right.

The problem Denton and Lee faced was that their BB-8 was going to face some unpredictable and difficult conditions, yet had to stand up to the rigours of a frantic filming schedule. “We had to operate on sand and rough terrain,” explains Denton. “But none of the methods out there would work in all those conditions while also hitting marks over and over again.

“There is a temptation to over engineer it, but sometimes you just have to look for the simplest solution. You are wasting a huge amount of money if the device is broken down and the film crew is waiting for you to fix it. It can be embarrassing if it doesn’t work. In that sense, we were doing fast turnaround prototypes that had to work. Usually,they only have to work for a few weeks but BB-8 had to survive a whole year’s shooting.”

In the end, the pair created eight different versions of the robot: some that could be used in rough outdoor terrain, one that would stay stationary and some that would work inside. The simplest of these was something they nicknamed the ‘wiggler’, which could be bolted to the floor and the ball could twist and tilt while its head could turn around on a shaft through the middle. It was this that allowed the team to get many of the iconic close-up shots that show BB-8 peeking around the door of the Millennium Falcon or sagging in defeat. A separate radio-control unit was used in each version to allow the head to twist and tilt on the body on cue, giving BB-8 its precocious and often comic characterisation.

The off-road versions had stabilisers fixed to the back, turning the robot into a threewheel- drive trike with the head fixed on a bracket, while another version was essentially a wheelbarrow pushed by a puppeteer. They also had a lightweight version that could be carried and a ‘bowling ball’ version that could be thrown into shot but remain upright.

But it was only during an enforced gap in filming, caused by actor Harrison Ford breaking his leg on set, that the pair returned to the idea of building a fully working sphere robot. “We started thinking about what would happen when we got to the red carpet,” recalls Denton. “Everyone would want to see BB-8 but they wouldn’t want to see a static version. That is what drove us to figure out how it was going to work. When we first rolled it out on stage in 2015, no-one was expecting it. There were people in the audience whose jaws just hit the floor.”

Their final robot uses a motor-driven pendulum to propel the beach-ball-sized sphere from the inside to a maximum speed of 4.3 mph. The head ‘floats’ on top of the body so that it can be tilted and turned, giving BB-8 its distinctive, almost childlike personality. It seemed so lifelike that photographers on the red carpet at the premiere called its name to get it to look their way while actor Warwick Davies posed for a selfie with it. Yet BB-8 still requires human help to come alive, with Denton driving its movements by radio control.

However, Denton believes that new technology is starting to take over some of the job of delivering emotion in robotic characters. “We are starting to have control systems that sit on a robotic head and pick up the movement of the actor,” he explains. “So if it is wearing a mask, for example, the control system will take control of the eyes or the face so that it looks in the correct direction and blinks when the actor moves.

“With new microcontrollers I can only see it improving. They allow you to mix the action of all the servomotors in a head into different expressions so you can get some really interesting facial movements. We have also been working on automated blinks or breath cycles,so you can have the robot working away by itself.”

Advances in movement tracking are also allowing engineers to produce robots with eyes that automatically follow other actors in a scene, making them look like they are part of the action.

Denton, who runs his own animatronics company, Micromagic Systems, developed a six-legged robot that works without a puppeteer altogether. An earlier version of his Hexapod robot appeared as one of the weird magical creatures in the Harry Potter movies when it was dressed up like a six-legged turtle.

His updated model uses facial tracking and gesture recognition technology so that it responds to what someone standing in front of it is doing**. Looking a little like a giant spider, it shifts its weight like a predator preparing to pounce as it tracks the movement of the person in front of it. “It is much closer to what I consider to be real robotics,” says Denton.

Robots in the real world

Outside the movie industry, robots have been largely consigned to performing repetitive, mechanical tasks, but as they start to move into people’s homes and everyday life, researchers are taking some lessons from Hollywood.

Dr Cynthia Breazeal from the Personal Robots Group at Massachusetts Institute of Technology has been developing what she describes as sociable robots. It began with an anthropomorphic robotic head called Kismet that could express itself by adjusting its gaze or facial movement but has since progressed into a fluffy 2.5-foot-tall robot called Leonardo, which features a visual tracking system so that it can interact with children. More recently, her team has developed a humanoid robot called Nexi, which has arms that move along with the face to make its body language realistic; with a flick of its eyebrows and clenching of its fists, Nexi can skip from joy into anger and then sadness.

Then there is Pepper, a Japanese creation built by Softbank Robotics, that uses artificial intelligence to give the $1,600 humanoid ‘emotions’ so that it can be used in shops or for customer service. Voice recognition allows it to recognise when it is being chastised and it responds by adopting a crestfallen pose or, if someone is feeling down, it will try to make them happy with unrelenting cheerfulness.

Toyota has also unveiled a diminutive robot called Kirobo mini, which has been designed to be a ‘communication partner’ and has since been touted by others as a companion for childless couples. With large eyes that blink, a high-pitched voice and a wobble as if it hasn’t quite cracked how to balance, it has a vulnerability that its designers say makes it cute. The robot can hold a basic conversation, make hand gestures and respond to human emotions.

There are even some who are hoping robots will become actors in their own right. Researchers at Osaka University in Japan have created an android they call Geminoid F that has been designed to look and behave just like a human. Actuators powered by air pressure beneath its rubber skin allow the robot to copy human facial expressions and mouth voice recordings.

It has already appeared in one film, Sayonara, about the aftermath of a nuclear power plant meltdown. While the performance is a little mechanical, it shows a sign of what may come.

“If you give robots a script and get them to hit their mark, they should be able to do it better than humans as they have ultimate memories,” says Denton. “The problem is that they would never give you nuances in each scene. With artificial intelligence that might be possible but it is a long way off.”

How to engineer a robot

Denton and Lee are now considering publishing the schematics in the future to allow schools and young engineers to build their own BB-8. The robot demonstrates some good engineering principles such as gyroscopic forces, electrical engineering, software and magnetism, and much of the technology needed to build it, such as the motors and gyroscope sensors, are available off the shelf.

One area that they are quiet on the details about is the problem of getting BB-8’s head to stay on top of its spherical body while still letting it move around freely. Although Denton does say that they had obvious help to crack that problem – the Force.

***

This article has been adapted from "Engineering personality into robots", which originally appeared in the print edition of Ingenia 69 (December 2016).