Hands as machines

As a device for seizing, holding and manipulating objects of all manner of shapes and sizes and with varying degrees of fragility, the human hand is almost unrivalled. Recent years have witnessed the birth of a mechanical challenger. Called the Dexterous Hand, its parent is the Shadow Robot Company, based in Gospel Oak, London. To watch one of these versatile hands unscrewing a bottle top, picking up an egg, or removing a credit card from a wallet is to witness something obviously mechanical, but uncannily familiar.

Entertaining, of course, but there is more to these robotic hands than the performance of party tricks. Set up as a company in 1997, Shadow sold its first robotic hand some three years later. An early customer was the Neuroinformatics Group at Bielefeld University in Germany. The university found that the hand’s human-like design served as the perfect test bed to devise, evaluate and improve the computational models of human manual intelligence on which they work. They have been buying the mechanical hands ever since.

Although many of Shadow Robot’s current customers are academics or otherwise involved in R&D, the company’s managing director Rich Walker foresees the sale of more and more hands intended not only to develop new ways of doing things, but actually to do them. Suppose the task to be carried out is in a toxic environment, or one that has to be sterile, or one contaminated by noxious fumes or otherwise too risky to allow the presence of a human. Suppose it involves dismantling a bomb, or working in deep space, or experimenting on lethal microbes. Or suppose the environment is safe but the person on whose behalf the robotic hands are acting is incapacitated.

In the pharmaceutical industry, for example, many of the tasks carried out by staff working in sterile areas are quite menial: taking a Petri dish and spreading a sample to be cultured across the growth media in it; or moving trays from one part of the lab to another. Although many such tasks have been automated, some still require the intervention of human hands – and training humans to maintain sterile conditions is hugely time consuming.

Shadow is also talking to people in the nuclear industry in the UK and Japan, not so much in connection with heavy engineering tasks but for analytics labs where staff are often working with radioactive and otherwise contaminated and toxic materials.

The design behind the robot hand

The principle underpinning the design of the current generation of Dexterous Hands is precisely what it was at the outset: a close copy of the human hand – in size, shape, overall appearance and function. The materials that it is fabricated from – aluminium, steel, delrin, polycarbonate, and polyurethane ‘flesh’ – are of course non-biological, but otherwise the differences are minor. The robot hand doesn’t, for example, imitate some of the more complex movements of the thumb. It could have been designed to do so, but Walker and colleagues felt that the gains would not justify the effort. Likewise with the movement of the outermost joint of the fingers. Some of us can move this independently of the penultimate joint; but for most of us, and for most purposes, movements of the two outermost finger segments tend to be coupled. And so they are in the Dexterous Hand. However, in all other respects anything your hand can do, so can the Dexterous Hand.

The Dexterous Hand has 24 joints driven by 20 individual motors; the difference in number reflecting the already mentioned decision to allow movements of the outermost finger joint to be actuated by those of the joint proximal to it. The 20 motors and their associated gearing are located in the ‘forearm’: the circular base on which the hand is mounted. The motors are linked to the joints and actuate them by cables; they can generate forces comparable to those of a human hand. The fingers have three degrees of freedom while the thumb has four. The hand and forearm together weigh 4.3 kilograms and can hold objects as heavy as 5 kilograms.

The Dexterous Hand can sense the force it is applying to whatever it grips by pressure sensors built into the surface of each figure tip. Extension of the sensing system to other areas of the hand is still a work in progress, but data on finger pressure alone is more than adequate for achieving and controlling most of the tasks it can undertake. The pressure being applied by the finger can also be inferred from strain gauge measurements of the tension in the cables that actuate joint movements.

Hand movements can be pre-programmed if the task to be performed is routine and repetitive. But more commonly the control system relies on signals from an operator wearing a motion-tracking glove. When Shadow began operating, the choice of such gloves was limited, but in recent years the technology has significantly improved.

To make practical use of the actions of a robotic hand it will usually need to be attached to some other piece of machinery, typically a robotic arm. Integrating the two sets of software and systems is not usually a problem, especially for other equipment that uses the Robot Operating System (ROS), an open-source set of software libraries and tools that Shadow has been using for the past 15 years. During that time, according to Walker, manufacturers of equipment like robot arms have become increasingly aware of the value of compatibility with the products of other companies.

The future of robots and integration with machine learning

An idea of what the future might hold can be gleaned from a 2019 demonstration that Shadow set up in collaboration with two Californian companies: SynTouch that specialises in manufacturing tactile sensors and HaptX, which makes haptic feedback gloves. An operator in California wearing a haptic glove used it to control an arm-mounted Dexterous Hand typing on a keyboard in London. As the tactile sensors on the robot’s fingertips responded to the press of each key, haptic data travelled over a network back to the human operator in California who, 5,000 miles away, could feel the pressure in real time that he (or rather the robot) was applying to the keys.

Walker’s company is piloting this approach to robotic control for a major engineering company. A system of this kind would allow a centrally based engineer to deal with critical equipment that needed immediate attention but was remotely located and difficult to reach. Robotic hands could save costly time.

Future robots should also be able to reap the benefits of machine learning, another aspiration that has been explored using the Dexterous Hand. In this case Shadow partnered with a third Californian company, Open AI. It is possible to build machines that, like humans, can learn by experience. But to perform an intricate physical task for real, and learn from mistakes, is slow and time consuming. Open AI has speeded things up by using computerised simulations of the task from which the system can devise effective finger movement patterns and strategies. When confronted with the real-life task, the computer controlling the robotic hand employs these strategies to ensure that it can achieve its goal promptly and efficiently.

The task that Open AI chose for this demonstration – the use of a robotic hand to manipulate objects in a purposeful manner – presents a major challenge; but Open AI’s Dactyl system rose to it. Holding a cube, each face of which was clearly marked with a letter, the robotic hand’s task was to manoeuvre the cube into a succession of chosen orientations. In front of the hand, and viewing both it and the cube from differing angles, were three cameras. Using continual updates on the coordinates of the fingertips and the orientation of the cube, the hand carried out the necessary manipulations. Armed with the knowledge of the previous simulations of the task, it performed with impressive speed and total proficiency.

Despite repeated enquiries about what might appear to be the most obvious application of a robotic hand – as a prosthetic substitute for the real thing – this use is not on the Shadow agenda. The goals, problems and constraints involved in building a prosthetic hand are quite different from those addressed by the Dexterous Hand, Walker points out. And, as he adds, there are other companies that specialise in this endeavour.

Outside of the factory and the warehouse, robots have yet to make a deep impression on our everyday lives. The advent of domestic robots would, of course, change all that. The potential is clearly vast but so far largely untapped, not least because the tasks to be done around the home, while mostly basic, are so varied and often so fiddly. To imagine an all-purpose robot that might be able to tackle them is surely to imagine a robot with hands. To reboot an old advertising slogan, the future’s bright, the future’s manual.