Did you know?
- Each year, more than 60,000 seasonal workers are needed to harvest the UK’s fruit and veg crops
- A shortage of workers has resulted in tonnes of fruit and veg going to waste
- Engineers are developing specialised picking and packing robots that could help plug the gap caused by this shortfall
The UK, like many north European countries, depends on seasonal migrant workers to harvest many of its crops. It is estimated that 63,000 parttime agricultural workers are needed in the UK annually. At the beginning of 2022, the government had allocated 40,000 seasonal agricultural worker licences, but the Russia/ Ukraine war dealt a severe blow to the size of the labour force, with many of 2021’s fruit pickers coming from these two countries.
The shortage of visas and delays in visa-processing have resulted in farmers leaving hundreds of tonnes of asparagus and millions of bunches of spring onions unpicked. Many other crops have been lost and ploughed back into the soil, harming the nation’s self-sufficiency in food production.
Growers and researchers have been working hard to find robotic solutions to help cover some of the jobs that seasonal workers do. The pace of development has quickened so rapidly that most companies and observers involved feel that human–robot cost parity will soon be reached. Within two, three or four years commercially viable robots could be picking and packing some of the UK’s summer crops.
Many companies developing horticultural robotics have focused on harvesting strawberries. It is a crop that is universally popular and always in demand. However, many things can affect business success, including diseases, or problems with packing, transport and harvesting. Robotics could help with all these issues and, in turn, be applied to other agricultural crops.
In northern Europe, most strawberry growers employ a ‘tabletop’ system, whereas in southern Europe, Australia and the US, farmers favour traditional on-the-ground growing. With robots, a standardised approach to how the crops are grown may develop in the future, but for now, developers are designing robots to cope with different farm set ups.
Tabletop systems usually have plants set 1.5 metres high. Pickers can conveniently view and pick fruit without having to kneel or bend to the floor. Plants are in lined growbags so that the plant doesn’t touch the soil – being exposed to wet and moist soil for too long can lead to fungal diseases. Strawberries are so-called because straw used to be put under the plants to keep them clean and avoid soil splash, which can blemish the fruit.
Whereas most domestic growers buy June-bearing strawberries that produce one large harvest in late spring or early summer, many commercial growers use ‘everbearers’. Everbearing strawberries, cultivated in polytunnels or large greenhouses, can produce fruit from the end of May to the end of October. The UK’s prototype machine-learning picking robots already ply their trade during these months, with some then transferring to Australia to further develop their picking skills.
How do they work?
Most companies developing robots have started their journeys from scratch. They are each developing their own ways of enabling their robots to see, grip, evaluate, transport, and pack fruit. The businesses know that if they can master strawberry picking, which is tricky, then they should be able to adapt their methodology to other crops.
Dogtooth Technologies is one of the most advanced robot fruit picking companies in the UK, with 70 of its fourth-generation robots already picking tabletop strawberries commercially. Dr Duncan Robertson, the company’s Co-Founder and CEO, says that the fifth-generation robot will achieve the same picking costs as humans. It will be demonstrated to customers next year with first deliveries in 2024/25. Human pickers can pick 15 to 30 kilograms an hour at a cost of roughly 50p a kilo and commercially viable robotic picking must be comparably cost effective. At the moment, picking robots are slower and not as dexterous compared to humans, but they can work longer hours and at night.
The first challenge for developers is to enable the robots to model their three-dimensional environments. Dr Robertson first worked with Mat Cook, one of Dogtooth’s three co-founders, at Microsoft Research in Cambridge helping to develop Kinect, the motion sensing add-on for the Xbox 360 gaming console. Dr Robertson says that although depth-sensing cameras work well in some applications, they perform poorly outdoors. Instead, Dogtooth’s robot uses vision systems based on simple RGB colour cameras. Dogtooth, like others, has used deep learning neural networks to train its vision systems using data provided by human annotators. These machine-learning algorithms enable its robots to interpret their environments using images as input.
The UK’s prototype machine-learning picking robots already ply their trade between May and October, with some then transferring to Australia to further develop their picking skills.
By using a stereo camera comprising two cameras set a few centimetres apart, Dogtooth’s robot can judge depth from 2D images, and therefore create detailed geometric 3D models of its surroundings. Dr Robertson says: “Our robot takes images of the growing strawberry plant and can understand what each pixel in the image means. It can distinguish individual strawberries and stalks as well as background elements such as leaves and polytunnel infrastructure. It can tell if a strawberry is sufficiently ripe, or if not, how long it is likely to take to ripen.” This ability to judge the health and amounts of future fruit is an important advantage that robots have over human pickers.
Dogtooth deliberately doesn’t rely on cloud computing resources for real-time picking. This avoids picking interruptions caused by network failure. It also reduces the data needed, and cost, for individual robots. It uses its AWS Cloud resource for higher latency functions such as processing the day’s data to inform the grower about expected future yields. Dr Robertson continues: “And when it spots a ripe strawberry, our system then directs the robot’s end effector [the device that picks] to accurately grip its stalk and cut it.”
Picking the fruit
Every company developing fruit picking robots seems to have devised a different end effector. Some are cup-shaped and embrace the fruit before breaking or cutting the stalk, others grip the fruit using soft ‘fingers’. Some have pincer-like extensions that grip and cut the stalk so as to avoid touching easily bruised soft fruits.
Dogtooth’s end effector comprises a pair of slender grippers and resembles a pair of tweezers. A separate cutting edge extends forward and cuts the stalk above the grippers leaving them holding the single strawberry by its stalk.
Some robots drop picked berries directly into a container, but Dogtooth’s carries them to a quality control system a few inches away where, under controlled lighting, the strawberry is scanned to detect 16 types of defects such as bruising, mould and asymmetry before being placed into a punnet or waste chute. Berries are weighed and punnets are filled until they reach the required retail weight.
Flexible farm robots
In July 2022, Simon Pearson, Professor of Agri-Food Technology Research at the University of Lincoln, co-chaired the report Review of Automation in Agriculture with George Eustice, who was then the UK government’s Environment Secretary. The report found that there was an active and emerging cluster of agri-robotic expertise funded by both private and public sector investment that will be able to replace some of the human harvesting roles on farms.
Simon’s own university team has made good progress in several areas of fruit picking. It has teamed up with Saga Robotics to run five proof-of- concept Thorvald robotic vehicles autonomously transporting strawberries from field to warehouse.
Saga already run lightweight Thorvald robots in the US, Norway and the UK to apply UV-C light to tabletop strawberries. This controls the powdery mildew that they are prone to, without using chemical fungicides. Looking like two-metre-tall sunbed arches, the robots go up and down rows of polytunnel strawberries once a week applying UV-C autonomously. Saga has found that this treatment works equally well for grapes. The company currently uses a dozen robots all year round at the 35-hectare Clock House Farm in Kent doing jobs such as spraying, crop forecasting and logistics.
Dogtooth has improved its fifth-generation robots by adding lithium-ion batteries that double the robots’ shift times from 8 to 16 hours. The new robot has been designed to be operated by customer personnel rather than Dogtooth’s. The company has also added onboard lighting to enable night picking and improve imaging conditions at tricky times of day, such as dusk and dawn.
Picking at night prolongs the berries’ shelf-life. Strawberries rapidly lose their freshness and quality if they spend more than an hour in warmth and sunlight after being picked. Picking at night reduces the temperature and light stress on fruit and maintains better internal and external quality with regards to sugars, colour and firmness.
Robots that pick fruit and veg then take it to be packed
University of Lincoln researchers have spent years working to program wheeled robots to transport strawberries from pickers (human and robotic) in growing areas to storage or packing locations. At the large Kent farm they operate on, the team estimates that 100 robots would be needed to service the 900 pickers in situ requiring complicated fleet logistics.
Unlike the dedicated pathways that robots work with in factories, farms are much less structured. There will be other farm workers, unaware members of the public using a public right of way, tractors, trailers, and other moving or static obstacles. This means that the robots’ navigation systems need proactive safety features.
Autonomous vehicles’ localisation systems require more than the GNSS-based solutions used by combine harvesters. These wouldn’t work in buildings and would be adversely impacted by the metal structures found in glasshouses and polytunnels. In addition, the traffic flows of dozens of robots need to be coordinated as robots cannot pass each other where growing tables are a metre apart and often hundreds of metres long. As well as avoiding congestion, they would need to be scheduled to reach the right picker or place at the right time to fully exploit harvesting capacity.
The researchers have devised topological maps with the data structure NEAR: N are the nodes representing physical locations in their geometrical map space; E are the edges connecting pairs of nodes; A is the set of navigation actions that the robot can perform; R is a set of Boolean conditions (true or false) that are used to define restrictions to autonomous navigation on the topology. This framework allows the robot to engage in any navigation action and underpins its learning, enabling it to reliably locate and track humans and robots across a large-scale farm environment.
The University of Lincoln team has focused on using the flexible Thorvald wheeled robots to fetch fruit from pickers and deliver them to the warehouse. Seasonal agricultural workers not only pick but spend nearly 20% of their time taking the fruit from its growing location to storage or packing facilities. The researchers have been working on programming autonomous navigation and safe interaction with pickers and other obstacles within a farm environment.
Packing is another area of horticulture where there are labour shortages and retention issues. When fruit and vegetables arrive at packing plants, they need to be sorted manually, graded, weighed, and then placed into packaging. The jobs are invariably repetitive, and the industry is challenged by high turnover and recruiting expenses.
In 2022, Wootzano Ltd developed its robotic system, Avarai, and started working with a leading pack-house corporation that supplies fresh produce to UK retailers. Dr Atif Syed is the Founder and CEO of Wootzano, and also received a Royal Academy of Engineering Princess Royal Silver Medal in 2022, as well as being a member of the Academy’s Enterprise Hub. His invention Wootzkin is a patented electronic skin made of modified elastomer, which is used by the Avarai robot. It contains metal electrodes that can be stretched without causing damage to the sensors within, measuring force and pressure.
Avarai is a collaborative robot (cobot) system that works alongside humans. It can estimate weight, trim fruits to size, identify defects, measure quality, pick, and gently place into punnets or other packaging.
Dr Syed says the Avarai cobot: “has been designed so that the relationship between the robot hand and the robot eye – the camera – is dynamic. So you can place the product anywhere and as long as Avarai can see it and reach it, it will, in real-time. It took 20-plus patents using algorithms and software to enable that, but it’s resulted in reduced cost on the hardware that ensures our customers get a product that is comparable in cost to human labour. There’s no need to reinvent the wheel, but sometimes you have to redesign it to make it more cost effective!” Wootzano is currently working on a commercial pilot line for packing vine tomatoes. By the end of 2023, Atif expects the Avarai robotic system to be fully commercialised for table grapes and vine tomatoes, and then scale up substantially thereafter to pack other fruit and vegetables.
Delivering agricultural robots
Leaders in the agricultural robotics field say that the optimism and excitement created by their rapid progress must be balanced with the reality that widespread adoption is a few years down the line.
Dr Robertson adds: “The UK has enormously innovative growers and they are enthusiastic early adopters of new technology. But we need the UK’s horticultural industry to be thriving in order to have the money to invest in robotic automation. Without adequate supply of seasonal agricultural workers that is impossible.”
In 2021, nearly 8,000 tonnes of berries worth £36 million were left to waste because of a shortage of pickers. As a response, some of the UK’s bigger producers have been developing farms in Senegal, Spain and Portugal so that they can continue supplying the UK market.
Professor Pearson says: “One of the reasons I undertook the government’s automation review with George Eustice MP was to ensure that the science evidence in this field is correct. Science and engineering have got a really important role to play in making sure that the facts are presented clearly so that we carry on getting investment in robotics, that we don’t lose farms in the process, and don’t cause massive disruption in supplies to shops.”
In 2020, UK farms produced about 2.5 million tonnes of vegetables and 650,000 tonnes of fruit. Together, these two agricultural strands generated nearly £3 billion worth of business. Agricultural robots have the potential to help grow these production levels by stepping into jobs that people don’t want to do. Exactly when the picking robots will match human costs is difficult to pin down, but Professor Pearson adds: “I’m asked that all the time, but I think it will be in three or four years.
Thanks to Professor Marc Hanheide, Dr Grzegorz Cielniak and Dr Amir Ghalamzan for their help with In-field logistics.
This article has been adapted from "Fruit pickers with potential", which originally appeared in the print edition of Ingenia 93 (December 2022).
Simon Pearson is a professor of agri-food technology and Founding Director of the Lincoln Institute of Agri-Food Technology at the University of Lincoln. The institute specialises in developing interdisciplinary solutions to key challenges within the agri-food sector, not least with the application and development of robotic and AI systems. Before joining Lincoln, he spent 15 years within the farming industry and supply chain.
Dr Duncan Robertson co-founded Dogtooth, a technology startup that develops and sells autonomous robots for picking delicate berry fruits such as strawberries and raspberries. Over the last two decades, Duncan has had a pivotal role in the success of several high-growth technology startups created to exploit emerging machine-learning capabilities in new commercial applications.
Dr Atif Syed founded Wootzano, a robotics company that develops dexterous robotic solutions that can delicately pack soft produce with ease. Dr Syed earned a PhD in electronics/bionanotechnology and engineering from the University of Edinburgh and has since received the Princess Royal Silver Medal for his patented electronic skin, Wootzkin, which allows robots to feel as humans do.
Keep up-to-date with Ingenia for freeSubscribe
Food & agriculture
Vertical farming for future growth
Agriculture faces mounting challenges in the 21st century. More of us live in cities than ever before and the global population is estimated to reach 11.2 billion by 2100. Science writer Sarah Griffiths spoke to Intelligent Growth Solutions to understand how produce grown inside under lights in trays arranged in vertically stacked towers could help meet growing demand for food.
Robots in the field
Four years ago, three engineers began developing a robot designed to autonomously pick and sort strawberries. Three designs later, the first group of 24 robots is reaching UK fields this summer. The robots will navigate the crops and select ripe fruit, before picking, inspecting and packing the strawberries.
Farming in the desert
Growing crops in hot, dry countries can be a challenge. Simple structures that use seawater to keep the growing environment cool are being used to farm fruit and vegetables in the Horn of Africa. Geoff Watts spoke to Charlie Paton, Founder and Director of Seawater Greenhouse, about the challenges of creating a greenhouse that cools instead of heats.
Q&A: Mamta Singhal
Mamta Singhal is a Commercialisation Manager for Coca-Cola European Partners, GB Supply Chain. Before this, she worked for two large toy manufacturers as a design engineer with Hasbro and as a project quality engineer for Mattel.
Other content from Ingenia
- Environment & sustainability
A young engineer’s perspective on the good, the bad and the ugly of COP27
- Environment & sustainability
- Issue 95
How do we pay for net zero technologies?
- How I got here
Electrifying trains and STEMAZING outreach
- Civil & structural
- Environment & sustainability
- Issue 95