Penny Endersby FREng: how to weather the rise of AI

Modern meteorology depends on an array of scientific disciplines, underpinned by physics and overlaid by a blend of information technologies. Penny Endersby has experience in both those key domains, but her journey to becoming the CEO of the Met Office was anything but straightforward. After a brief spell in fuel cell research, she moved to research in materials engineering for armoured vehicles before switching to information technologies. At the Met Office, she now leads an organisation of about 2,300 people and a revenue approaching £300 million, leading research into weather and climate change. 

Like many school students, Endersby had little exposure to engineering as a career option. At least her girls school was, unlike many, “very, very good at getting girls to do science,” she says. “But they had no idea about engineering whatsoever.”

Yet, Endersby wasn’t completely ignorant of the engineer’s life. Her father worked for British Gas. She remembers him bringing home instruments that he would take out for nighttime noise surveys. He would start setting things up at home, showing her how to turn the sounds in the room into visual representations on paper. “I remember one of us sneezed and the needle went whoosh.” Endersby isn’t sure if her father really needed test his equipment or if he wanted to entertain and educate his daughters (Endersby’s younger sister grew up to be a chemical engineer), but the experience stuck. 

Another nudge towards engineering came in sixth form when Endersby undertook an extended period of work experience at the GEC Hirst Research Centre, which was sponsored by WISE (the Women and Science in Engineering campaign), and saw early work on photo-optical communications. In the mid-1980s, fibre optics weren’t widely used. She recalls peering down a microscope at optical waveguides, checking to see if they were usable. She was excited by “that sense of seeing something that no one else knew yet”.

Intent on keeping her options open, she chose to study for a natural sciences degree at the University of Cambridge. Endersby considered doing a PhD there but instead opted to work on solid oxide fuel cells at British Gas, the company that had sponsored her degree. After a short spell, she moved to the Ministry of Defence (MoD) and its Fort Halstead research site in London, before it moved to Porton Down and became the Defence Science and Technology Laboratory (Dstl). 

“I have not regretted [not doing a PhD], although I sometimes feel a little sheepish, surrounded by deep experts,” Endersby says. But she thinks that her research career more than made up for the missing academic qualification. “I’ve spent all my life as a working research scientist in industry and in the Ministry of Defence.” It is just that much of her work was secret and didn’t make it into a list of published papers.

Time for change

The MoD had brought her back to ceramics and their use in armour, in this case investigating how to make armoured vehicles much lighter, such as removing the large slabs of metal as armour to prevent projectiles from smashing into a vehicle. The idea was to see if electrical energy could repel incoming projectiles instead. It turned out that the science behind the concept of ‘electrical armour’, as it came to be called, wasn’t too bad, but it proved difficult to put it into a vehicle that could operate safely with personnel on board in conflict zones.

Endersby’s research career had to change to match the MoD’s priorities, a message she likes to get across when she talks to young engineers. You have to adapt your career to meet the needs of the client. And this next career change was more in response to circumstances than research demands. She was happily progressing along the scientific path at Porton Down, where she then worked. As part of a small team of researchers she says that she was “really quite allergic to any kind of management. I not only did not want to do management, but I also actively thought was it a bit of a waste of time that got in my way.”

Endersby changed her mind when the MoD wanted someone to lead a new electrical team. She decided to apply for the job. “I didn’t want someone who didn’t know about my science telling me what to do.” She laughs that this was “a really rubbish reason”, but it resulted in what she describes as a life-changing conversion. She’d wanted to do science since about the age of 16, and yet here she was setting off on a new path that would mean less time spent at the research frontiers. 

Despite her previous distaste for management, Endersby took to it. “To my utter confoundment and surprise, I found that I was actually really rather good at leadership and I enjoyed it. I could achieve more with a team of 10 than I could on my own. And I enjoyed developing people. I enjoyed the ability to deliver more impactful things.”

Endersby first led groups working on shock and acoustics as well as materials, her original beat. In 2009, she went on to head all physics work at Porton Down, with teams of 300 or 400 people. A brief detour deeper into management proved to be an important lesson. It turned out that even as a part of the MoD’s management team, she missed the connection with research. Endersby was temporarily in charge of ‘safety and estates’, with little connection to science. Previously she may not have been doing research herself, but it was around her all the time. “I hadn’t realised how much energy I took from seeing and understanding and sharing and finding ways to make better scientific advancements, even if I was not a researcher.”

Endersby wanted to be back rubbing shoulders with researchers. She could have returned to physics, but by then the MoD had another research challenge. Would she like to be director of the Information Management department? Cybersecurity was making its presence felt at Dstl. She admits that this was “one of those imposter syndrome moments. One minute you’re leading a physics department, you’ve been steeped in physics for 20 years, and the next minute you’re leading a bunch of coders, and you can’t code your way out of a paper bag.” Nevertheless, she took the plunge. In 2015 she was put in charge of Dstl’s Cyber and Information Systems Division, working on a very different security challenge.

Emerging technologies 

Endersby’s move into the cyber world in 2012 had amusing consequences. “I could not say we were creating an offensive cyber capability, because it was not declared. So I was recruiting people into something [where] I couldn’t tell them the exciting stuff they were going to do.”

Around the time that Endersby’s career “went cyber”, a colleague at Dstl spotted a call from the Royal Academy of Engineering for its Visiting Professors (VP) scheme. VPs are working engineers who spend time at a university so that students could have face-to-face encounters with real-world engineering. ‘Why not apply for that?’ was their suggestion to Endersby. 

Endersby liked the idea. She sees the scheme as “a laudable attempt to bridge the gap between the theory of engineering and the practice of it”. Despite Endersby being relatively new to the cyber world, the selection decided that it would be a good idea to match her with one of the UK’s prestigious computer science schools. “I was thinking ‘goodness me, I knew nothing about this’, and I’m suddenly a professor of computer science at the University of Southampton. I’d have been much more comfortable being professor of physics, or even in materials for engineering, than I would have been as a professor of computer science.” 

In the event, after saying that she wouldn’t be up to lecturing on computer science, Endersby could certainly fulfil the VP’s role of giving students some real-world contact. She could also make connections between her engineering work on the emerging challenges of AI in intelligence and the theory of cyber assurance, both areas of growing academic interest. It also gave the academics an opportunity to connect with someone who could talk about defence work, an area where, as Endersby puts it, “you don’t necessarily put out what you can’t do”.

The weather wildcard

In her career progression, Endersby narrowly missed out on becoming the head of Dstl. “Maybe next time,” was the message. But she still had ambitions albeit with a clear idea of what she wanted. “I’d like to be promoted. I don’t want to work in London. I like working for the government. I like doing something that’s valuable and purposeful in life.” Out of the blue she saw an advert for the top job at the Met Office. “It just ticked every box. I could see it was my dream job.”

As she saw it, while she might not put it like that, her credentials also made her a dream candidate. “They wanted somebody who could be a credible scientist and represent their science outwards and run an organisation of more than 2,000 people with a big turnover.” She had been through the Civil Service training system, rising through the ranks from being a junior manager, through middle and senior management to an executive role. She could also throw in the visiting professorship and time as a trustee of the Wiltshire Wildlife Trust as evidence of her long-standing interest in nature. 

Yet another section on Endersby’s CV was as one of the first engineers chartered to the Institute of Physics. She went on to chair the institute’s charter engineering committee. “I’ve always ridden the boundary between physics and engineering” – natural territory for meteorology. Nevertheless, Endersby admits, “I think I was a bit of a wild card for them.” 

When Endersby arrived at the Met Office a key item on her agenda was commissioning a new supercomputer system. It takes powerful computers to run the models of the atmosphere that underpin weather forecasts and predictions of climate change. Endersby believes that in future the Met Office may not run its own computer system and will join the general migration to ‘the cloud’. Then there was the growing importance of IT, especially AI, in the organisation’s work. 

With so much riding on computer expertise, the Met Office is naturally asking what AI might do for its work. “We are applying AI and machine learning in many ways in the Met Office” (See ‘Weathering the climate storm’). As she sees it, the jury is out on AI’s ability to replace physical modelling: there can be no AI models without the data that goes into the weather models. “It’s all built on AI being trained off the physical models.” Maybe one day AI can learn directly from observations rather than physical models. But not yet.

“I am really quite open minded about how far this will be another useful tool we add to our suite – I’m sure it will be at least that – versus the big disruptor that means that what we do will be completely transformed and we might not need the physical models. In the remaining time of my career, I think I’m going to be riding two horses.”