Using data in engineering

Using data to gain insights has become a major priority across all industries in recent times, with the ability to capture and process data opening up many opportunities. In engineering, data is transforming whole industries, through use of realtime analytics and machine learning for predictive maintenance for example, as well as allowing engineers to efficiently manage large-scale physical assets by generating virtual representations as digital twins (Creating a virtual replica, Ingenia 87). Data is fast becoming the world’s most valuable resource.

The untapped potential of data-driven innovation to boost productivity, create new products and services, and create new businesses and jobs has led to major investment from both the UK government and the private sector. The government has invested more than £2.3 billion since 2014 and has a 10-year vision to transform the UK’s data capabilities via the National Data Strategy and National Artificial Intelligence (AI) Strategy. Venture capital investment in data-driven AI companies is also skyrocketing. In 2021 Tech Nation reported that UK AI companies received $3.3 billion between January and August that year, already surpassing the $3 billion invested during the whole of 2020.

Open data in particular – which the Open Data Institute defines as data that anyone can access, use or share – is already used by hundreds of UK companies to offer new insights into everything from travel to recruitment. Transport for London (TfL) and the Ordnance Survey are two of the most high-profile data providers, which has fuelled the creation of dozens of popular apps, particularly in relation to travel planning and geo-location.

Innovating using data is hitting the mainstream and making transformative changes across engineering disciplines, from aerospace and automotive to energy and construction.

Making use of data

Data science is an interdisciplinary area where (mainly) three fields intersect: mathematics and statistics, computer science, and business knowledge – although this is specific to where it’s being applied. The aim of using data science is to extract insights and uncover patterns from data that can be in unstructured formats – such as images and audio – or structured formats – such as tables with rows and columns. Often, these disparate formats also contain meaningless data items, known as noisy data.

As data science evolves, so does the data science process workflow that’s usually followed to complete a project. This workflow is defined by four main phases, which are usually interconnected:

• business understanding, which involves scoping and planning the project. It’s imperative to address the ‘right’ question before diving into the relevant data.
• data preparation and understanding, which focuses on importing and aggregating data from various data sources, cleaning it by correcting incorrect or inaccurate data, and then wrangling the data by mapping it to more valuable formats for the task at hand.
• modelling, which involves building and evaluating a model to define assumptions about the data-generating process, to find patterns or to make predictions about unseen data.
• deployment, in which the model is integrated into a software environment suitable for users and is regularly checked and updated.

Data science is multidimensional and involves many related tasks. Consequently, it has led to many job specialisations, including data analysts, data engineers and machine-learning engineers, each focusing on specific parts of the process, with many overlapping skills and blurred boundaries. There are challenges associated with all these roles and their associated tasks that are common to many scientific disciplines including engineering.

Data challenges

One of the biggest barriers to successful adoption of data science is insufficient investment in data engineering. This focuses on building stable and optimised data systems and development and deployment ecosystems, to support the data science process. There are, for example, increasing demands for data storage and management skills. This is for several reasons, including the large volumes of data being collected and analysed; growing requirements around high computational efficiency, and the need to run engineering simulations that can explore behaviours under variable conditions.

Another reason is to understand emerging computing paradigms, including edge computing, in which system architectures will be decentralised, moving computing and storage closer to the data source to deal with issues such as unpredictable networks, latency and bandwidth limitations. These are all data engineering skills and taking advantage of these methods and tools can help build robust systems, to handle high-volume and high-velocity data flows that can be scaled and adapted.

Data analytics is the science of analysing data to extract valuable insights, and it is becoming a core skill set for engineering. Now, tools developed to handle large and complex datasets have enabled large-scale, real-time analytics. Four broad categories – descriptive, diagnostic, predictive, and prescriptive analytics – can be used depending on the requirements. Descriptive analytics focuses on using historical data to identify trends and relationships. Civil engineering, for example, uses descriptive analytics to identify relationships between parameters that interact and affect the stability and earthquake rigidity of buildings. Similarly, mechanical engineering uses descriptive analytics to identify trends in equipment performance and spot faulty machinery, to reduce the time taken for failure recovery and increase productivity.

Predictive analytics is used to make predictions about future outcomes using data-driven models. Here, classical statistical methodologies, machine learning and deep learning techniques are all used for predictive modelling, depending on the task, type and volume of data available. In practice, predictive analytics can be used in various fields of engineering. In transport planning, for example, engineers use time–series analysis for traffic forecasting to design transport management solutions. Electric vehicle (EV) engineers can use models to predict the ability of EVs to complete specific journeys for given battery levels and collaborate with transport engineers to determine the optimal locations of charging points (as in ‘Electrifying a maintenance fleet’). Chemical engineers use predictive analytics to calculate chemical properties of mixtures and materials. A model trained on spectrometry data for mixtures, for example, could predict the chemical components of an unseen mixture to ensure that it meets the specified requirements, before being further used in production. This is a major asset to improve productivity and support quality assurance.

Finally, diagnostic analytics is used to identify causal relationships by comparing trends and investigating the reasons behind them, such as a drop in performance, whereas prescriptive analytics uses data to determine an optimal course of action. Electrical engineers use the prescriptive analytics to design an optimal power grid that avoids failures and outages in a building by introducing interventions to prevent costly maintenance activities and downtime.

However, while harnessing the data analytics skill sets is vastly improving the efficiency and accuracy of solutions and driving progress, finding people with these highly technical skills is becoming a major problem for most industries, including engineering (see ‘Building data skills’). This is a problem that will need to be solved, if data science is to be used to its full potential to unlock innovation across sectors in the UK.

Building data skills

Data science is a relatively new discipline, and the skills required to address more complex problems are highly specialised and wide-ranging. The 2021 report Quantifying the UK Data Skills Gap found that there were as many as 234,000 data roles still to be filled.

 As demand for data skills continues to accelerate, engineering will need to remain creative about ways to recruit, retain and retrain its existing workforce. A change in engineering education can also ensure that all engineering graduates are equipped with the data science skills they will need to drive innovation over the coming decades and help to install a data-driven culture across engineering businesses.

 Apart from the technical and coding skills that a data scientist should have, the ability to work collaboratively and communicate their findings in an audience-specific way are equally important. Typical challenges for data scientists include combining the appropriate scientific tools, the responsibility of aggregating various data sources, preparing the data and asking the right questions.

There are numerous paths that someone can take to become a data scientist. In recent years many universities have added bachelor’s degrees in data science to their curriculum. Similarly, more and more universities offer relevant master’s courses that usually accept candidates with undergraduate degrees in IT, maths, computer science, engineering, physics, or related fields. For people who want to acquire a more focused background, a PhD can provide them with not only in-depth technical knowledge but also transferable skills to apply edgy techniques in industry and of course in research.

But hiring new talent is very competitive and demand is vastly outstripping supply, so upskilling existing employees is a viable alternative. Training presents a perfectly good way to get upskilled, providing the employer accepts the constraints. It can take months to learn the ideas behind one data science skill and years to become proficient.

Alternatively, consultancies can offer employers the comfort of a team approach and a broader set of data expertise. Outsourcing data skills can enable projects to progress immediately, but it will need a healthy budget.

There are also some pragmatic downsides for the business when knowledge is outsourced. When the consultants hand over the deliverables and leave the premises, will the business really understand what is happening under the hood, as insights are served up from a shiny new analytics dashboard? Will it realise when the dashboard has become inaccurate because the data on which the model was trained is now unrepresentative? It is therefore imperative that the data scientists left in the business ensure that all the relevant knowledge is transferred from the consultancy team.

The talent supply gap and the problems that industry faces to upskill employees, has led to new ideas around ways to engage existing talent on data-driven problems.

One model, run by NICD, supports an organisation’s existing workforce to better understand its own data. NICD’s team of data scientists brings skills covering data wrangling, analytics, scalable compute, machine learning, visualisation and more to address an industry problem. Upskilling is achieved through collaborative projects, where organisations use current business problems (resolved by using their own data) as the testbed to build relevant skills. These projects focus on innovations, creating efficiencies or exploring new products and services that can be integrated into their existing infrastructure to maximise impact. This engagement method also removes the need to take employees away from the ‘day job’ to undertake external training and allows the organisation to understand the relevant data skills required to address their own business needs.

The outcome of projects like these is to leave collaborating organisations with a solution that they understand, built from the ground up, which they can continue to develop using their own in-house talent and skills they have obtained on the journey.