Serious games

Serious games

Beyond pure entertainment, do video games offer a credible alternative to established practices in simulation-based training and design? Professor Robert Stone of the University of Birmingham describes just some of the projects that are helping to change the sceptical preconceptions of today’s engineering and scientific communities.

The entertainment games industry has pioneered the use of multiple ‘cameras’ within a virtual world in order to enhance players’ situational awareness. In this virtual world scenario, that capability is exploited in a submarine rescue simulation developed for the Royal Navy’s Submarine Escape, Rescue & Abandonment (SMERAS) team. Three external views of a disabled Russian Kilo Class submarine are displayed within the rescue submersible compartment at the same time. The view through the vessel’s forward viewing dome shows the submarine’s fin with damaged mast well and escaping air bubbles. The other two displays, mounted to the right of the pilot’s seating position, are representative of pictures from closed-circuit cameras mounted on the submersible’s external superstructure © HFI Defence Technology Centre

In the June 2010 edition of Ingenia (issue 43), Dr Siân Harris helped present a review of the development of the video game industry in the UK, concluding with a brief mention of the cross-over of gaming technologies into other real-world sectors, including engineering, defence and medicine. Referred to by some research and development organisations as ‘serious games’, the exploitation of software and hardware from the entertainment industry for real-world applications in education, training, design and rapid prototyping is not as new as some might first think.

Battlezone, a successful but simple 3D tank game published in 1980 for the Atari console, was developed a year later to support training for the US Army’s Bradley military vehicle. Then, in 1988, a major step forward in the history of interactive 3D was witnessed, courtesy of Mindscape’s game The Colony. This remarkable first-person space combat game featured a crashed spaceship and a multi-level underground colony infested with aliens. The aim of the game was, through logical reasoning, to undertake the recovery of a replacement reactor to power the damaged ship and to rescue a group of children in cryogenic support.

The significance of The Colony, however, was not just its revolutionary (for 1988) 3D and first-person ‘shooter’ qualities. The product’s underlying development software was commercialised as a 3D modelling and visualisation toolkit (Virtus WalkThrough) and was subsequently modified for use as a virtual scene planning tool for the 1989 20th Century Fox science fiction film The Abyss. This exploitation path – from a game to a highly usable 3D toolkit to a real-world application – set the scene for today’s serious gamers.

Today, numerous proprietary and Open Source 3D toolkits are available, similar in style and function toVirtus WalkThrough, but based on modern design techniques and processes developed by the mainstream videogames community. These tools are being used to create all manner of virtual worlds and scenarios, for an ever-growing list of serious applications.

What is a serious game?

Unfortunately, and despite much academic discussion in online and conference forums, there is no single definition or unifying characteristic that adequately describes the subject. To many, serious games are games with a purpose – games that go beyond entertainment to deliver engaging learning experiences to trainees and students in a wide range of sectors. The term can equally be applied to the use of games for marketing (‘advergaming’), social networking, religious pursuits, executive role playing and much more.

Furthermore, serious games can be used to describe what some might consider more traditionally as simulation. Here, developers seek to take advantage of the fact that today’s gaming hardware and software is significantly more accessible and affordable than was the case in the virtual reality (VR) era of the 1990s. Certainly, when one compares today’s games-based simulation community with that of its VR predecessor, it becomes evident that developing highly believable 3D worlds is more achievable and affordable – by individuals with a much wider range of skills and abilities – than it was 10 to 15 years ago. No longer does one have to rely on graphics supercomputers costing many tens, if not hundreds of thousand of pounds. No longer are developers constrained by highly expensive software packages with crippling year-on-year maintenance costs.

Threat awareness trainees learn to manoeuvre the PackBot (seen in the foreground), one of the most successful battle-tested robots in the world. The realism in the scenarios lies partly in the obstacles, the weather and lighting effects generated by the graphic artists © HFI Defence Technology Centre

Online serious games forums and advice sites are plentiful, with many offering low-cost, even free, 3D models, texture assets and games engines – the software tools that power the real-time worlds as they appear on-screen.

The quality of many of today’s serious games is not only superior to that produced by professional simulation tools, it also benefits from a look and feel that is, some claim, highly familiar and acceptable to the up-and-coming generation of ‘gaming-savvy’ end users. It is also this quality, coupled with dynamic styles of gameplay, which sets serious gaming apart from its VR predecessor. Contemporary entertainment and serious games donotrequire their users to wear head-mounted displays, stereoscopic glasses, interactive gloves and other outlandish wearable technologies. They engage and immerse their users by virtue of well-designed content and compelling storylines.

It is not possible in an article of this size to do justice to the shear breadth of serious games activities evident across the globe. Instead, a brief review of some recent real-world applications follows, drawing attention to benefits gained from the technologies made available from the worldwide gaming development community.

Simulation for Defence

The international defence community has embraced gaming technologies with significant commitment. Projects such as the Bradley Trainer (mentioned above) and, in recent years, America’s Army, have convinced many nations to adopt games-based technologies to upgrade the delivery of training and education to their armed forces. The very fact that many mainstream video game titles involve some form of combat, set in many relevant contexts – jungle, desert, urban, polar, and so on – means that today there is widespread access to collections of highly detailed virtual models of battlefield equipment, military avatars and 3D terrain models. These assets make the development of defence training simulators more straightforward and cost-effective than ever before.

Besides the obvious combat-related applications one might expect, there are other, equally important military training applications currently under development, many of which are being pulled through to real classroom settings by experienced members of specialist defence establishments. For example, a recent games-based simulation tool has been developed to help deliver threat awareness lessons for new recruits to the explosive ordnance disposal (EOD) community. EOD instructors are under increasing pressure to prepare specialists for a variety of national and international operations, so it is crucial to provide opportunities to train these personnel with realistic simulator technologies.

One EOD threat awareness trainer produced by the Human Factors Integration Defence Technology Centre is based around a typical small town in the UK and features a petrol station, shops, office blocks, a multi-story car park, railway station, industrial area and a school. The scenario can be used to guide trainees through processes such as interacting with witnesses and members of the police, setting up safety cordons and deploying specialised remotely controlled vehicles to deal with a bomb threat. This classroom tool relies heavily on the realistic light and shadow effects that are offered by gaming development tools. For example, it is important to simulate time of day accurately, as what may be visible at midday through the virtual cameras of a remotely-controlled bomb disposal robot may be invisible just a few hours later. All of the 3D vehicles can be teleoperated via virtual CCTV displays in the back of the EOD support vehicle, using a standard keyboard, an Xbox gamepad, or even replicas of the real systems’ own control interfaces.

Another key strength of present-day games software packages is their ability to display, or render multiple images or virtual camera views in real time. Often used in games where competition occurs between two or more participants in real time, this multi-view rendering quality can be exploited to excellent effect to help train situational awareness of remote activities. The threat awareness trainer described above, for example, enables trainees to deploy a remotely controlled virtual bomb disposal vehicle from the rear of the mobile Incident Command Post, driving the vehicle using replica controls and, via two virtual closed-circuit TVs, images generated from virtual cameras onboard the vehicle.

The submersible rescue simulation described as part of the SubSafe project (see separate box on page 44) uses a similar solution, displaying external views from cameras mounted on the underside of the vessel. This assists operators to make accurate final approaches to the escape hatch on the disabled submarine casing, once that hatch disappears from the view provided by the submersible’s main viewing dome.

Underwater interactivity

Another good example of multi-view rendering is a demonstrator project for the Royal Navy’s survey and mine countermeasures teams. Here, the underwater 3D environment is created using data collated from bathymetric and seabed mapping exercises. This data is then displayed to the operators as part of a multi-view interface, such that the 3D topography is enhanced using virtual reconstructions of possible views from a remotely-controlled submersible or diver under varying conditions of lighting and turbidity. In this way, planning the safe approach to, for example, a mine or some other hazardous subsea feature can be prepared for in advance.

The underwater effects generated for this demonstrator were originally designed for HMS Scylla which was scuttled in March 2004 off Whitsand Bay near Plymouth by the National Marine Aquarium to become Europe’s first artificial reef. The virtual reconstruction of the wreck site was created to allow schoolchildren not only to explore the ship by ‘piloting’ a remotely-controlled submersible, but also to experiment, again virtually, with how changes in ocean temperature might affect certain colonies of starfish, sea urchins and algae.

As a result of this endeavour, the National Marine Aquarium is now a key collaborator in a project that seeks to exploit serious games in the development of highly detailed virtual rural and coastal worlds for the benefit of patients recovering from surgery, in care homes or undergoing psychotherapy. This project builds on yet another key strength of more recent 3D toolkits and engines from the gaming community, namely their ability to handle highly realistic natural environments, such as fields, forests and coastlines. Games likeFar Cry, Crysis and the Call of Duty series demonstrate the levels of detail that can be achieved, sometimes down to the smallest rock, leaf or blade of grass.

Combating stress

‘Virtual Stonehenge’ was created for English Heritage to demonstrate a number of significant developments beyond those achieved with the original virtual reality model, created for presentation at the London Planetarium by the astronomer Sir Patrick Moore. As well as an accurate night-time starfield, based on internet data sources, the model boasts real-time sunset and sunrise effects, with dynamic light rays and shadow casts. Each stone can be interrogated to reveal underlying reference data and, in one case, a small multimedia window relating to Sir Christopher Wren can be launched simply by mouse-clicking on a representation of graffiti he himself carved into one of the stones © HIT Team, University of Birmingham

This particular application of high-fidelity ‘natural’ virtual worlds aims to support a range of psychotherapeutic treatments, from combating post-traumatic stress and attentional deficit disorders to pain control and lack of sleep. The use of simulation in psychological therapies is not new. The 1990s were awash with studies supporting the use of VR in treating a wide range of conditions, including agoraphobia (fear of open spaces), acrophobia (fear of heights) and arachnophobia (fear of spiders). VR was even used to support counselling régimes following the terrorist atrocities of 11 September 2001.

However, more recently, there has been a surge of interest in promoting the health-restoring power of ‘green and blue’ natural environments. Research conducted since the early 1980s suggests that exposing individuals to rural and coastal settings, and even town gardens and parks, can promote stress reduction and enhance mental recovery following tasks requiring high levels of attention. Studies have also shown that these restorative environments, even if they are viewed through the window of a hospital ward, can reduce post-operative recovery times and the need for pharmaceutical pain relief.

Virtual restorative environments, based on serious games technologies, are now being developed to deliver similar benefits to those individuals who are unable to access and experience real natural environments, such as patients in hospitals, hospices, civilian and military rehabilitation centres, care homes, and so on. The current project with the National Marine Aquarium is at present focusing on the development of a comprehensive ‘blue-green’ virtual environment, based on part of the South West Coastal Path between Heybrook and Wembury Bays, just east of Plymouth.

What next?

These examples are but a small handful of real-world applications that are currently benefiting from the use of technologies originally developed for computer and video console entertainment purposes. For the serious games arena to achieve more than, and develop further and faster than, its VR predecessor, proponents and developers must do more than simply deliver prototypes and products that simply look impressive. Unfortunately, this is probably one of the field’s biggest weaknesses, with very few case studies providing reliable reports of the real-world outcomes of well-designed experiments and evaluations.

Nevertheless, it is fair to say that, while serious games still have some way to go before conclusive statements can be made as to their training efficacy (promoting positive skills or knowledge transfer and minimising skill fade, for example), their appearance on the technology-based training stage in many countries has, in the main, met with a surprisingly positive reception. This is despite the fact that the term ‘serious games’ seems to have attracted as many opponents as proponents and there exists a good number of sceptics who are still recovering from premature investment in the costly, unreliable and over-hyped immersive display and supercomputing technologies of the virtual reality era.