The technologies that recreate historic artworks

The Cardinal lies on his back; his full regalia billowing in folds on either side of the couch. His eyelids are part-open, revealing eyes rolled back in their deep sockets. His mouth is twisted and hands clasped over his chest. In life, the Cardinal never allowed his portrait to be recorded, so his face is rendered from a death mask. The Tomb of Cardinal Tavera, carved by Spanish sculptor Alonso Berruguete, is richly decorated with allegorical figures representing the virtues that the Spanish church believed he embodied, when he died in the city of Valladolid in 1545 as Primate of All Spain and Grand Inquisitor.

However, this particular artwork isn’t in Spain. It is in Bishop Auckland, County Durham. It sits in the Spanish Gallery, an art gallery dedicated to Spanish art, history and culture that opened in April 2022, and is a perfect replica of what is believed to be one of the outstanding examples of Spanish Renaissance sculpture. The heavy original is fixed to the floor at the Hospital de Tavera, just outside of Toledo’s city walls, founded by Cardinal Juan Pardo de Tavera in the 16th century.

This replica is the work of Factum Arte, a Madrid-based company, founded by British artist Adam Lowe. Lowe prefers to call his team’s work ‘re-materialisations’, rather than replicas or copies, and they are only one part of the company’s output. Factum Arte uses state-of- the-art techniques to scan and photograph works of art, often revealing details that the human eye cannot see, creating a digital record of the artwork that can be preserved in case of the original deteriorating or even being destroyed. “Only a small fraction of the world’s art heritage is recorded, and this poses a huge risk of catastrophic loss to society and scholarship,” Lowe says. Variations of these techniques, using archive photography and scans of other works, can even allow artwork that no longer exists to be reverse-engineered and ‘re-materialised’ using techniques including additive manufacturing, CNC machining (cutting into a material), and colour printing.

It’s not just museums like the Spanish Gallery that benefit from Factum Arte’s work. The organisation has made new versions of artworks such as Paolo Veronese’s vast canvas The Wedding at Cana: the original, which was removed (some would say stolen) from Venice by Napoleon’s troops, hangs opposite the Mona Lisa in the Louvre and the re-materialisation is now in the location for which Veronese painted it – the San Giorgio Monastery in Venice – allowing it to be seen in its original context. Factum Arte has also reconstructed Caravaggio’s Nativity, which was stolen from Sicily in 1969 and is believed to have been destroyed by the Mafia, now restored to the location where the original was cut from its frame; one of Vincent Van Gogh’s seven sunflower paintings, destroyed in the US bombing of Japan in the Second World War; and Graham Sutherland’s portrait of Sir Winston Churchill, painted for his 80^th birthday, which Churchill hated so much that his wife had it burned to ashes.

Preserving heritage with new technologies

Lowe founded Factum Arte in 2001 with Spanish engineer and artist Manuel Franquelo. Initially, it operated as a workshop for contemporary artists to produce some of their work, including Marina Abramović, Anish Kapoor and Mariko Mori. In 2009, Lowe founded the Factum Foundation for Digital Technology in Preservation.

Lowe had been shocked by how little attention had been given to the protection of artistic heritage and was convinced that engineering and technology could help. He says that the use of technology to preserve heritage is underused: “One of the things that shocks me most is when somewhere as important as Notre-Dame burns down: it turns out that the only 3D scan done there was by a US academic casually over a couple of days in his free time. There was no systematic recording of that building using different technologies.”

As part of a 2001 project to record and produce facsimiles of ancient Egyptian tombs in the Valley of the Kings, Franquelo, working with a team of artists, conservators and engineers, developed a key piece of technology that is still vital to Factum today: the Lucida scanner. This close-range, noncontact laser scanner records the surface texture of low-relief objects such as paintings and bas-relief sculptures in very high resolution without being affected by the material or colour of the object being scanned. A polished gold surface is recorded with the same fidelity as marble. The foundation had used a variety of commercial 3D scanning systems but had found they were unable to produce accurate data from dark or glossy surfaces. The Lucida scanner moves a thin red laser line across the surface of an artwork, while two cameras positioned at 45° on either side of the line record how it is deformed by the texture it encounters. The integral software processes the captured images as 3D information in real time, while the system stores raw black and white video for future high-resolution post-processing.

The result is a ‘depth map’. The camera can move laterally on rails across the surface of an object and can also be pushed closer or pulled further away. Designed as a highly portable system, Lucida is now used at institutions including London’s National Gallery and Victoria and Albert Museum, Madrid’s Prado, Washington DC’s National Museum of Art, and the Louvre. By removing the colour from an object, it allows curators to study details such as an artist’s brush strokes, the ‘pounce marks’ made to transfer designs from a cartoon (a preparatory design on paper) to make a tapestry, or the influence of the texture of a wooden panel on the composition of a renaissance painting.

The National Gallery used the Lucida to scan the surface of Van Gogh’s Sunflowers to determine the texture the artist created with his use of thick paint so that it could re-materialise the painting of six sunflowers lost in Japan. As the two paintings were produced within days of each other, Lowe reasoned that Van Gogh would have used the same brushes and possibly even the same paint tubes to make both paintings and that therefore the texture of the brushstrokes on one could be extrapolated to model the other.

Lucida is often used alongside panoramic photography to record an object’s colours. Although such devices are used in industry and defence – for thermal imaging in conflict zones, for example – Factum Arte worked with German manufacturer Clauss, which developed the first electronic panoramic camera, to optimise the system. Lighting is a major problem for this application. Factum used a 600-millimetre telephoto lens to capture images, but to achieve optimum focus and depth of field this had to be set to a very small aperture, necessitating long exposure times and making the equipment vulnerable to vibration. To avoid this and speed up the process – time was a factor because the project before the camera was developed involved recording a whole fresco-covered room in the Vatican in three days – the team used an ultraviolet (UV) filtered high-speed flash coupled with a twin lens zoom system mounted on a separate panoramic head, which allows the flash to be positioned at the optimum angle to minimise reflections.

Factum Foundation is also working with the Bodleian Library on a project called ARCHiOx (Analysis and Recording Cultural Heritage in Oxford), which uses another new piece of equipment alongside Lucida – a prototype photometric stereo photographic system called Selene. Developed for capturing surface texture and colour of flat or nearly flat objects such as paintings, murals and sculptural bas-reliefs simultaneously, Selene captures 2D images under several different lighting angles and combines them electronically. Four flashes are synchronised using an electronic board to work with a mirrorless camera on a motorised mounting to scan documents, printing plates and other objects. “It’s the first time that any major library anywhere in the world has focused on 3D recordings of their objects,” Lowe says. “People talk about the material culture of the book with great reverence. But when it comes to digitising libraries, it’s very much a case of extracting information from the books so you can have it in a digital format. We wanted to show that if you also have 3D recording, you can start to see and understand the object in very different ways.”

The ARCHiOx project has recorded several 18th and 19th century etching plates from the Rawlinson, Lister and Gough collections, including a design identified as the work of William Blake, to an accuracy of 25 μm. “Being able to record an engraving with enough resolution to make a 3D print of it on an elevated printer, and then to make a print from that replica, is incredibly exciting,” Lowe said. The system was also used to scan Sanskrit manuscripts scratched into the waxy surface of palm leaves centuries ago, with equally striking results. He adds: “As the leaf ages and browns, it becomes unreadable. But with Selene, we could record the surface and extract the Sanskrit script very, very clearly. This to me was a real excitement.”

Another part of the project involved scanning an eighth century manuscript. “In the margin of the borders, we recorded letters that have never been seen – inscriptions and writing that can’t be seen by the human eye. It’s the first time, I’d say, we’ve actually got a clear result where no one was expecting it at all.”

Making physical objects for Factum Arte involves both additive and subtractive techniques. “We use primarily now, what we call elevated printing, which is a 3D-printed technology based on multiple layers of UV-cured ink,” says Lowe. “It uses electrostatics to attract the ink particles and the UV light hardens it in layers. Each layer is about 10 μm thick, so 100 layers get you to a millimetre. We use that to build the relief texture, but not the colour because the UV ink has a volume: it doesn’t look like oil paint. We’ve built piezoelectric printers in-house for printing colour. So we’ll build the volume of the brush marks with elevated printing, but we need a different character of ink to get the colour on top.”

Subtractive techniques include three- and seven-axis CNC milling, often to make moulds into which re-materialisations are cast. For the Tomb of Cardinal Tavera, moulds were made from physical stereo-lithographic prints and the monument itself cast in sections out of a marble composite. These sections were then joined together and hand-painted to replicate the patina (an aged finish that forms naturally over time) of the original.

Artworks understood and restored

As well as producing re-materialisations, Factum’s techniques can be used in restoration. For example, Lowe worked on a portrait by Sir Joshua Reynolds whose varnish had yellowed badly over time, a common problem with old paintings.

Usually, restorers remove yellowed varnish with a solvent to reveal the original pigments beneath. But Reynolds, a relentless experimenter, habitually tinkered with the composition of his paints to achieve illusionistic effects. The result of this was that dissolving the varnish would also have dissolved the pigmented glazes that Reynolds used to make the painting. Comparing the portrait with less deteriorated Reynolds works to determine precisely which pigments he had used, Lowe’s team digitally removed the yellow cast from their colour scan of the painting and printed this image onto a 3D print of the wooden panel on which Reynolds had worked. “Conventional restoration would have lost everything that made the painting specific, so they decided they couldn’t touch it. We were able to do more or less what a restorer would do, but without ever physically making contact with the paint surface.”

Lowe believes that Factum Foundation’s techniques can even enhance understanding of artworks. “There’s a whole new world of data analysis. Whether it’s based on machine learning or artificial intelligence, both are dependent on pattern recognition and number-crunching on a very large scale. I’m pretty certain that from the surface data we are recording, we will in time be able to attribute the hand of specific artists with great degrees of confidence. I’d be very surprised if we can’t record paintings by El Greco and say, ‘El Greco painted this part himself, his son Jorge Manuel painted this part, and several studio assistants painted these bits over here’.” Lowe is currently working on a project to test this theory with a team from the physics, materials science and engineering, and art history departments at Case Western Reserve University in Cleveland, Ohio [see box].

Though not himself an engineer, Lowe’s experience with mechanical, electronic, and software engineers has left him with a deep appreciation for the way that the disciplines mesh and collaborate, especially on projects that might seem to a layperson to be well out of their comfort zone. “I would love everyone to get excited by the way that engineering is now playing very creative roles in the production and preservation of works of art. For me, this is a beautiful example of what happens when people work together."

***

This article has been adapted from "Technology to recreate artworks", which originally appeared in the print edition of Ingenia 92 (September 2022).