The technologies that recreate historic artworks
Did you know?
🖼️ How can priceless artworks be recreated?
- Only a small percentage of the world’s art is recorded in high resolution, preventing it from being lost forever if destroyed, stolen or deteriorated
- Modern technologies such as scanning and 3D printing are being used to produce copies of priceless artworks
- Engineers and artists are also using such techniques to restore paintings and even discover if an artwork is genuine
The Cardinal lies on his back; his full regalia billowing in folds on either side of the couch. His eyelids are part-open, revealing eyes rolled back in their deep sockets. His mouth is twisted and hands clasped over his chest. In life, the Cardinal never allowed his portrait to be recorded, so his face is rendered from a death mask. The Tomb of Cardinal Tavera, carved by Spanish sculptor Alonso Berruguete, is richly decorated with allegorical figures representing the virtues that the Spanish church believed he embodied, when he died in the city of Valladolid in 1545 as Primate of All Spain and Grand Inquisitor.
However, this particular artwork isn’t in Spain. It is in Bishop Auckland, County Durham. It sits in the Spanish Gallery, an art gallery dedicated to Spanish art, history and culture that opened in April 2022, and is a perfect replica of what is believed to be one of the outstanding examples of Spanish Renaissance sculpture. The heavy original is fixed to the floor at the Hospital de Tavera, just outside of Toledo’s city walls, founded by Cardinal Juan Pardo de Tavera in the 16th century.
This replica is the work of Factum Arte, a Madrid-based company, founded by British artist Adam Lowe. Lowe prefers to call his team’s work ‘re-materialisations’, rather than replicas or copies, and they are only one part of the company’s output. Factum Arte uses state-of- the-art techniques to scan and photograph works of art, often revealing details that the human eye cannot see, creating a digital record of the artwork that can be preserved in case of the original deteriorating or even being destroyed. “Only a small fraction of the world’s art heritage is recorded, and this poses a huge risk of catastrophic loss to society and scholarship,” Lowe says. Variations of these techniques, using archive photography and scans of other works, can even allow artwork that no longer exists to be reverse-engineered and ‘re-materialised’ using techniques including additive manufacturing, CNC machining (cutting into a material), and colour printing.
It’s not just museums like the Spanish Gallery that benefit from Factum Arte’s work. The organisation has made new versions of artworks such as Paolo Veronese’s vast canvas The Wedding at Cana: the original, which was removed (some would say stolen) from Venice by Napoleon’s troops, hangs opposite the Mona Lisa in the Louvre and the re-materialisation is now in the location for which Veronese painted it – the San Giorgio Monastery in Venice – allowing it to be seen in its original context. Factum Arte has also reconstructed Caravaggio’s Nativity, which was stolen from Sicily in 1969 and is believed to have been destroyed by the Mafia, now restored to the location where the original was cut from its frame; one of Vincent Van Gogh’s seven sunflower paintings, destroyed in the US bombing of Japan in the Second World War; and Graham Sutherland’s portrait of Sir Winston Churchill, painted for his 80th birthday, which Churchill hated so much that his wife had it burned to ashes.
Preserving heritage with new technologies
Lowe founded Factum Arte in 2001 with Spanish engineer and artist Manuel Franquelo. Initially, it operated as a workshop for contemporary artists to produce some of their work, including Marina Abramović, Anish Kapoor and Mariko Mori. In 2009, Lowe founded the Factum Foundation for Digital Technology in Preservation.
Lowe had been shocked by how little attention had been given to the protection of artistic heritage and was convinced that engineering and technology could help. He says that the use of technology to preserve heritage is underused: “One of the things that shocks me most is when somewhere as important as Notre-Dame burns down: it turns out that the only 3D scan done there was by a US academic casually over a couple of days in his free time. There was no systematic recording of that building using different technologies.”
Factum Arte uses state-of-the-art techniques to scan and photograph works of art, often revealing details that the human eye cannot see, creating a digital record of the artwork that can be preserved in case of the original deteriorating or even being destroyed.
Recreating paintings with a 3D printer
🖌️ No paintbrush required
Factum Arte uses a variety of techniques to make artworks. Some are drawn from the latest developments in manufacturing technology, while others are not even used in industry yet. They range from casting to advanced milling and include several types of 3D printing. While some of these technologies are used by contemporary artists to make objects, others are more often in use for re-materialising existing or lost art.
Perhaps the most striking example is using 3D printing to make objects that most people might consider to be not three-dimensional at all: paintings. The texture of the surface on which the original artist painted can have an influence on the composition: knots and grain in wood panels could direct the artist on how to place elements of the picture. Some artists’ techniques create characteristic 3D texture to their works: both Rembrandt and Van Gogh used very thick paint that they almost sculpted onto their canvases; a technique known as impasto. Almost every artist creates texture with their brushstrokes, which are an integral part of the viewer’s experience of looking at a painting.
Factum uses elevated printing to make the textured surface for a re-materialised painting. Similar to flatbed printing, which the company uses in an advanced form to print colour onto surfaces, elevated printing uses an UV-cured polymer ink to build up a textured surface layer upon layer. Each layer is between 10 μm and 15 μm thick and reproduces the depth map recorded using a Lucida scanner (sometimes manipulated digitally) in a physical form.
In general, the print from the elevated printer is not itself used as the substrate for a final re-materialised image but is used to make a silicon mould from which a substrate is cast in a material more suitable for the finished work. Gesso – a binder mixed with powdered chalk or gypsum – is often used, as artists have used it for centuries to prepare the surfaces of materials for painting.
The white textured surface thus created is then printed in colour using a modified flatbed Epson Stylus Pro 1180 printer. The modification allows the operator to overprint very precisely many times using inks of varying opacity. Before printing, Factum’s technicians use Photoshop to split the image file into several layers to extract as much information as possible about the different tones present. The deepest layer contains information about the darkest tones and subsequent layers capture lighter tones. Varying the opacity of the ink allows detail that might otherwise be lost in the darkest areas of the picture to be retained. The darkest areas are overprinted the most times to produce a very wide range of dark tones: this can be very effective when printing an image by an artist for whom dark colours are particularly important, such as Joshua Reynolds who often painted his subjects in rich dark clothing.
Another benefit of overprinting in this way is that it is very well suited to working on textured surfaces with micron level detail. To ensure that the ink lands on precisely the right position on the surface, a version of the Lucida texture map is first printed onto a clear acetate sheet, which is used to register the printer – that is, to position the printing head to a high precision – before replacing the clear sheet with the textured relief surface.
In some cases, rather than 3D-printing a textured substrate, the surface to be printed is made by routing; this is particularly useful when the relief depth is near or beyond the upper limit of what is achievable using elevated printing. Factum uses industrial routing machines controlled by a combination of commercial and in-house developed software. This technique was used to make replicas of the mediaeval Mappa Mundi, owned by Hereford Cathedral: the original map was drawn onto a single sheet of calf-skin vellum, which has, over seven centuries, warped and buckled. Incredibly precious, the original map is displayed behind glass that is only removed every other year. One facsimile was routed from plaster and left unprinted so that blind and partially sighted people can feel it to get an impression of the map’s appearance. Another version was colour printed to give a more accessible version of the map in its current state of preservation for study by conservators, researchers and the general public. Routing techniques are also used by some of Factum’s contemporary art clients, notably Marina Abramović, who makes objects routed from alabaster.
As part of a 2001 project to record and produce facsimiles of ancient Egyptian tombs in the Valley of the Kings, Franquelo, working with a team of artists, conservators and engineers, developed a key piece of technology that is still vital to Factum today: the Lucida scanner. This close-range, noncontact laser scanner records the surface texture of low-relief objects such as paintings and bas-relief sculptures in very high resolution without being affected by the material or colour of the object being scanned. A polished gold surface is recorded with the same fidelity as marble. The foundation had used a variety of commercial 3D scanning systems but had found they were unable to produce accurate data from dark or glossy surfaces. The Lucida scanner moves a thin red laser line across the surface of an artwork, while two cameras positioned at 45° on either side of the line record how it is deformed by the texture it encounters. The integral software processes the captured images as 3D information in real time, while the system stores raw black and white video for future high-resolution post-processing.
The result is a ‘depth map’. The camera can move laterally on rails across the surface of an object and can also be pushed closer or pulled further away. Designed as a highly portable system, Lucida is now used at institutions including London’s National Gallery and Victoria and Albert Museum, Madrid’s Prado, Washington DC’s National Museum of Art, and the Louvre. By removing the colour from an object, it allows curators to study details such as an artist’s brush strokes, the ‘pounce marks’ made to transfer designs from a cartoon (a preparatory design on paper) to make a tapestry, or the influence of the texture of a wooden panel on the composition of a renaissance painting.
The National Gallery used the Lucida to scan the surface of Van Gogh’s Sunflowers to determine the texture the artist created with his use of thick paint so that it could re-materialise the painting of six sunflowers lost in Japan. As the two paintings were produced within days of each other, Lowe reasoned that Van Gogh would have used the same brushes and possibly even the same paint tubes to make both paintings and that therefore the texture of the brushstrokes on one could be extrapolated to model the other.
Lucida is often used alongside panoramic photography to record an object’s colours. Although such devices are used in industry and defence – for thermal imaging in conflict zones, for example – Factum Arte worked with German manufacturer Clauss, which developed the first electronic panoramic camera, to optimise the system. Lighting is a major problem for this application. Factum used a 600-millimetre telephoto lens to capture images, but to achieve optimum focus and depth of field this had to be set to a very small aperture, necessitating long exposure times and making the equipment vulnerable to vibration. To avoid this and speed up the process – time was a factor because the project before the camera was developed involved recording a whole fresco-covered room in the Vatican in three days – the team used an ultraviolet (UV) filtered high-speed flash coupled with a twin lens zoom system mounted on a separate panoramic head, which allows the flash to be positioned at the optimum angle to minimise reflections.
Factum Foundation is also working with the Bodleian Library on a project called ARCHiOx (Analysis and Recording Cultural Heritage in Oxford), which uses another new piece of equipment alongside Lucida – a prototype photometric stereo photographic system called Selene. Developed for capturing surface texture and colour of flat or nearly flat objects such as paintings, murals and sculptural bas-reliefs simultaneously, Selene captures 2D images under several different lighting angles and combines them electronically. Four flashes are synchronised using an electronic board to work with a mirrorless camera on a motorised mounting to scan documents, printing plates and other objects. “It’s the first time that any major library anywhere in the world has focused on 3D recordings of their objects,” Lowe says. “People talk about the material culture of the book with great reverence. But when it comes to digitising libraries, it’s very much a case of extracting information from the books so you can have it in a digital format. We wanted to show that if you also have 3D recording, you can start to see and understand the object in very different ways.”
The ARCHiOx project has recorded several 18th and 19th century etching plates from the Rawlinson, Lister and Gough collections, including a design identified as the work of William Blake, to an accuracy of 25 μm. “Being able to record an engraving with enough resolution to make a 3D print of it on an elevated printer, and then to make a print from that replica, is incredibly exciting,” Lowe said. The system was also used to scan Sanskrit manuscripts scratched into the waxy surface of palm leaves centuries ago, with equally striking results. He adds: “As the leaf ages and browns, it becomes unreadable. But with Selene, we could record the surface and extract the Sanskrit script very, very clearly. This to me was a real excitement.”
Another part of the project involved scanning an eighth century manuscript. “In the margin of the borders, we recorded letters that have never been seen – inscriptions and writing that can’t be seen by the human eye. It’s the first time, I’d say, we’ve actually got a clear result where no one was expecting it at all.”
Making physical objects for Factum Arte involves both additive and subtractive techniques. “We use primarily now, what we call elevated printing, which is a 3D-printed technology based on multiple layers of UV-cured ink,” says Lowe. “It uses electrostatics to attract the ink particles and the UV light hardens it in layers. Each layer is about 10 μm thick, so 100 layers get you to a millimetre. We use that to build the relief texture, but not the colour because the UV ink has a volume: it doesn’t look like oil paint. We’ve built piezoelectric printers in-house for printing colour. So we’ll build the volume of the brush marks with elevated printing, but we need a different character of ink to get the colour on top.”
Subtractive techniques include three- and seven-axis CNC milling, often to make moulds into which re-materialisations are cast. For the Tomb of Cardinal Tavera, moulds were made from physical stereo-lithographic prints and the monument itself cast in sections out of a marble composite. These sections were then joined together and hand-painted to replicate the patina (an aged finish that forms naturally over time) of the original.
In the margin of the borders, we recorded letters that have never been seen – inscriptions and writing that can’t be seen by the human eye.Adam Lowe, Director of Factum Arte
Solving questions of attribution and even fraud
Can we determine who painted which part of historic works?
One of the most ambitious projects at Factum Foundation is its collaboration with Case Western Reserve University (CWRU) in Cleveland, Ohio, to develop ways to use Factum’s digital recording techniques to determine a painting’s attribution. The study aims to teach artificial intelligence systems to recognise the distinctive way that artists form brushstrokes on canvas, similar to handwriting recognition. This could not only give a valuable new way for the art world to determine whether a picture is genuine or a copy, but could even determine which members of an artist’s workshop contributed to a finished work and which parts were painted by whom.
Initially, the goal of the project is to study late works by El Greco, the Crete-born artist best known for his work in 16th-century Spain. He established a workshop in Toledo where he took on students, including his son Jorge Manuel, who worked with him as assistants. El Greco often painted only the portions of his large artworks that he considered most important himself and entrusted his students to reproduce his high idiosyncratic style in the other parts of the pictures; a common practice for artists of that and other periods. It is partly for this reason that El Greco’s work is often the subject of debates about attribution.
A team from CWRU’s department of physics, materials science and engineering developed the system. Using several paintings of lilies by students from Cleveland Institute of Art, four artists created three virtual ‘patches’ to areas of the paintings to simulate how assistants might have worked on them. The analysis focused on the subtle detail that can be thought of as the dynamic movements of the bristles as they respond to each hand. In reality, the analysis is numerical and is looking at the paintings in different ways. After training the system on the ‘patches’ produced by each artist, the system successfully identified 98% of them, connecting them to other samples made by the same hand.
Artworks understood and restored
As well as producing re-materialisations, Factum’s techniques can be used in restoration. For example, Lowe worked on a portrait by Sir Joshua Reynolds whose varnish had yellowed badly over time, a common problem with old paintings.
Usually, restorers remove yellowed varnish with a solvent to reveal the original pigments beneath. But Reynolds, a relentless experimenter, habitually tinkered with the composition of his paints to achieve illusionistic effects. The result of this was that dissolving the varnish would also have dissolved the pigmented glazes that Reynolds used to make the painting. Comparing the portrait with less deteriorated Reynolds works to determine precisely which pigments he had used, Lowe’s team digitally removed the yellow cast from their colour scan of the painting and printed this image onto a 3D print of the wooden panel on which Reynolds had worked. “Conventional restoration would have lost everything that made the painting specific, so they decided they couldn’t touch it. We were able to do more or less what a restorer would do, but without ever physically making contact with the paint surface.”
Lowe believes that Factum Foundation’s techniques can even enhance understanding of artworks. “There’s a whole new world of data analysis. Whether it’s based on machine learning or artificial intelligence, both are dependent on pattern recognition and number-crunching on a very large scale. I’m pretty certain that from the surface data we are recording, we will in time be able to attribute the hand of specific artists with great degrees of confidence. I’d be very surprised if we can’t record paintings by El Greco and say, ‘El Greco painted this part himself, his son Jorge Manuel painted this part, and several studio assistants painted these bits over here’.” Lowe is currently working on a project to test this theory with a team from the physics, materials science and engineering, and art history departments at Case Western Reserve University in Cleveland, Ohio [see box].
Though not himself an engineer, Lowe’s experience with mechanical, electronic, and software engineers has left him with a deep appreciation for the way that the disciplines mesh and collaborate, especially on projects that might seem to a layperson to be well out of their comfort zone. “I would love everyone to get excited by the way that engineering is now playing very creative roles in the production and preservation of works of art. For me, this is a beautiful example of what happens when people work together."
This article has been adapted from "Technology to recreate artworks", which originally appeared in the print edition of Ingenia 92 (September 2022).
Adam Lowe was trained in Fine Art at the Ruskin School of Drawing in Oxford and the Royal College of Art in London. He has been an adjunct professor of the MS in Historic Preservation at Columbia University, New York, since 2016. In 2019, Lowe became a Royal Designer for Industry, awarded by the Royal Society of Arts. He has written extensively about originality, authenticity, and preservation.
Keep up-to-date with Ingenia for freeSubscribe
Design & manufacturing
Is engineering productive?
The way in which productivity is measured in the UK does not account for advances in engineering and what these have added to GDP. However, a new group of statisticians, engineers and economists are working to change this and ensure that productivity is measured properly. Professor Will Stewart FREng argues that engineers also have a part to play in promoting engineering’s role in greater productivity.
A centre fit for future transport
As focus in the automotive industry turns to vehicles that are greener, safer and smarter, a new centre at the University of Warwick – a collaboration between WMG, Jaguar Land Rover and Tata Motors European Technical Centre – is addressing the challenges that are posed by their development. Professor David Mullins, Interim Head of WMG, discusses how the National Automotive Innovation Centre has been designed to be a collaborative research and education space with the facilities and scale so designers, engineers and researchers can come together and find a way forward for all of us.
Driven to an electrifying future
Thanks to his own career progress from apprentice to Executive Director of Product Engineering at Jaguar Land Rover, Nick Rogers FREng takes a special interest in the car company’s young engineers. He talked to Michael Kenward OBE about the challenge of simultaneously developing new car models, with the added complication of managing the transition to electric vehicles.
Wheelchairs that access all areas
Surprised by how little change there had been in wheelchair development over the years, former toolmaker Mike Spindle challenged himself to design and build a high-tech, lightweight wheelchair that could be created bespoke for each user. He spoke to Dominic Joyeux about the evolution of his game-changing design.
Other content from Ingenia
- Environment & sustainability
A young engineer’s perspective on the good, the bad and the ugly of COP27
- Environment & sustainability
- Issue 95
How do we pay for net zero technologies?
- How I got here
Electrifying trains and STEMAZING outreach
- Civil & structural
- Environment & sustainability
- Issue 95