Abstract
Conservation of damaged oil paintings requires manual inpainting of losses1,2, leading to months-long treatments of considerable expense; 70% of paintings in institutional collections are locked away from public view, in part because of treatment cost3,4. Recent advancements in digital image reconstruction have helped to envision treatment results, although without any direct means of achieving them5,6,7,8. Here I describe the physically applied digital restoration of a painting, a highly damaged oil-on-panel attributed to the Master of the Prado Adoration from the late fifteenth century. In parallel, 5,612 losses spanning 66,205 mm2 and 57,314 colours were infilled with a reversible laminate mask comprising a colour-accurate bilayer of printed pigments on polymeric films. To ensure the effectiveness of the restoration, ethical principles in painting conservation were implemented quantitatively for digital mask construction, a critically important foundation lacking in the current digital restoration literature. The infill process took 3.5 h, an estimated 66 times faster than conventional inpainting, and the result closely matched the simulation. This approach grants greatly increased foresight and flexibility to conservators, enabling the restoration of countless damaged paintings deemed unworthy of high conservation budgets.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
27,99 € / 30 days
cancel any time
Subscribe to this journal
Receive 51 print issues and online access
199,00 € per year
only 3,90 € per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data supporting the findings of this study are available within the paper, its Supplementary Information, Code Ocean and from the author upon request.
Code availability
The image processing code for digital infill mask generation is available on Code Ocean: https://codeocean.com/capsule/0882403/tree.
References
Stoner, J. H. & Rushfield, R. (eds.) Conservation of Easel Paintings (Routledge, 2020).
Idelson, A. I. & Severini, L. in The Encyclopedia of Archaeological Sciences (ed. López Varela, S. L.) (Wiley, 2018).
Corona, L. Stored collections of museums: an overview of how visible storage makes them accessible. Collect. Curation 44, 1–8 (2025).
Stone, A. Treasures in the Basement? An Analysis of Collection Utilization in Art Museums. RAND dissertation series, RAND School of Public Policy (2002).
Zeng, Y., Gong, Y. & Zeng, X. Controllable digital restoration of ancient paintings using convolutional neural network and nearest neighbor. Pattern Recognit. Lett. 133, 158–164 (2020).
O’Brien, C., Hutson, J., Olsen, T. & Ratican, J. Limitations and possibilities of digital restoration techniques using generative AI tools: reconstituting Antoine François Callet’s Achilles Dragging Hector’s Body Past the Walls of Troy. Arts Commun. 1, 1793 (2023).
Liu, X., Wan, J. & Wang, N. Ancient painting inpainting with regional attention-style transfer and global context perception. Appl. Sci. 14, 8777 (2024).
Xu, Z. et al. A comprehensive dataset for digital restoration of Dunhuang murals. Sci. Data 11, 955 (2024).
Stubbs-Lee, D. A. A conservator’s investigation of museums, visible storage, and the interpretation of conservation. Collections 5, 265–323 (2009).
Vecco, M. & Piazzai, M. Deaccessioning of museum collections: what do we know and where do we stand in Europe? J. Cult. Heritage 16, 221–227 (2015).
Keene, S., Stevenson, A. & Monti, F. Collections for People: Museums’ Stored Collections as a Public Resource (UCL Institute of Archaeology, 2008).
Jessell, B. Helmut Ruhemann’s inpainting techniques. J. Am. Inst. Conserv. 17, 1–8 (1977).
Tate-Harte, A. & Thickett, D. Calculating the carbon footprint of interventive and preventive conservation at English Heritage, UK. Stud. Conserv. 69, 323–332 (2024).
Johansson, E. A Detailed Conservation Report of a Heavily Retouched Painting from the Otto Valstad Collection. Master's thesis, Univ. of Oslo (2014).
Scott, D. A. Art restoration and its contextualization. J. Aesthetic Educ. 51, 82–104 (2017).
Amura, A. et al. Image analysis applied to the planning of a canvas painting restoration intervention. Ge-conservacion 18, 339–346 (2020).
Kumar, P. & Gupta, V. Preserving artistic heritage: a comprehensive review of virtual restoration methods for damaged artworks. Arch. Comput. Methods Eng. 32, 1199–1227 (2025).
Rojas, D. J. B., Fernandes, B. J. T. & Fernandes, S. M. M. A review on image inpainting techniques and datasets. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) 240–247 (IEEE, 2020).
Yang, J. & Ruhaiyem, N. I. R. Review of deep learning-based image inpainting techniques. IEEE Access 12, 138441–138482 (2024).
Barcelos, I. M., Rabelo, T. B., Bernardini, F., Monteiro, R. S. & Fernandes, L. A. F. From past to present: a tertiary investigation of twenty-four years of image inpainting. Comput. Graphics 123, 104010 (2024).
Elharrouss, O., Damseh, R., Belkacem, A. N., Badidi, E. & Lakas, A. Transformer-based image and video inpainting: current challenges and future directions. Artif. Intell. Rev. 58, 124 (2025).
Li, H., Hu, L., Liu, J., Zhang, J. & Ma, T. A review of advances in image inpainting research. Imaging Sci. J. 72, 669–691 (2024).
Bugeau, A., Bertalmío, M., Caselles, V. & Sapiro, G. A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19, 2634–2645 (2010).
Khalid, S. et al. A review on traditional and artificial intelligence-based preservation techniques for oil painting artworks. Gels 10, 517 (2024).
Sizyakin, R. et al. Crack detection in paintings using convolutional neural networks. IEEE Access 8, 74535–74552 (2020).
Maali Amiri, M. & Messinger, D. W. Virtual cleaning of works of art using a deep generative network: spectral reflectance estimation. Heritage Sci. 11, 16 (2023).
Palomero, C. M. T. & Soriano, M. N. Digital cleaning and “dirt” layer visualization of an oil painting. Opt. Express 19, 21011–21017 (2011).
Munoz-Pandiella, I., Andujar, C., Cayuela, B., Pueyo, X. & Bosch, C. Automated digital color restitution of mural paintings using minimal art historian input. Comput. Graphics 114, 316–325 (2023).
Merizzi, F. et al. Deep image prior inpainting of ancient frescoes in the Mediterranean Alpine arc. Heritage Sci. 12, 41 (2024).
Priego, E., Herráez, J., Denia, J. L. & Navarro, P. Technical study for restoration of mural paintings through the transfer of a photographic image to the vault of a church. J. Cult. Heritage 58, 112–121 (2022).
Cricchio, C. The restoration of the panel painting depicting the Adoration of Shepherds with a Saint Bishop. CeROArt. Conservation, exposition, Restauration d’Objets d’Art https://doi.org/10.4000/ceroart.5224 (2017).
Nocheseda, C. J. C., Santos, M. F. A., Espera, A. H. & Advincula, R. C. 3D digital manufacturing technologies, materials, and artificial intelligence in art. MRS Commun. 13, 1102–1118 (2023).
Elkhuizen, W. et al. Gloss, color, and topography scanning for reproducing a painting’s appearance using 3D printing. J. Comput. Cult. Heritage 12, 27:1–27:22 (2019).
Dardes, K. & Rothe, A. (eds.) The Structural Conservation of Panel Paintings: Proceedings of a Symposium at the J. Paul Getty Museum (Getty Publications, 1998).
Mecklenburg, M. F., Charola, A. E. & Koestler, R. J. (eds.) New Insights into the Cleaning of Paintings: Proceedings from the Cleaning 2010 International Conference (Smithsonian Institution Scholarly Press, 2019).
Yoo, W. S., Kang, K., Kim, J. G. & Yoo, Y. Extraction of color information and visualization of color differences between digital images through pixel-by-pixel color-difference mapping. Heritage 5, 3923 (2022).
Antropov, S. & Bratasz, Ł. Development of craquelure patterns in paintings on panels. Heritage Sci. 12, 89 (2024).
Karianakis, N. & Maragos, P. An integrated system for digital restoration of prehistoric Theran wall paintings. In 2013 18th International Conference on Digital Signal Processing (DSP) 1–6 (IEEE, 2013).
Ridderbos, B., van Buren, A. & van Veen, H. T. Early Netherlandish Paintings: Rediscovery, Reception, and Research (Getty Publications, 2005).
Crowe, J. The Early Flemish Painters: Notices of Their Lives and Works (John Murray, 1872).
Hand, J. O. & Wolff, M. Early Netherlandish Painting (National Gallery of Art, 1986).
de Loo, G. H. Hans Memlinc in Rogier van der Weyden’s Studio. Burlington Magazine for Connoisseurs 52, 160–177 (1928).
Cohen, E. J., Bravi, R., Bagni, M. A. & Minciacchi, D. Precision in drawing and tracing tasks: different measures for different aspects of fine motor control. Hum. Mov. Sci. 61, 177–188 (2018).
Komarova, N. L. & Jameson, K. A. A quantitative theory of human color choices. PLoS ONE 8, e55986 (2013).
Emery, K. J. & Webster, M. A. Individual differences and their implications for color perception. Curr. Opin. Behav. Sci. 30, 28–33 (2019).
Smet, K. A. G., Webster, M. A. & Whitehead, L. A. A simple principled approach for modeling and understanding uniform color metrics. J. Opt. Soc. Am. A Opt. Image. Sci. Vis. 33, A319–A331 (2016).
Abasi, S., Amani Tehran, M. & Fairchild, M. D. Distance metrics for very large color differences. Color Res. Appl. 45, 208–223 (2020).
Song, A., Faugeras, O. & Veltz, R. A neural field model for color perception unifying assimilation and contrast. PLoS Comput. Biol. 15, e1007050 (2019).
Pazzaglia, M. et al. Loss and beauty: how experts and novices judge paintings with lacunae. Psychol. Res. 85, 1838–1847 (2021).
Saunders, D. Ultra-violet filters for artificial light sources. Tech. Bull. 13, 61–68 (1989).
Acknowledgements
I thank the following institutions for digital scans of paintings: The Museo Nacional del Prado, The National Gallery of Art, The Birmingham Museums Trust, Metropolitan Museum of Art, The Staatliche Museen zu Berlin and The Alte Pinakothek. I acknowledge partial support from the John O. and Katherine A. Lutz Memorial Fund. The physical materials used in this study were procured with the resources I provided. This study was conducted in part through the use of equipment and facilities at MIT.nano, MIT Microsystems Technology Laboratories, MIT Department of Mechanical Engineering and MIT Libraries.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The author declares no competing interests.
Peer review
Peer review information
Nature thanks Aviva Burnstock, Hartmut Kutzke and Gennaro Vessio for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
Extended Data Fig. 1 Damage areas.
Four broad types of damage are shown overlayed onto the damaged image of the painting. Three complete panel fissures stem from warping of the wood support boards.
Extended Data Fig. 2 Visual overlay of head of infant.
All images are of identical scale and rotated to angularly align with each other. Three infants from other paintings by MPA are shown, with the lower row of images corresponding to 50% opacity overlays on the damaged infant in the painting used in this study. a, The damaged infant in the painting used in this study. b, Infant from the Nativity held by the Birmingham Museums Trust. c, Infant from: Master of the Prado Adoration, Adoration of the Magi, 1460–1470, Oil, Height: 60 cm; Width: 55 cm, (P001558). Madrid, Museo Nacional del Prado. ©Photographic Archive Museo Nacional del Prado. d, Infant from The Presentation in the Temple held by the National Gallery of Art, which was used to digitally restore the infant in this study. Scale bars are 1 cm. Credits: b, adapted with permission from Birmingham Museums Trust under a CC0 licence; c, adapted with permission from Museo del Prado, Madrid, © Photographic Archive Museo Nacional del Prado, © Archivo Fotográfico Museo Nacional del Prado; d, adapted with permission from the National Gallery of Art, Washington DC.
Extended Data Fig. 3 Restoration model of tolerance.
a, Cross-sectional diagram of bilayer alignment, showing misaligned white and color ink layers relative to the area of damage. Below are two plots of the damage. Absolute damage solely includes the exposed damage and overfilling. Perceived damage more closely follows the visual effect as seen by a human observer; the sharply contrasting exposed white ink layer is more damaging visually than overfilling. b, Area expansion diagram of layer alignment, showing a circular damaged region and a masking color ink region that is radially expanded. c, Plot of the minimum restorable cross-sectional feature dimension as a function of the combined lateral alignment tolerance and the target restoration effectiveness. In this work, the combined tolerance of 161.2 µm and restoration effectiveness of 1.0 (intersection denoted by star) leads to a calculated minimum restorable dimension of 511 µm. d, Plot of the minimum restorable feature radius as a function of alignment tolerance.
Extended Data Fig. 4 Mask construction pipeline.
a, Base images are prepared for further processing through brightening, blurring, and coordinate conversion. b, The base mask is constructed through analysis of human-perceived visual differences and identification of features based on contrast with local context. Further processing accounts for tolerance analysis. c, Manual adjustments and final corrections are made to the base mask, which is split for fabrication.
Extended Data Fig. 5 Key phases in digital mask construction.
a, General artifacts are initially identified through thresholding perceived differences in value and saturation. b, Following application of tolerance analysis-driven considerations, larger areas end up consolidated with the majority of microscopic regions excluded. c, Contextual features, which are more easily visually perceived owing to their contrast with a uniform background, are separately masked. d, The combined general artifacts and contextual features mask spans most areas that require infilling based on human visual perception. e, Manual additions incorporate large area losses, such as the exposed panel support around the edges of the work, that are not visually significant from an algorithmic standpoint but are identifiable by a human viewer based on broader context. f, The final masking areas overlayed on the damaged painting.
Extended Data Fig. 6 Digital infill mask color statistics.
a, Scatter plot of all 57,314 colors used in the infill mask on HSV axes, with log color scale denoting the number of pixels each color is used for. b, Histogram of infill mask colors based on the number of pixels each color is used for. The vast majority of colors represent regions smaller than 10,000 px2 (17.9 mm2). c, Histogram of the number of color variants (i.e., combinations of value and saturation) per hue.
Extended Data Fig. 7 Simulated restoration and actual result.
a, The simulated overlay of the digital mask on the image of the damaged painting. b, The physical restoration result, of remarkable similarity to the simulated version.
Extended Data Fig. 8 Mask removability.
a, After 2 months of storage, the applied laminate mask can be removed from the painting at the weaker interface of the varnish layer and applied mask, with solvent application assisting in areas with remaining mask fragments. b, A small segment of a fabricated infill mask is placed in a beaker with white spirit. c, After a minute, the infill mask is dissolved.
Extended Data Fig. 9 An alternative restoration attempt.
The physical laminate masking approach and digital mask construction enable the testing of different restorations in a vastly expedited time frame. A physical restoration based on a different infill mask is depicted here. a, The infill mask areas, which were constructed with a lower visually-perceived difference threshold and more aggressively target losses than the final mask used in this work. b, The color infill mask, which was color-adjusted to a higher color value. c, The physical restoration attempt, showing good color matching with the sky and poorer color matching in darker areas of the painting, with infill regions more easily distinguishable as compared to the final restoration in this work.
Supplementary information
Supplementary Information
This file includes Supplementary notes and 18 Supplementary figures.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kachkine, A. Physical restoration of a painting with a digitally constructed mask. Nature 642, 343–350 (2025). https://doi.org/10.1038/s41586-025-09045-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41586-025-09045-4
