Research Articles | Challenge Journal of Structural Mechanics

From ruins to reconstruction: Harnessing text-to-image AI for restoring historical architectures

Kawsar Arzomand, Michael Rustell, Tatiana Kalganova


DOI: https://doi.org/10.20528/cjsmec.2024.02.004
View Counter: Abstract | 279 times | ‒ Full Article | 67 times |

Full Text:

PDF

Abstract


The preservation of cultural heritage has become increasingly important in the face of conflicts and natural disasters that threaten historical sites worldwide. This study explores the application of artificial intelligence (AI), specifically text-to-image generation technologies, in reconstructing heritage sites damaged by these adversities. Utilising detailed textual descriptions and historical records, this study employed AI to produce accurate visual representations of damaged heritage sites, bridging the gap between traditional documentation and modern digital reconstruction methods. This approach not only enhances the architectural design process across various disciplines but also initiates a paradigm shift towards more dynamic, intuitive, and efficient heritage conservation practices. The methodology integrates data collection, iterative AI-generated image production, expert review, and comparative analysis against historical data to evaluate reconstruction accuracy and authenticity. By integrating AI with traditional preservation practices, this study advocates a balanced approach to conserving cultural legacies, ensuring their preservation and revitalisation for future generations. Preliminary findings suggest that AI-generated imagery holds significant promise for enhancing digital heritage preservation by offering novel approaches for visualising and understanding historical sites. These findings also highlight the need to address ethical, technical, and collaborative challenges to enhance the precision, reliability, and applicability of AI technologies in the field of cultural heritage. This study contributes to digital humanities and archaeological conservation, demonstrating AI's potential to support and complement traditional heritage preservation methods and suggests a pathway for substantial methodological evolution in the field.


Keywords


text-to-image synthesis; architectural heritage; AI-assisted reconstruction; prompt engineering; digital archaeology; cultural heritage preservation

References


Adetayo AJ (2024). Reimagining Learning through AI Art: The Promise of DALL-E and MidJourney for Education and Libraries. Library Hi Tech News.

Angouri J, Paraskevaidi M, Wodak R (2017). Discourses of cultural heritage in times of crisis: the case of the Parthenon Marbles. Journal of Sociolinguistics, 21(2), 208–237.

Asal V, Avdan N, Ackerman G (2023). Breaking taboos: Why insurgents pursue and use CBRN weapons. Journal of Peace Research, 60(2), 193–208.

Aubry M, Berggren WA, Dupuis C, Ghaly H, Ward D, King C, Knox RWO, Ouda K, Youssef M, Galal WF (2009). Pharaonic Necrostratigraphy: A review of geological and archaeological studies in the Theban Necropolis, Luxor, West Bank, Egypt. Terra Nova, 21(4), 237–256.

Bassier M, Yousefzadeh M, Vergauwen M (2020). Comparison of 2D and 3D wall reconstruction algorithms from point cloud data for as-built BIM. Journal of Information Technology in Construction, 25, 173–192.

Bayram B, Nemli G, Özkan T, Oflaz OE, Kankotan B, Çetin İ (2015). Comparison of laser scanning and photogrammetry and their use for digital recording of cultural monument case study: Byzantine Land Walls-Istanbul. Isprs Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences, II-5(W3), 17–24.

Becker C, Laycock R (2023). Embracing deepfakes and AI‐generated images in neuroscience research. European Journal of Neuroscience, 58(3), 2657–2661.

Bennoui-Ladraa B, Chennaoui Y (2018). Use of photogrammetry for digital surveying, documentation and communication of the cultural heritage. Example regarding virtual reconstruction of the access doors for the Nameless Temple of Tipasa (Algeria). Studies in Digital Heritage, 2(2), 121–137.

Berto S, Demetrescu E, Fanini B, Bonetto J, Salemi G (2021). Analysis and validation of the 3D reconstructive process through the extended matrix framework of the Temple of the Roman Forum of Nora (Sardinia, CA), in: ArcheoFOSS XIII Workshop—Open Software, Hardware, Processes, Data and Formats in Archaeological Research, Basel, Switzerland, 18.

Bevilacqua MG, Caroti G, Piemonte A, Ulivieri D (2019). Reconstruction of lost architectural volumes by integration of photogrammetry from archive imagery with 3-D models of the Status Quo. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W9, 119–125.

Bharati P (2023). Leveraging AI for archaeological insights: generating visual representations of historical events - a case study of the Hastinapur Flood during the Reign of King Nichakshu. International Journal for Research in Applied Science and Engineering Technology, 11(10), 142–145.

Biljecki F, Stoter J, Ledoux H, Zlatanova S, Çöltekin A (2015). Applications of 3D City models: state of the art review. ISPRS International Journal of Geo-Information, 4(4), 2842–2889.

Brisco R, Hay L, Dhami S (2023). Exploring the role of text-to-image AI In concept generation. Proceedings of the Design Society, 3, 1835–1844.

Cao Y, Bowker MA, Delgado-Baquerizo M, Xiao B (2023). Biocrusts protect the Great Wall of China from erosion. Science Advances, 9(49).

Carter AK, Stark MT, Quintus S, Zhuang Y, Wang H, Heng P, Chhay R (2019). Temple occupation and the tempo of collapse at Angkor Wat, Cambodia. Proceedings of the National Academy of Sciences, 116(25), 12226–12231.

Chase AF, Chase DZ, Weishampel JF, Drake JB, Shrestha RL, Slatton KC, Awe JJ, Carter WE (2011). Airborne LiDAR, archaeology, and the ancient Maya landscape at Caracol, Belize. Journal of Archaeological Science, 38(2), 387–398.

Chen C (2021). Angkor Wat: a transcultural history of heritages. Journal of Southeast Asian Studies, 52(1), 133–140.

Chen C, Leask A, Phou S (2016). Symbolic, experiential and functional consumptions of heritage tourism destinations: The case of Angkor World Heritage Site, Cambodia. International Journal of Tourism Research, 18(6), 602–611.

Cobb PJ (2023). Large language models and generative AI, oh my! Advances in Archaeological Practice, 11(3), 363–369.

Debevec P (2004). The Parthenon, in: ACM SIGGRAPH 2004 Computer animation festival on - SIGGRAPH ’04, (p. 188). New York, New York, USA: ACM Press.

Denker A (2017). 3d visualization and photo-realistic reconstruction of the Great Temple of Bel. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2(W3), 225–229.

Elcheikh Z (2019). Palmyra: A story of ruins, struggle(s) and beyond. Chronos, 39, 105–123.

Evans DH, Fletcher RJ, Pottier C, Chevance J-B, Soutif D, Tan BS, Im S, Ea D, Tin T, Kim S, Cromarty C, De Greef S, Hanus K, Bâty P, Kuszinger R, Shimoda I, Boornazian G (2013). Uncovering archaeological landscapes at Angkor using lidar. Proceedings of the National Academy of Sciences, 110(31), 12595–12600.

Fincham D (2012). The Parthenon sculptures and cultural justice. Fordham Intellectual Property, Media & Entertainment Law Journal, 23, 943.

Fulford M, Wallace-Hadrill A (1999). Towards a history of pre-Roman Pompeii: Excavations beneath the house of Amarantus (I.9.11–12), 1995–8. Papers of the British School at Rome, 67, 37–144.

George P (2022). AI Trends in Digital Humanities Research. Trends in Computer Science and Information Technology, 7(2), 026–034.

Gröger G, Plümer L (2012). CityGML – Interoperable Semantic 3D City Models. ISPRS Journal of Photogrammetry and Remote Sensing, 71, 12–33.

Grün A, Remondino F, Zhang L. (2004). Photogrammetric Reconstruction of the Great Buddha of Bamiyan, Afghanistan. The Photogrammetric Record, 19(107), 177–199.

Gualandi ML, Gattiglia G, Anichini F (2021). An Open System for Collection and Automatic Recognition of Pottery through Neural Network Algorithms. Heritage, 4(1), 140–159.

Hamilakis Y (2002). The Other ‘Parthenon’: Antiquity and National Memory at Makronisos. Journal of Modern Greek Studies, 20(2), 307–338.

Hammer E, Seifried R, Franklin K, Lauricella A (2018). Remote Assessments of the Archaeological Heritage Situation in Afghanistan. Journal of Cultural Heritage, 33, 125–144.

Heikkinen J (2009). Close-Range Constrained Image Sequences. ISPRS Journal of Photogrammetry and Remote Sensing, 64(3), 267–274.

Horn C, Ivarsson O, Lindhé C, Potter R, Green A, Ling J (2022). Artificial Intelligence, 3D Documentation, and Rock Art—Approaching and Reflecting on the Automation of Identification and Classification of Rock Art Images. Journal of Archaeological Method and Theory, 29(1), 188–213.

Hsu Y-C, Yang Z, Buehler MJ (2022). Generative Design, Manufacturing, and Molecular Modeling of 3D Architected Materials Based on Natural Language Input. APL Materials, 10(4).

Hu G, Mortazavian P, Kittler J, Christmas W (2013). A Facial Symmetry Prior for Improved Illumination Fitting of 3D Morphable Model, International Conference on Biometrics (ICB), 1–6.

Huang C-T, Geng T, Liu J (2023). Capturing the characteristics of mis/disinformation propagation over the internet. Proceedings Volume 12542, Disruptive Technologies in Information Sciences VII, 125420P.

Kadhim I, Abed FM (2023). A critical review of remote sensing approaches and deep learning techniques in archaeology. Sensors, 23(6), 2918.

Kenig N, Monton Echeverria J, Muntaner Vives A (2023). Human beauty according to artificial intelligence. Plastic and Reconstructive Surgery - Global Open, 11(7), e5153.

Keyvanfar A, Shafaghat A, Rosley MS (2022). Performance comparison analysis of 3D reconstruction modeling software in construction site visualization and mapping. International Journal of Architectural Computing, 20(2), 453–475.

Knell S (2022). Experimental museology: institutions, representations, users. Museum Management and Curatorship, 37(3), 330–332.

Ko J, Ajibefun J, Yan W (2023). Experiments on generative AI-powered parametric modeling and BIM for architectural design. arXiv, 2308.00227.

Lee H-H, Chang AX (2022). Understanding pure CLIP guidance for Voxel grid NeRF models. arXiv, 2209.15172.

Liu V, Chilton LB (2021). Design guidelines for prompt engineering text-to-image generative models. arXiv, 2109.06977.

Lyu Y, Wang X, Lin R, Wu J (2022). Communication in human–AI co-creation: Perceptual analysis of paintings generated by text-to-image system. Applied Sciences, 12(22), 11312.

Mahmoud H, Alfons R, M Reffat R (2019). Analysis of the driving forces of urban expansion in Luxor City by remote sensing monitoring. International Journal of Integrated Engineering, 11(6).

Maiwald F, Vietze T, Schneider D, Henze F, Münster S, Niebling F (2017). Photogrammetric analysis of historical image repositories for virtual reconstruction in the field of digital humanities. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2(W3), 447–452.

Manning J (2012). Thebes (Diospolis Magna), Ptolemaic and Roman Periods, in: The Encyclopedia of Ancient History. Wiley.

Mazzaglia A (2021). The information system of Pompeii sustainable preservation project. A tool for the collection, management and sharing of knowledge useful for conservation and renovation of archaeological monuments. ArcheoFOSS XIII Workshop—Open Software, Hardware, Processes, Data and Formats in Archaeological Research, Basel, Switzerland, 14.

Meister J, Garbe P, Trappe J, Ullmann T, Es-Senussi A, Baumhauer R, Lange-Athinodorou E, El-Raouf AA (2021). The sacred waterscape of the Temple of Bastet at Ancient Bubastis, Nile Delta (Egypt). Geosciences, 11(9), 385.

Natampally M (2014). Reconstrucción Visual (Gráfica, Ilustrada y Digital) Del Templo Hampi. Virtual Archaeology Review, 5(10), 117.

Navarro-Mateu D, Carrasco O, Cortes Nieves P (2021). Color-patterns to architecture conversion through conditional generative adversarial networks. Biomimetics, 6(1), 16.

Nayak R, Balabantaray BK (2021). Generative adversarial network for heritage image super resolution. Computer Vision and Image Processing, 161–173.

Newton A, Dhole K (2023). Is AI art another industrial revolution in the making? arXiv, 2301.05133.

Nichol A, Dhariwal P, Ramesh A, Shyam P, Mishkin P, Sutskever I, Chen M (2021). GLIDE: towards photorealistic image generation and editing with text-guided diffusion models. arXiv, 2112.10741.

Oppenlaender J (2022). The creativity of text-to-image generation. Proceedings of the 25th International Academic Mindtrek Conference, New York, USA, 192–202.

Orengo HA, Krahtopoulou A, Garcia-Molsosa A, Palaiochoritis K, Stamati A (2015). Photogrammetric re-discovery of the hidden long-term landscapes of Western Thessaly, Central Greece. Journal of Archaeological Science, 64, 100–109.

Pakkeerappa P, Thomas J (2006). Strategic role of Hampi development authority in promoting tourism in Karnataka: a study. Atna - Journal of Tourism Studies, 1(1), 86–95.

Pan J, Li L, Yamaguchi H, Hasegawa K, Thufail FI, Tanaka S (2020). Fused 3D transparent visualization for Large-scale cultural heritage using deep learning-based monocular reconstruction. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2, 989–996.

Pierrot-Deseilligny M, De Luca L, Remondino F (2011). Automated image-based procedures for accurate artifacts 3D modeling and orthoimage generation. Geoinformatics FCE CTU, 6, 291–299.

Pollegioni P, Woeste KE, Chiocchini F, Del Lungo S, Olimpieri I, Tortolano V, Clark J, Hemery GE, Mapelli S, Malvolti ME (2015). Ancient humans influenced the current spatial genetic structure of common walnut populations in Asia. PLOS ONE, 10(9), e0135980.

Powell S (2018). Etched in stone: sixteenth-century visual and material evidence of Śaiva Ascetics and Yogis in complex non-seated Āsanas at Vijayanagara. Journal of Yoga Studies, 1, 45–106.

Raja R, Seland EH (2022). The Paradox of Palmyra: An ancient Anomalopolis in the desert. Journal of Urban Archaeology, 5, 177–189.

Rajangam K, Sundar A (2021). Reading the entanglements of nature-culture conservation and development in contemporary India. Journal of South Asian Development, 16(1), 7–32.

Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M (2022). Hierarchical text-conditional image generation with clip latents. arXiv, 2204.06125.

Reade JE (2002). The Ziggurrat and Temples of Nimrud. Iraq, 64, 135–216.

Remondino F (2011). Heritage recording and 3D modeling with photogrammetry and 3D scanning. Remote Sensing, 3(6), 1104–1138.

Remondino F, Rizzi A (2010). Reality-based 3D documentation of natural and cultural heritage sites‒techniques, problems, and examples. Applied Geomatics, 2(3), 85–100.

Rihani N (2023). Interactive immersive experience: Digital technologies for reconstruction and experiencing Temple of Bel using crowdsourced images and 3D photogrammetric processes. International Journal of Architectural Computing, 14, 396.

Rombach R, Blattmann A, Ommer B (2022). Text-guided synthesis of artistic images with retrieval-augmented diffusion models. arXiv, 2207.13038.

Sbrogiò L (2022). Parametric approach to the reconstruction of timber structures in campanian Roman houses. Virtual Archaeology Review, 13(26), 45–61.

Scherer AK (2007). Population structure of the classic period Maya. American Journal of Physical Anthropology, 132(3), 367–380.

Schettino P (2016). Successful strategies for dealing with new technology in museums: a case study of immersive technology at the Immigration Museum, Melbourne. Museum International, 68(1–2), 130–135.

von Schwerin J, Richards-Rissetto H, Remondino F, Agugiaro G, Girardi G (2013). The MayaArch3D project: a 3D WebGIS for analyzing ancient architecture and landscapes. Literary and Linguistic Computing, 28(4), 736–753.

Scorrano G, Viva S, Pinotti T, Fabbri PF, Rickards O, Macciardi F (2022). Bioarchaeological and palaeogenomic portrait of two Pompeians that died during the eruption of Vesuvius in 79 AD. Scientific Reports, 12(1), 6468.

Senatore MR, Ciarallo A, Stanley J (2014). Pompeii damaged by volcaniclastic debris flows triggered centuries prior to the 79 A.D. Vesuvius Eruption. Geoarchaeology, 29(1), 1–15.

Shelach-Lavi G, Wachtel I, Golan D, Batzorig O, Amartuvshin C, Ellenblum R, Honeychurch W (2020). Medieval long-wall construction on the Mongolian Steppe during the eleventh to thirteenth centuries AD. Antiquity, 94(375), 724–741.

Shishido H, Ito Y, Kawamura Y, Matsui T, Morishima A, Kitahara I (2017). Proactive preservation of world heritage by crowdsourcing and 3D reconstruction technology. IEEE International Conference on Big Data, 4426–4428.

Soto-Martín O, Fuentes-Porto A, Martín-Gutiérrez J (2020). A digital reconstruction of a historical building and virtual reintegration of mural paintings to create an interactive and immersive experience in virtual reality. Applied Sciences, 10(2), 597.

Spallone R, Lamberti F, Olivieri LM, Ronco F, Castagna L (2022). AR and VR for enhancing museums’ heritage through 3D reconstruction of fragmented statue and architectural context. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(W1), 473–480.

Steinfeld K (2023). Clever little tricks: a socio-technical history of text-to-image generative models. International Journal of Architectural Computing, 21(2), 211–241.

Taveekitworachai P, Abdullah F, Dewantoro MF, Thawonmas R, Togelius J, Renz J (2023). ChatGPT4PCG competition: character-like level generation for science birds. arXiv, 2303.15662.

Toubekis G, Jansen M, Jarke M (2017). Long-term preservation of the physical remains of the destroyed Buddha figures In Bamiyan (Afghanistan) using virtual reality technologies for preparation and evaluation of restoration measures. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 5(W2), 271–278.

Wahbeh W, Nebiker S, Fangi G (2016). Combining public domain and professional panoramic imagery for the accurate and dense 3D reconstruction of the destroyed Bel Temple in Palmyra. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 3(5), 81–88.

Wang X, Lasaponara R, Luo L, Chen F, Wan H, Yang R, Zhen J (2020). Digital heritage. Manual of Digital Earth, Singapore, 565–591.

Wotzlaw J-F, Bastian L, Guillong M, Forni F, Laurent O, Neukampf J, Sulpizio R, Chelle-Michou C, Bachmann O (2022). Garnet petrochronology reveals the lifetime and dynamics of Phonolitic Magma chambers at Somma-Vesuvius. Science Advances, 8(2).

Xie Y, Pan Z, Ma J, Jie L, Mei Q (2023). A prompt log analysis of text-to-image generation systems. Proceedings of the ACM Web Conference 2023, New York, USA, 3892–3902.

Xu Z, Wu TH, Shen Y, Wu L (2016). Three dimentional reconstruction of large cultural heritage objects based on UAV video and TLS Data. The International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences, XLI(B5), 985–988.

Yang M (2017). Crossing between the Great Wall of China and the “Great” Trump Wall. Palgrave Communications, 3(1), 25.

Zeng X, Jin T (2023). 3D Reconstruction of buildings based on transformer-MVSNet. In: Chen X and Srivastava HM (Eds.), 3rd International Conference on Applied Mathematics, Modelling, and Intelligent Computing (CAMMIC 2023), 191.

Zhou Y, Li P, Ye Z, Yue L, Gui L, Jiang X, Li X, Liu Y (2022). Building information modeling‐based 3D reconstruction and coverage planning enabled automatic painting of interior walls using a novel painting robot in construction. Journal of Field Robotics, 39(8), 1178–1204.


Refbacks

  • There are currently no refbacks.