Abstract
Recent breakthroughs in generative AI have opened the door to new research perspectives in the domain of art and cultural heritage, where a large number of artifacts have been digitized. There is a need for innovation to ease the access and highlight the content of digital collections. Such innovations develop into creative explorations of the digital image in relation to its malleability and contemporary interpretation, in confrontation to the original historical object. Based on the concept of the autonomous image, we propose a new framework towards the production of self-explaining cultural artifacts using open-source large-language, face detection, text-to-speech and audio-to-animation models. The goal is to start from a digitized artwork and to automatically assemble a short video of the latter where the main character animates to explain its content. The whole process questions cultural biases encapsulated in large-language models, the potential of digital images and deepfakes of artworks for educational purposes, along with concerns of the field of art history regarding such creative diversions.
Abstract (translated)
近期在生成式人工智能领域的突破为艺术和文化遗产领域的新研究视角打开了大门，尤其是在大量文物已经数字化的情况下。为了简化数字收藏的访问并突出其内容，需要创新性的解决方案。这些创新激发了对可塑性和当代诠释下数字图像关系的创造性探索，并与原始历史对象形成了对比。 基于自主图像的概念，我们提出了一种新的框架，用于利用开源大型语言模型、面部检测技术、文字转语音和音频到动画转换模型自动生成具有自我解释性质的文化艺术品。我们的目标是从数字化的艺术作品出发，自动组装一个短片，在该短片中主要角色会动起来并解释其内容。 整个过程质疑了大型语言模型中的文化偏见，探讨了数字图像以及艺术作品的深度伪造技术在教育领域的潜在应用，并引发了艺术史领域对这类创意延伸的关注和担忧。
URL
https://arxiv.org/abs/2506.05368
