Reflection-driven Artificial Intelligence in Art History – Explainable Hybrid Models for Image Search and Analysis

The project aims to provide both practically, in this case machine-aided, and theoretically oriented considerations on the use of image similarity assessments in art history, whose fundamental importance for scientific knowledge processes is widely recognized. These considerations are intended to differ from the few existing approaches in the field of art history in two respects.

First, art-historical expertise in the form of extensive (digitized or digitally available) text inventories will be exploited. For this purpose, domain-specific knowledge graphs are to be created in a semi-automatic and interdisciplinary effort, and then used to train hybrid artificial intelligence (AI) models.

Second, the performance of the machine is not simply to be pushed to the furthest extent possible, but rather we aim to make the AI-generated results more explainable—by incorporating expert texts and representing them in machine-interpretable knowledge graphs—to penetrate the black-box property of data-driven deep learning models.

The demonstration of plain feasibility is thus accompanied by a systematic reflexive examination in four research scenarios, where we investigate the impact of different text resources and knowledge graphs on AI-generated results, and eventually on art-historical research processes. In particular, we want to draw on discussions about the problems of applying AI methods in art history that arose during meetings of the first phase of the Priority Program (SPP). The perspective of our project is to secure greater acceptance for AI, which is generally heavily criticized in the humanities, by addressing the methodological problems of using it.