Art curators could face competition as researchers have developed an artificial intelligence (AI) that can spot ‘hidden connections’ between paintings.
Parallels in styles, themes and motifs can link artworks painted in vastly different points in time and space — and shine new light on both works.
However, even the most knowledgeable of art critics could never take in the millions of paintings from across the centuries to be able to make all such connections.
The image retrieval software created by researchers from the Massachusetts Institute of Technology (MIT) and Microsoft, however, has no such constraints.
In tests, the AI returned successfully revealed hidden connections between works held by New York’s Metropolitan Museum of Art and Amsterdam’s Rijksmuseum.
Art curators could face competition as researchers have developed an artificial intelligence (AI) that can spot ‘hidden connections’ between paintings. Pictured, the pairing of Francisco de Zurbarán’s ‘The Martyrdom of Saint Serapion’ and Jan Asselijn’s ‘The Threatened Swan’ in the Rijksmuseum that inspired the researchers to develop the art-comparing AI
‘Image retrieval systems let users find images that are semantically similar to a query image, serving as the backbone of reverse image search engines and many product recommendation engines,’ said paper author and computer scientist Mark Hamilton.
‘Restricting an image retrieval system to particular subsets of images can yield new insights into relationships in the visual world,’ the MIT expert added.
‘We aim to encourage a new level of engagement with creative artefacts.’
The team reportedly took inspiration from the Rijksmuseum’s special ‘Rembrandt–Velázquez’ exhibit, which featured works from both Dutch and Spanish masters.
The collection included one unlikely, but similar, pair: Francisco de Zurbarán’s ‘The Martyrdom of Saint Serapion’ and Jan Asselijn’s ‘The Threatened Swan’, which bear visual similarities while portraying scenes of profound altruism, the researchers said.
‘These two artists did not have a correspondence or meet each other during their lives, yet their paintings hinted at a rich, latent structure that underlies both of their works,’ explained Mr Hamilton.
The researchers’ so-called ‘MosAIc’ system is capable of identifying similar analogous works from across different periods, artists and cultures by finding the closest matching artwork to a given painting.
For example, when given a query such as ‘which glassware is closest to this painting of a blue crane’, MosAIc returns an image of a slender blue Persian sprinkler.
The AI works by creating ‘trees’ that show the similarities between artworks — and from this allow it to pick out those images that are the most similar.
The researchers’ so-called ‘MosAIc’ system is capable of identifying similar analogous works from across different periods, artists and cultures by finding the closest matching artwork to a given painting. For example, when given a query such as ‘which glassware is closest to this painting of a blue crane’, MosAIc returns an image of a slender blue Persian sprinkler, pictured
‘We hope this work inspires others to think about how tools from information retrieval can help other fields like the arts, humanities, social science, and medicine,’ Mr Hamilton added.
‘These fields are rich with information that has never been processed with these techniques and can be a source for great inspiration for both computer scientists and domain experts.’
‘This work can be expanded in terms of new datasets, new types of queries, and new ways to understand the connections between works.’
‘As interesting as it is for algorithms to find connections, if you want to know why the connection exists, or what it means, you need an art historian or curator,’ Victoria & Albert Museum exhibition project curator Rosalind McKever told the Times.
‘One of our roles is researching, questioning and communicating the stories behind the connections.’
‘We are always looking at the bigger picture, beyond the collections of even the largest museums.’
A pre-print of the researchers’ article, which has not yet been peer-reviewed, can be read on the arXiv repository.
HOW ARTIFICIAL INTELLIGENCES LEARN USING NEURAL NETWORKS
AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn.
ANNs can be trained to recognise patterns in information – including speech, text data, or visual images – and are the basis for a large number of the developments in AI over recent years.
Conventional AI uses input to ‘teach’ an algorithm about a particular subject by feeding it massive amounts of information.
AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn. ANNs can be trained to recognise patterns in information – including speech, text data, or visual images
Practical applications include Google’s language translation services, Facebook’s facial recognition software and Snapchat’s image altering live filters.
The process of inputting this data can be extremely time consuming, and is limited to one type of knowledge.
A new breed of ANNs called Adversarial Neural Networks pits the wits of two AI bots against each other, which allows them to learn from each other.
This approach is designed to speed up the process of learning, as well as refining the output created by AI systems.