The IK Prize awards projects that use digital technology to offer new ways of exploring the Tate’s collection. Previous winners include After Dark – a project devised by The Workers which saw robots roam the Tate’s galleries at night – and Tate Sensorium, an immersive installation that used sound and scent to heighten the experience of viewing paintings.
This year, Tate teamed up with Microsoft and set a brief to create a project using artificial intelligence. The winning proposal, Recognition, uses AI to match images from Reuters news feed with artworks from the Tate’s collection. Images are displayed online in a virtual gallery and in a digital display at Tate Britain.
The AI analyses images in various ways. Facial recognition determines the gender, age and emotional status of people in paintings and photographs. Object recognition can detect anything from tables to instruments and fruit while composition recognition analyses colours, shapes and composition. The system also analyses context, looking at image titles and tags. Images which match closely in all four categories are considered a match and displayed side by side.
A photo of a man searching for coins thrown into the River Ganges, for example, is paired with John Constable’s painting The Gleaners, Brighton. Both feature a bird, a beach, a man standing on the beach and a vaguely similar composition.
Alongside images with obvious similarities, the system throws up some odd couplings. Photos of football fans are paired with 17th century portraits of men in coats and tights and an image of a Southern Railway train appears alongside a painting by Eduardo Paolozzi. There are ‘mistakes’, too – one picture of an Olympian biting her gold medal is matched with a painting of a woman eating a biscuit, with the AI assuming the medal is a snack.
It is these odd couplings that prove most fascinating, encouraging viewers to consider the connections between seemingly disparate sets of images and highlighting the differences in the way humans and machines process information.
“Some matches don’t make sense to a human but that’s quite interesting,” says Angelo Semeraro, who worked with Fabrica colleagues Coralie Gourguechon, Monica Lanaro and Isaac Vallentin and AI specialists JoliBrain to create Recognition. “We can analyse this data and [work out] what the AI is thinking rather than just looking at it as a mistake.”
Semeraro says the team was initially fascinated with exploring the intersection between art and AI. “For us it was both unexplored and fascinating territory,” he told press. After looking at the Tate’s collection of 50,000 artworks, the team hit upon the idea of exploring recurrences between past artworks and current events, using AI as a creative tool while helping people understand how AI works. Microsoft provided mentorship throughout the project and the team received a £15,000 prize plus £90,000 to put the idea into production.
Recognition will run online for three months, building a vast virtual gallery. It is also on display at Tate Britain alongside an interactive one which allows visitors to select their own matches from pre-selected groups.
Tate says the project was chosen because it posed interesting questions about how our brains work and the role machines can play in helping us understand art.
Four projects were shortlisted for the prize – the runners up were The Wandering Intelligence, a project which used facial recognition and sound detection to allow artworks to respond to visitors and effectively think for themselves; OSCAR, an intelligent interactive machine that can create artwork and Texting Tate, an intelligent chat bot that can describe artwork with help from the public. The Wandering Intelligence was devised by Ross Frame and Tom Wyatt, OSCAR by Unit Lab and Texting Tate by Michael Erier.
The post IK Prize: Fabrica uses AI to match news photos with classic paintings appeared first on Creative Review.