Professor Alan Smeaton, PhD candidate Lorin Sweeney and Dr Graham Healy are doing fascinating work on memorability. Have a read below to see what that has to do with CSI: Crime Scene Investigation.
Professor Alan Smeaton writes:
Some things in our lives are more memorable than others. Typically, we remember things to which we have some emotional attachment, like photos or videos of family or loved ones or of holidays. When it comes to visual content to which we have no attachment, that we’re neutral towards, even then there will be some photos or short videos what we are likely to remember more than others. Memorability, the likelihood of being remembered, is a built-in property of all videos and images. It is similar to aesthetics or beauty in that it is hard to pinpoint, but it exists. People perceive some images or short video clips as more or less beautiful, aesthetic or memorable than others. Not only that, but people tend to agree on which clips are more memorable or beautiful than others.
As a result of developments in AI, machine learning and the availability of data, we can now compute the value of an image’s or video’s memorability with accuracy and reliability comparable to a human’s and the team at Insight has one of the world-leading techniques for doing this as evidenced on international benchmarks.
So what can we do with this new tool ?
Our long-term goals are to see this used in applications like educational content or advertising. Imagine tailoring your internet advert or your presentation material so the most memorable parts, visually, also contain your most important messages. Imagine a director editing a movie so that the most memorable scenes are the one with the most important parts of the storyline. Or imagine an episode of a TV crime series where the clues to “whodunnit” are actually filmed to be the least memorable so when the big reveal happens at the end of the episode or series, people get the greatest surprise because the giveaway scenes with the clues were filmed to be least memorable.
We analysed 39 episodes from the first five series of the original CSI: Crime Series Investigation TV program, from the early 2000s. Each episode had been meticulously annotated with scene and shot type, on-screen presence of the major characters, dialogue and relationship of the scene to the overall plotline. Using our memorability computation tools we computed the likely memorability of each shot and then analysed the relationships between memorability and different aspects of the episodes.
Across the first 5 series we found that the “motive” and the “evidence” scene types are more memorable than any others, which was surprising since we assumed directors would want to bury or mask those scenes which give away the perpetrators and make for a better cliff-hanger ending. We also found that scenes with the characters Catherine (played by Marg Helgenberger) and Gil Grissom (played by William Petersen) were constructed to be much more memorable than scenes with the minor characters like Nick or Warrick. Finally, we discovered that over the first five series, 75 percent of the episodes were directed by just three different directors and each has his own directorial style in terms of what scene types are recorded as more or less memorable. Each person has a kind of director’s fingerprint, and given an episode we can predict who was the director.
Our next step for our technique for computing memorability of images or videos is to apply it to movies and to short form video adverts see what that reveals. We also want to incorporate it as a form of recommendation into a video editing suite so that the editor gets prompts as to how to crop or clip raw video so as to make it more memorable, if that is wanted.