image

Insight Brain: Lorin Sweeney on Searching for a Memorable Moment

Submitted on Thursday, 29/09/2022

Lorin Sweeney is a PhD candidate in Insight@DCU. His primary area of interest within this is memorability and he explains this below:

We spend our lives diving from one sea to another – when our minds are not suspended in the sea of sleep, they are sifting through the never-ending sea of sensory information. Even when you sit down to relax in the evening, you might watch TV, scroll through social media, sip tea, and chat with your family, perhaps even all at the same time. While our memories are typically rich and detailed, the chasm between what we sense and ultimately remember is deep; our brains simply cannot remember everything. When asked about that evening just a week later, some details might have crystallised into a memory (e.g., what you did that day), while others may have completely evanesced (e.g., what you had for dinner). While various cognitive processes—emotional state, attention level, extent of fatigue—influence what you will remember from any given day, the events themselves exert a large influence over what you will ultimately remember. For example, a particularly distinctive scene from a TV show may etch itself into your memory even if you are groggy or distracted, and this very same scene is equally likely to cling to the memory of everyone who watches it. This intrinsic likelihood is called its memorability, and is the focus of my research. More specifically, I work on understanding what makes media in its many modalities memorable, and designing machine learning models to predict how memorable given media will be.

Donald Hebb coined the catchy phrase: “cells that fire together, wire together”, to encapsulate the neurophysiological essence of memory formation. In essence, clusters of firing neurons connect up together to form cell assemblies that will subsequently fire as a single unit, i.e., if any of the constituent neurons are stimulated, they all fire. This cell assembly is how memories are represented in the brain. A transient memory is formed from the initial firing, and with repeated firing, the cells wire into a more permanent memory. The process of strengthening (wiring) the cell assembly is called consolidation, and typically only happens when we are asleep. Depending on the type of memory, it will be consolidated during one of two phases of sleep. Slow wave sleep (SWS) is primarily for explicit memory—experiences and fact based information—and rapid eye movement (REM) sleep is for implicit memory—motor skills and stimulus conditioning. While we are awake, information constantly flows into the brain, but most of it does not solidify; it simply evanesces because it has no relevance.

A current favoured hypothesis is that memorability represents a priority signal for a stimulus, and can be viewed as a proxy for human importance. We cannot remember everything we perceive, therefore our brains must rapidly and efficiently decide what to encode in memory. Extending this, I hypothesised that a moment of memorability should exist. That is to say, when you are watching something, if your brain decides to remember it, there should be an identifiable moment where it chooses to do so, and then form a transient memory. In order to explore this hypothesis, I devised and conducted an electroencephalography (EEG) experiment where participants’ brain waves were recorded while they watched a series of short (3s) video clips, and then 3-5 days later they played an online memory game to see which ones they remembered. The idea is that we should be able to identify the electrical signal created by the brain when it decides to either remember, or forget a stimulus. Ultimately, if a memorable moment can be identified, we want to figure out what sets it apart from other moments, and if the brainwaves can be used as a machine learning feature to predict if a given person will remember a given video.