Bridging the gap: Brain-inspired solutions to address catastrophic forgetting in artificial neural networks
Memory is essential for our existence and identity. The goal of learning is to have the memory stored in the brain for rapid or intuitive retrievals as needed. It applies to everything we do in everyday life, such as skills, habits, reasoning, social interactions, and decision-making. While we constantly learn since birth, many wish we could have learned faster and remembered more. In reality, we complain frequently about forgetting, whereas, sometimes, we cannot remove the bad memories we want to forget. As we grow older, dementia becomes one of the top concerns that may affect our lives and those around us.
While artificial intelligence (AI) has taken inspiration from human brains and made remarkable advances in deep learning, one challenge artificial neural networks (ANNs) have experienced is catastrophic forgetting, where the network encounters a sudden and drastic performance drop in previously mastered tasks when learning new information. Catastrophic forgetting typically happens when an ANN is trained to learn multiple tasks sequentially. However, humans can learn large numbers of tasks throughout their lifetime without forgetting those previously learned.
Before delving into AI and catastrophic forgetting, we will first examine how the human brain acquires and holds memories by focusing on three key aspects:
- Declarative and non-declarative memory with temporal time gradients: The temporal aspects of declarative and non-declarative memory provide insight into how the human brain organizes information over time.
- Highly regulated and selective nature of human memory: Human memory exhibits a high degree of regulation and selectivity, influencing the retention and retrieval of information.
- Memory consolidation through replays during sleep: Memory consolidation through replays during sleep sheds light on the processes that strengthen and solidify memories.
Given the complexity of the brain mechanisms to ensure remembering and learning in parallel without inference from each other, we will better understand the potential gaps…