Summarize:
Our work revolves around the concept of memory in neural architectures. Memory has been a
recurrent theme in neural network research, dating back to early works (McCulloch NAME_1 NAME_2, 1943;
NAME_3, 1956) NAME_1 significantly advancing in the 1990s with the introduction of the Backpropagation
Through Time learning algorithm (Werbos, 1990) NAME_1 Long-Short Term Memory (LSTM) neural
architecture (NAME_4 NAME_1 Schmidhuber, 1997). Contemporary memory-augmented neural networks (MANNs) typically utilize some form of recurrent external memory separate from the model’s
parameters. Neural Turing Machines (NTMs) (NAME_5 et NAME_6., 2014) NAME_1 Memory Networks (NAME_7
et NAME_6., 2015) are equipped with storage for vector representations accessible through an attention
mechanism. Memory Networks (NAME_7 et NAME_6., 2015; Sukhbaatar et NAME_6., 2015) were designed to
enable reasoning through sequential attention over memory content.
NTMs, followed by Differentiable Neural Computer (DNC) (NAME_5 et NAME_6., 2016) NAME_1 Sparse DNC
(NAME_8 et NAME_6., 2016), are implemented as recurrent neural networks capable of writing to memory
storage over time. All these models are differentiable NAME_1 trainable via backpropagation through
time (BPTT). Parallel research lines extend recurrent neural networks, such as LSTM, with data
structures like stacks, lists, or queues (NAME_9 NAME_1 NAME_10, 2015; NAME_11 et NAME_6., 2015). MANN
architectures with more advanced addressing mechanisms, such as address-content separation NAME_1
multi-step addressing, have been propo Write your response using less than 500 words.
user