Matrix Orthogonalization Improves Memory in Recurrent Models

Researchers propose orthogonalizing the memory matrix of recurrent neural networks to improve their associative recall abilities, specifically in noisy associative recall tasks.