This page contains the theoretical part of the RNN topic for the Deep Learning course at the Master in Artificial Inteligence of the Universitat Politècnica de Catalunya.
Recurrent Neural Networks
These are the slides of this part.
Additional material:
- Understanding LSTM Networks in Colah’s blog
- Stanford CS224 Course, RNN lecture notes
- Zachary C. Lipton, A Critical Review of Recurrent Neural Networks for Sequence Learning, arXiv:1506.00019
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016) The Deep Learning Book chapter 10
- Klaus Greff, Rupesh Kumar Srivastava, Jan Koutnik, Bas R. Steunebrink, Jurgen Schmidhuber, LSTM: A Search Space Odyssey, arXiv:1503.04069
- The Unreasonable Effectiveness of RNNs by Andrej Karpathy
- Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton, Speech Recognition with Deep Recurrent Neural Networks, arXiv:1303.5778 / ICASSP 2013
- Rafal Jozefowicz, Wojciech Zaremba, Ilya Sutskever, An Empirical Exploration of Recurrent Network Architectures, ICML, 2015.
Ultra detailed video about how to implement and train a vanilla RNN to generate characters directly in python and the corresponding jupyter notebook
- Ultra detailed video about how to implement and train a LSTM to generate characters directly in python and the corresponding jupyter notebook