This is a draft course description for CE III: Deep Learning taught in Fall 2023.

Course content

Basics of deep learning, learning networks, Shallow vs. Deep learning etc.; Machine learning theory – training and test sets, evaluation, etc. Theory of Generalization; Multi-layer perceptrons, error back-propagation; Deep convolutional networks, Computational complexity of feed forward and deep convolutional neural networks; Unsupervised deep learning including auto-encoders; Deep belief networks; Restricted Boltzman Machines; Deep Recurrent Neural Networks (BPTT, LSTM, etc.); GPU programming for deep learning CuDNN; Generative adversarial networks (GANs); Sparse coding and auto-encoders; Data augmentation, elastic distortions, data normalization; Mitigating overfitting with dropout, batch normalization, dropconnect; Novel architectures, ResNet, GoogleNet, etc