Deep Learning Networks

The course explores basic and advanced topics on MLP (NN1.0), CNN (NN2.0), and NAS (Neural Architecture Search) for deep learning. Basic topics: Sigmoid/ReLU activations, dropout, regularization, and BP learning of net's parameters. More advanced: (1) unifying MLP and CNN learning methods, (2) unifying classification and regression applications, and (3) balancing training and generalization, and (4) applying input/output residual learning to mitigate curse of depth. This ultimately leads to an architecture engineering system (XNAS), a combination of joint parameter/structure X-learning and reinforcement learning paradigms.