On August 22, 2020, CCMA Director Jinchao Xu was invited to teach a mini-course “Introduction to Deep Learning” as part of the 2020 PKU Biostatistics Summer School hosted by the PKU Department of Biostatistics and the Beijing International Center for Mathematical Research.
In this course, which consisted of a three-hour lecture, Prof. Xu began his discussion with an elementary introduction to deep learning, including deep neural network models, training algorithms, and applications. Next, he introduced a general function class of deep neural networks and discussed their mathematical properties. He discussed the convolutional neural network (CNN) as a special deep neural network and considered its applications for image classification. In particular, he presented a new CNN framework, known as MgNet[1], which is motivated and derived from the classical multigrid methods used to solve discretized partial differential equations. He also analyzed the most commonly used training algorithms, including the stochastic gradient descent method (SGD), and introduced the new extended regularized dual averaging (XRDA) method[2][3] as an alternative, noting that both of these can be effective for training sparse deep neural networks. Finally, he introduced some applications for biostatistics.
More than 200 participants, principally graduate students and professors, attended this lecture, which ended with an engaging and informative Q&A session.
References:
[1] He, Juncai, and Jinchao Xu. “MgNet: A unified framework of multigrid and convolutional neural network.” Science China Mathematics 62, no. 7 (2019): 1331-1354.
[2] He, Juncai, Xiaodong Jia, Jinchao Xu, Lian Zhang, and Liang Zhao. “Make \ell_1 regularization effective in training sparse CNN.” Computational Optimization and Applications 77, no. 1 (2020): 163-182.
[3] Siegel, Jonathan W., and Jinchao Xu. “Extended Regularized Dual Averaging.” arXiv preprint arXiv:1904.02316 (2019).