A summer short course on numerical PDEs and machine learning, sponsored by the Northeast Tianyuan Mathematics Center and presented by Jilin University, was held online from July 5 to July 9, 2020. CCMA director Jinchao Xu was invited as a main lecturer in this short course.
In this five-day course, Xu focused on combining the finite element method, multigrid methods with machine learning. Xu began with an introduction to the basic theory of the finite element, multigrid method and deep learning based on some recent works [1,2,3]. He then discuss the relationship between finite element and ReLU deep neural network [2] and presented a new convolutional neural network (CNN), known as MgNet,[1] which was derived by making minor modifications of a classic multigrid method for solving partial differential equations. Compared with most existing network structures such as ResNet, MgNet has the distinction of its mathematical foundation in view of classic multigrid method and its effective performance for image-classification problems in comparison with existing CNNs. Xu also discussed the approximation property of deep neural networks[3], including the widely used ReLU neural network[4].
In the face of the danger posed by the Covid-19 pandemic, the course was offered online, which both ensured the safety of all the participants and enabled teachers and students to take the course in far greater numbers than would have been possible in a face-to-face context. On average, close to 500 people joined each class session of the five-day course. Further, the online environment did not adversely affect the classroom atmosphere: Many participants contributed to class discussions, and especially the Q&A time for the very last class ran beyond the one-hour mark.
Reference:
[1] Juncai He and Jinchao Xu. “MgNet: A unified framework of multigrid and convolutional neural network.” Science China Mathematics 62, no. 7 (2019): 1331-1354.
[2] Juncai He, Lin Li, Jinchao Xu and Chunyue Zheng. “ReLU Deep Neural Networks and linear Finite Elements.” Journal of Computational Mathematics, (2018)
[3] Siegel Jonathan W. and Jinchao Xu. “Approximation rates for neural networks with general activation functions.” Neural Networks (2020).
[4] Jinchao Xu. “The Finite Neuron Method and Convergence Analysis.” to appear on CiCP, (2020).