 |
Prof. Boris Mordukhovich Wayne State University, Detroit, USA In this talk, we study convergence properties of Sharpness-Aware Minimization (SAM) method, which has been proposed quite recently (2021) while has already drawn strong attention of researchers and practitioners due to highly successful applications to models of machine learning, particularly to deep neural networks. However, the fundamental convergence properties of SAM have not been yet investigated in optimization theory. We now establish such properties including local and global convergence of iterates to stationarity points, convergence of the gradients sequence to the origin, and convergence of the iterative sequence to the optimal solution. The universality of our convergence analysis is based on the recently developed inexact gradient descent methods applied to the SAM framework. Numerical experiments are conducted on deep learning models to confirm the practical aspects of our convergence analysis. This talk is based on joint research with P. D. Khanh., H.-C. Luong, and D. B. Tran. |
