A Review of Optimization Methods in Deep Learning - Khatana, Amita; Narang, VK; Thada, Vikas (International Journal of Computer Sciences and Engineering, 2025)
Deep learning technique is an emerging field of machine learning. In recent years, it has been successfully used in different fields, such as image classification, natural language processing, computer vision, speech reorganization, etc. When compared to the machine learning, deep learning has a high learning ability to extract features of large datasets. Deep learning came into existence in 1971 when Ivakhnenka used group method of data handling algorithm (GMDH) to train 8-layered neural network [1]. This paper focuses on the artificial neural network, learning techniques and optimization methods of deep learning like stochastic gradient descent, batch gradient descent, mini-batch gradient descent and ADAM.