A Preconditioned Unconstrained Optimization Method for Training Multilayer Feed-forward Neural Network

Abstract

Non-linear unconstrained optimization methods constitute excellent neural network training methods characterized by their simplicity and efficiency. In this paper, we propose a new preconditioned conjugate gradient neural network training algorithm which guarantees descent property with standard Wolfe condition. Encouraging numerical experiments verify that the proposed algorithm provides fast and stable convergence.