A Heuristic Global Optimisation Algorithm for Training Recurrent Neural Networks

Recurrent neural networks are an efficient tool for the solution of problems of automatic speech recognition. On the other hand, using the Reduced Gradient (RG) algorithm for isolated word recognition is known to be better than Backpropagation in locally recurrent architectures. The implementation of RG guarantees the convergence to a Kuhn-Tucker point which, however, is in general neither the global minimum nor a satisfactory sub-optimal solution. In this paper we have experimented a new heuristic method based on a combined use of RG and a deterministic global algorithm for unconstrained optimisation. The preliminary numerical experiences of such method have evidenced very promising results.