The Decoupling Network Assumptions for Optimal Learning in Recurrent Neural Networks

Many researchers have recently focused their efforts on devising efficient algorithms, mainly based on optimization schemes, for learning the weights of recurrent neural networks. Like for feedforward networks, however, these learning algorithms may get stuck in local minima during gradient descent, thus discovering sub-optimal solutions. In this paper, we give sufficient conditions which guarantee local minima free error surfaces. Moreover, we provide an example which shows the constructive role of the proposed theory in designing networks suitable for solving a given task.