Non−suspiciousness: A Generalisation of Convexity in the Frame of Foundations of Numerical Analysis and Learning

The effectiveness of connectionist models in emulating intelligent behaviour is strictly related to the capability of the learning algorithms to find optimal or near-optimal solutions. In this paper, a canonical reduction of gradient descent dynamics is proposed, allowing the formulation of the neural network learning as a finite continuous optimisation problem, under some non-suspiciousness conditions. In the linear case, the non-suspect nature of the problem guarantees the implementation of an iterative method with O(n^2) as computational complexity. Finally, since non-suspiciousness is a generalisation of the concept of convexity, it is possible to apply this theory to the resolution of nonlinear problems.