Comparison of convergence and stability properties for the state and output solutions of neural networks

A typical neuron cell is characterized by the state variable and the neuron output, which is obtained by passing the state through a nonlinear active device implementing the neuron activation. The paper introduces a wide class of neural networks for which the state solutions and the output solutions enjoy the same convergence and stability properties. The class, which includes as a special case the standard cellular neural networks, is characterized by piecewise-linear Lipschitz continuous neuron activations, Lipschitz continuous (possibly) high-order interconnections between neurons and asymptotically stable isolated neuron cells. The paper also shows that if we relax any of the assumptions on the smoothness of the neuron activations or interconnecting structure, or on the stability of the isolated neuron cells, then the equivalence between the convergence properties of the state solutions and the output solutions is in general no longer guaranteed. To this end, three relevant classes of neural networks in the literature are considered, where each class violates one of the assumptions made in the paper, and it is shown that the state solutions of the networks enjoy stronger convergence properties with respect to the output solutions or viceversa.