Learning the dynamic nature of speech with back-propagation for sequences

A novel learning algorithm is proposed, called Back-Propagation for Sequences (BPS), for a particular class of dynamic neural networks in which some units have local feedback. These networks can be trained to respond to sequences of input patterns and seem particularly suited for phoneme recognition. They exhibit a forgetting behavior and consequently only recently past information is taken into account for classification purposes. BPS permits online weight updating and it has the same time complexity and space requirements as back-propagation (BP) applied to feedforward networks. We present experimental results for problems connected with Automatic Speech Recognition. © 1992.