The problem of inductive inference of regular grammars has recently been faced with recurrent neural networks by many researchers [Giles et al., 1992; Pollack, 1991; Watrous & Kuhn, 1992]. We claim that recurrent radial basis function (R 2 BF ) networks [Gori et al., 1993a] are very well-suited for dealing with such inferential problems, because of their clustered representation of the network states. On the other hand, the main problems that seem to affect the success of such inferential methods is that of gradient vanishing [Bengio et al., 1993] and of bifurcation of the weight space trajectory [Doya, 1993] when learning long-term dependencies , no matter what recurrent network is used. In particular, in this paper, we propose using a modular neural network architecture in which the activations of each module are updated with their own rate.
Gori, M., Maggini, M., G., S. (1994). Scheduling of Modular Architectures for Inductive Inference of Regular Grammars. In Proceedings of the workshop on Combining Symbolic and Connectionist Processing at the 11th European Conference on Artificial Intelligence (ECAI '94) (pp.78-87).
Scheduling of Modular Architectures for Inductive Inference of Regular Grammars
GORI, MARCO;MAGGINI, MARCO;
1994-01-01
Abstract
The problem of inductive inference of regular grammars has recently been faced with recurrent neural networks by many researchers [Giles et al., 1992; Pollack, 1991; Watrous & Kuhn, 1992]. We claim that recurrent radial basis function (R 2 BF ) networks [Gori et al., 1993a] are very well-suited for dealing with such inferential problems, because of their clustered representation of the network states. On the other hand, the main problems that seem to affect the success of such inferential methods is that of gradient vanishing [Bengio et al., 1993] and of bifurcation of the weight space trajectory [Doya, 1993] when learning long-term dependencies , no matter what recurrent network is used. In particular, in this paper, we propose using a modular neural network architecture in which the activations of each module are updated with their own rate.File | Dimensione | Formato | |
---|---|---|---|
ECAI94.pdf
non disponibili
Tipologia:
Pre-print
Licenza:
NON PUBBLICO - Accesso privato/ristretto
Dimensione
323.44 kB
Formato
Adobe PDF
|
323.44 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11365/38153
Attenzione
Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo