The problem of inductive inference of regular grammars has recently been faced with recurrent neural networks by many researchers [Giles et al., 1992; Pollack, 1991; Watrous & Kuhn, 1992]. We claim that recurrent radial basis function (R 2 BF ) networks [Gori et al., 1993a] are very well-suited for dealing with such inferential problems, because of their clustered representation of the network states. On the other hand, the main problems that seem to affect the success of such inferential methods is that of gradient vanishing [Bengio et al., 1993] and of bifurcation of the weight space trajectory [Doya, 1993] when learning long-term dependencies , no matter what recurrent network is used. In particular, in this paper, we propose using a modular neural network architecture in which the activations of each module are updated with their own rate.

Gori, M., Maggini, M., G., S. (1994). Scheduling of Modular Architectures for Inductive Inference of Regular Grammars. In Proceedings of the workshop on Combining Symbolic and Connectionist Processing at the 11th European Conference on Artificial Intelligence (ECAI '94) (pp.78-87).

Scheduling of Modular Architectures for Inductive Inference of Regular Grammars

GORI, MARCO;MAGGINI, MARCO;
1994-01-01

Abstract

The problem of inductive inference of regular grammars has recently been faced with recurrent neural networks by many researchers [Giles et al., 1992; Pollack, 1991; Watrous & Kuhn, 1992]. We claim that recurrent radial basis function (R 2 BF ) networks [Gori et al., 1993a] are very well-suited for dealing with such inferential problems, because of their clustered representation of the network states. On the other hand, the main problems that seem to affect the success of such inferential methods is that of gradient vanishing [Bengio et al., 1993] and of bifurcation of the weight space trajectory [Doya, 1993] when learning long-term dependencies , no matter what recurrent network is used. In particular, in this paper, we propose using a modular neural network architecture in which the activations of each module are updated with their own rate.
1994
Gori, M., Maggini, M., G., S. (1994). Scheduling of Modular Architectures for Inductive Inference of Regular Grammars. In Proceedings of the workshop on Combining Symbolic and Connectionist Processing at the 11th European Conference on Artificial Intelligence (ECAI '94) (pp.78-87).
File in questo prodotto:
File Dimensione Formato  
ECAI94.pdf

non disponibili

Tipologia: Pre-print
Licenza: NON PUBBLICO - Accesso privato/ristretto
Dimensione 323.44 kB
Formato Adobe PDF
323.44 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11365/38153
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo