Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. While the cross-entropy loss is usually justified from a probabilistic perspective, this paper shows an alternative and more direct interpretation of this loss in terms of t-norms and their associated generator functions, and derives a general relation between loss functions and t-norms. In particular, the presented work shows intriguing results leading to the development of a novel class of loss functions. These losses can be exploited in any supervised learning task and which could lead to faster convergence rates that the commonly employed cross-entropy loss.

Giannini, F., Marra, G., Diligenti, M., Maggini, M., Gori, M. (2020). On the relation between Loss Functions and T-Norms. In Proceedings of the Conference on Inductive Logic Programming (pp.36-45). Cham : Springer [10.1007/978-3-030-49210-6_4].

On the relation between Loss Functions and T-Norms

Giannini, Francesco;Marra, Giuseppe;Diligenti, Michelangelo;Maggini, Marco;Gori, Marco
2020-01-01

Abstract

Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. While the cross-entropy loss is usually justified from a probabilistic perspective, this paper shows an alternative and more direct interpretation of this loss in terms of t-norms and their associated generator functions, and derives a general relation between loss functions and t-norms. In particular, the presented work shows intriguing results leading to the development of a novel class of loss functions. These losses can be exploited in any supervised learning task and which could lead to faster convergence rates that the commonly employed cross-entropy loss.
2020
978-3-030-49209-0
978-3-030-49210-6
Giannini, F., Marra, G., Diligenti, M., Maggini, M., Gori, M. (2020). On the relation between Loss Functions and T-Norms. In Proceedings of the Conference on Inductive Logic Programming (pp.36-45). Cham : Springer [10.1007/978-3-030-49210-6_4].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11365/1082451