Unsupervised Nonparametric Density Estimation: A Neural Network Approach

One major problem in pattern recognition is estimating probability density functions. Unfortunately, parametric techniques rely on an arbitrary assumption on the form of the underlying, unknown density function. On the other hand, nonparametric techniques, such as the popular k(n)-Nearest Neighbor (not to be confused with the k-Nearest Neighbor classification algorithm), allow to remove such an assumption. Albeit effective, the k(n)-Nearest Neighbor is affected by a number of limitations. Artificial neural networks are, in principle, an alternative family of nonpararnetric models. So far, artificial neural networks have been extensively used to estimate probabilities (e.g., class-posterior probabilities). However, they have not been exploited to estimate instead probability density functions. This paper introduces a simple, neural-based algorithm for unsupervised, nonparametric estimation of multivariate densities, relying on the k(n)-Nearest Neighbor technique. This approach overcomes the limitations of k(n)-Nearest Neighbor, possibly improving the estimation accuracy of the resulting pdf models. An experimental investigation of the algorithm behavior is offered, exploiting random samples drawn from a mixture of Fisher-Tippett density functions.