Mixtures of probabilistic principal component analysers

Michael E. Tipping, Christopher M. Bishop

Research output: Contribution to journalArticlepeer-review

Abstract

Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition.
Original languageEnglish
Pages (from-to)443-482
Number of pages40
JournalNeural Computation
Volume11
Issue number2
DOIs
Publication statusPublished - 15 Feb 1999

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • Principal component analysis
  • projections
  • non-linear variants
  • probabilistic
  • compression
  • handwritten
  • digit recognition

Fingerprint

Dive into the research topics of 'Mixtures of probabilistic principal component analysers'. Together they form a unique fingerprint.

Cite this