Variational inference for Student-t MLP models

Nguyen Thi-hang, Ian T. Nabney

Research output: Contribution to journalArticle

Abstract

This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.
Original languageEnglish
Pages (from-to)2989-2997
Number of pages9
JournalNeurocomputing
Volume73
Issue number16-18
DOIs
Publication statusPublished - Oct 2010

Fingerprint

Neural Networks (Computer)
Multilayer neural networks
Noise
Students
Statistical Models
Contracts
Linear Models
Gases
Datasets

Bibliographical note

10th Brazilian Symposium on Neural Networks (SBRN2008). NOTICE: this is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Thi-hang, Nguyen and Nabney, Ian T. (2010). Variational inference for Student-t MLP models. Neurocomputing, 73 (16-18), pp. 2989-2997. DOI 10.1016/j.neucom.2010.07.009

Keywords

  • variational inference
  • student-t noise
  • multi-layer perceptrons
  • EM algorithm
  • forecast

Cite this

Thi-hang, Nguyen ; Nabney, Ian T. / Variational inference for Student-t MLP models. In: Neurocomputing. 2010 ; Vol. 73, No. 16-18. pp. 2989-2997.
@article{a0d3f64bc33c4915af6b789625865760,
title = "Variational inference for Student-t MLP models",
abstract = "This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.",
keywords = "variational inference, student-t noise, multi-layer perceptrons, EM algorithm, forecast",
author = "Nguyen Thi-hang and Nabney, {Ian T.}",
note = "10th Brazilian Symposium on Neural Networks (SBRN2008). NOTICE: this is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Thi-hang, Nguyen and Nabney, Ian T. (2010). Variational inference for Student-t MLP models. Neurocomputing, 73 (16-18), pp. 2989-2997. DOI 10.1016/j.neucom.2010.07.009",
year = "2010",
month = "10",
doi = "10.1016/j.neucom.2010.07.009",
language = "English",
volume = "73",
pages = "2989--2997",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Elsevier",
number = "16-18",

}

Thi-hang, N & Nabney, IT 2010, 'Variational inference for Student-t MLP models', Neurocomputing, vol. 73, no. 16-18, pp. 2989-2997. https://doi.org/10.1016/j.neucom.2010.07.009

Variational inference for Student-t MLP models. / Thi-hang, Nguyen; Nabney, Ian T.

In: Neurocomputing, Vol. 73, No. 16-18, 10.2010, p. 2989-2997.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Variational inference for Student-t MLP models

AU - Thi-hang, Nguyen

AU - Nabney, Ian T.

N1 - 10th Brazilian Symposium on Neural Networks (SBRN2008). NOTICE: this is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Thi-hang, Nguyen and Nabney, Ian T. (2010). Variational inference for Student-t MLP models. Neurocomputing, 73 (16-18), pp. 2989-2997. DOI 10.1016/j.neucom.2010.07.009

PY - 2010/10

Y1 - 2010/10

N2 - This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.

AB - This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.

KW - variational inference

KW - student-t noise

KW - multi-layer perceptrons

KW - EM algorithm

KW - forecast

UR - http://www.scopus.com/inward/record.url?scp=78650306976&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2010.07.009

DO - 10.1016/j.neucom.2010.07.009

M3 - Article

VL - 73

SP - 2989

EP - 2997

JO - Neurocomputing

JF - Neurocomputing

SN - 0925-2312

IS - 16-18

ER -