Neural network modelling with input uncertainty

theory and application

Dan Cornford, W.A. Wright, Guillaume Ramage, Ian T. Nabney

Research output: Contribution to journalArticle

Abstract

It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network’s weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.
Original languageEnglish
Pages (from-to)169-188
Number of pages20
JournalJournal of VLSI Signal Processing Systems for Signal Image and Video Technology
Volume26
Issue number1-2
DOIs
Publication statusPublished - 2000

Fingerprint

Neural networks
Radar systems
Markov processes
Monte Carlo methods
Satellites
Sampling
Uncertainty

Bibliographical note

The original publication is available at www.springerlink.com

Keywords

  • Bayesian inference methods
  • neural networks
  • noise
  • errors in variable
  • Bayesian neural network framework
  • input noise
  • noise process exists
  • noiseless input
  • Markov chain Monte Carlo
  • satellite radar backscatter system
  • sea surface wind vectors

Cite this

Cornford, Dan ; Wright, W.A. ; Ramage, Guillaume ; Nabney, Ian T. / Neural network modelling with input uncertainty : theory and application. In: Journal of VLSI Signal Processing Systems for Signal Image and Video Technology. 2000 ; Vol. 26, No. 1-2. pp. 169-188.
@article{94febeccae0144258b0cac4cb34ee9a2,
title = "Neural network modelling with input uncertainty: theory and application",
abstract = "It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network’s weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.",
keywords = "Bayesian inference methods, neural networks, noise, errors in variable, Bayesian neural network framework, input noise, noise process exists, noiseless input, Markov chain Monte Carlo, satellite radar backscatter system, sea surface wind vectors",
author = "Dan Cornford and W.A. Wright and Guillaume Ramage and Nabney, {Ian T.}",
note = "The original publication is available at www.springerlink.com",
year = "2000",
doi = "10.1023/A:1008111920791",
language = "English",
volume = "26",
pages = "169--188",
journal = "Journal of VLSI Signal Processing Systems for Signal Image and Video Technology",
issn = "0922-5773",
publisher = "Kluwer",
number = "1-2",

}

Neural network modelling with input uncertainty : theory and application. / Cornford, Dan; Wright, W.A.; Ramage, Guillaume; Nabney, Ian T.

In: Journal of VLSI Signal Processing Systems for Signal Image and Video Technology, Vol. 26, No. 1-2, 2000, p. 169-188.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Neural network modelling with input uncertainty

T2 - theory and application

AU - Cornford, Dan

AU - Wright, W.A.

AU - Ramage, Guillaume

AU - Nabney, Ian T.

N1 - The original publication is available at www.springerlink.com

PY - 2000

Y1 - 2000

N2 - It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network’s weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.

AB - It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which accounts for input noise provided that a model of the noise process exists. In the limit where the noise process is small and symmetric it is shown, using the Laplace approximation, that this method adds an extra term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable, and sampling this jointly with the network’s weights, using a Markov chain Monte Carlo method, it is demonstrated that it is possible to infer the regression over the noiseless input. This leads to the possibility of training an accurate model of a system using less accurate, or more uncertain, data. This is demonstrated on both the, synthetic, noisy sine wave problem and a real problem of inferring the forward model for a satellite radar backscatter system used to predict sea surface wind vectors.

KW - Bayesian inference methods

KW - neural networks

KW - noise

KW - errors in variable

KW - Bayesian neural network framework

KW - input noise

KW - noise process exists

KW - noiseless input

KW - Markov chain Monte Carlo

KW - satellite radar backscatter system

KW - sea surface wind vectors

UR - http://www.scopus.com/inward/record.url?scp=0034244839&partnerID=8YFLogxK

UR - http://www.springerlink.com/content/u33127m600401485/

U2 - 10.1023/A:1008111920791

DO - 10.1023/A:1008111920791

M3 - Article

VL - 26

SP - 169

EP - 188

JO - Journal of VLSI Signal Processing Systems for Signal Image and Video Technology

JF - Journal of VLSI Signal Processing Systems for Signal Image and Video Technology

SN - 0922-5773

IS - 1-2

ER -