Outlier detection in scatterometer data

Neural network approaches

Robert Bullen, Dan Cornford, Ian T. Nabney

Research output: Contribution to journalArticle

Abstract

Satellite-borne scatterometers are used to measure backscattered micro-wave radiation from the ocean surface. This data may be used to infer surface wind vectors where no direct measurements exist. Inherent in this data are outliers owing to aberrations on the water surface and measurement errors within the equipment. We present two techniques for identifying outliers using neural networks; the outliers may then be removed to improve models derived from the data. Firstly the generative topographic mapping (GTM) is used to create a probability density model; data with low probability under the model may be classed as outliers. In the second part of the paper, a sensor model with input-dependent noise is used and outliers are identified based on their probability under this model. GTM was successfully modified to incorporate prior knowledge of the shape of the observation manifold; however, GTM could not learn the double skinned nature of the observation manifold. To learn this double skinned manifold necessitated the use of a sensor model which imposes strong constraints on the mapping. The results using GTM with a fixed noise level suggested the noise level may vary as a function of wind speed. This was confirmed by experiments using a sensor model with input-dependent noise, where the variation in noise is most sensitive to the wind speed input. Both models successfully identified gross outliers with the largest differences between models occurring at low wind speeds. © 2003 Elsevier Science Ltd. All rights reserved.

Original languageEnglish
Pages (from-to)419-426
Number of pages8
JournalNeural Networks
Volume16
Issue number3-4
DOIs
Publication statusPublished - 18 Mar 2003

Fingerprint

Noise
Neural networks
Observation
Oceans and Seas
Sensors
Radiation
Equipment and Supplies
Water
Measurement errors
Aberrations
Microwaves
Satellites

Bibliographical note

NOTICE: this is the author’s version of a work that was accepted for publication in Neural Networks. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Bullen, Robert; Cornford, Dan and Nabney, Ian T. (2003). Outlier detection in scatterometer data: neural network approaches. Neural Networks, 16 (3-4), pp. 419-426. DOI 10.1016/S0893-6080(03)00013-3

Keywords

  • generative topographic mapping
  • input-dependent noise
  • neural network sensor models
  • outliers
  • scatterometer

Cite this

Bullen, Robert ; Cornford, Dan ; Nabney, Ian T. / Outlier detection in scatterometer data : Neural network approaches. In: Neural Networks. 2003 ; Vol. 16, No. 3-4. pp. 419-426.
@article{637cb38032ef4fd9b3a7659168ac5a7d,
title = "Outlier detection in scatterometer data: Neural network approaches",
abstract = "Satellite-borne scatterometers are used to measure backscattered micro-wave radiation from the ocean surface. This data may be used to infer surface wind vectors where no direct measurements exist. Inherent in this data are outliers owing to aberrations on the water surface and measurement errors within the equipment. We present two techniques for identifying outliers using neural networks; the outliers may then be removed to improve models derived from the data. Firstly the generative topographic mapping (GTM) is used to create a probability density model; data with low probability under the model may be classed as outliers. In the second part of the paper, a sensor model with input-dependent noise is used and outliers are identified based on their probability under this model. GTM was successfully modified to incorporate prior knowledge of the shape of the observation manifold; however, GTM could not learn the double skinned nature of the observation manifold. To learn this double skinned manifold necessitated the use of a sensor model which imposes strong constraints on the mapping. The results using GTM with a fixed noise level suggested the noise level may vary as a function of wind speed. This was confirmed by experiments using a sensor model with input-dependent noise, where the variation in noise is most sensitive to the wind speed input. Both models successfully identified gross outliers with the largest differences between models occurring at low wind speeds. {\circledC} 2003 Elsevier Science Ltd. All rights reserved.",
keywords = "generative topographic mapping, input-dependent noise, neural network sensor models, outliers, scatterometer",
author = "Robert Bullen and Dan Cornford and Nabney, {Ian T.}",
note = "NOTICE: this is the author’s version of a work that was accepted for publication in Neural Networks. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Bullen, Robert; Cornford, Dan and Nabney, Ian T. (2003). Outlier detection in scatterometer data: neural network approaches. Neural Networks, 16 (3-4), pp. 419-426. DOI 10.1016/S0893-6080(03)00013-3",
year = "2003",
month = "3",
day = "18",
doi = "10.1016/S0893-6080(03)00013-3",
language = "English",
volume = "16",
pages = "419--426",
journal = "Neural Networks",
issn = "0893-6080",
publisher = "Elsevier",
number = "3-4",

}

Outlier detection in scatterometer data : Neural network approaches. / Bullen, Robert; Cornford, Dan; Nabney, Ian T.

In: Neural Networks, Vol. 16, No. 3-4, 18.03.2003, p. 419-426.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Outlier detection in scatterometer data

T2 - Neural network approaches

AU - Bullen, Robert

AU - Cornford, Dan

AU - Nabney, Ian T.

N1 - NOTICE: this is the author’s version of a work that was accepted for publication in Neural Networks. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Bullen, Robert; Cornford, Dan and Nabney, Ian T. (2003). Outlier detection in scatterometer data: neural network approaches. Neural Networks, 16 (3-4), pp. 419-426. DOI 10.1016/S0893-6080(03)00013-3

PY - 2003/3/18

Y1 - 2003/3/18

N2 - Satellite-borne scatterometers are used to measure backscattered micro-wave radiation from the ocean surface. This data may be used to infer surface wind vectors where no direct measurements exist. Inherent in this data are outliers owing to aberrations on the water surface and measurement errors within the equipment. We present two techniques for identifying outliers using neural networks; the outliers may then be removed to improve models derived from the data. Firstly the generative topographic mapping (GTM) is used to create a probability density model; data with low probability under the model may be classed as outliers. In the second part of the paper, a sensor model with input-dependent noise is used and outliers are identified based on their probability under this model. GTM was successfully modified to incorporate prior knowledge of the shape of the observation manifold; however, GTM could not learn the double skinned nature of the observation manifold. To learn this double skinned manifold necessitated the use of a sensor model which imposes strong constraints on the mapping. The results using GTM with a fixed noise level suggested the noise level may vary as a function of wind speed. This was confirmed by experiments using a sensor model with input-dependent noise, where the variation in noise is most sensitive to the wind speed input. Both models successfully identified gross outliers with the largest differences between models occurring at low wind speeds. © 2003 Elsevier Science Ltd. All rights reserved.

AB - Satellite-borne scatterometers are used to measure backscattered micro-wave radiation from the ocean surface. This data may be used to infer surface wind vectors where no direct measurements exist. Inherent in this data are outliers owing to aberrations on the water surface and measurement errors within the equipment. We present two techniques for identifying outliers using neural networks; the outliers may then be removed to improve models derived from the data. Firstly the generative topographic mapping (GTM) is used to create a probability density model; data with low probability under the model may be classed as outliers. In the second part of the paper, a sensor model with input-dependent noise is used and outliers are identified based on their probability under this model. GTM was successfully modified to incorporate prior knowledge of the shape of the observation manifold; however, GTM could not learn the double skinned nature of the observation manifold. To learn this double skinned manifold necessitated the use of a sensor model which imposes strong constraints on the mapping. The results using GTM with a fixed noise level suggested the noise level may vary as a function of wind speed. This was confirmed by experiments using a sensor model with input-dependent noise, where the variation in noise is most sensitive to the wind speed input. Both models successfully identified gross outliers with the largest differences between models occurring at low wind speeds. © 2003 Elsevier Science Ltd. All rights reserved.

KW - generative topographic mapping

KW - input-dependent noise

KW - neural network sensor models

KW - outliers

KW - scatterometer

UR - http://www.scopus.com/inward/record.url?scp=0037381861&partnerID=8YFLogxK

UR - https://www.sciencedirect.com/science/article/pii/S0893608003000133?via%3Dihub

U2 - 10.1016/S0893-6080(03)00013-3

DO - 10.1016/S0893-6080(03)00013-3

M3 - Article

VL - 16

SP - 419

EP - 426

JO - Neural Networks

JF - Neural Networks

SN - 0893-6080

IS - 3-4

ER -