Evaluation and benchmarking of image database navigation tools

William Plant*, Gerald Schaefer

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for image database navigation tools. Such systems allow the browsing of large image collections in an intuitive manner enabling the user to find or annotate images effectively and efficiently. We present previous work aimed at testing such systems and propose a set of guidelines for a standardised benchmark that can be used to evaluate and compare different image browsing approaches. We argue that such a benchmark needs to be based on a standardised image database, and should include user tasks such as target and category search, and annotation as well as record the opinion of the users. The performance on each of the selected tasks needs to be quantifiable by a defined set of measures in order to allow for comparison between systems.

Original languageEnglish
Title of host publicationProceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009
PublisherCSREA
Pages248-254
Number of pages7
Volume1
ISBN (Print)978-1-6013-2119-0
Publication statusPublished - 1 Dec 2009
EventInternational Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009 - Las Vegas, NV, United States
Duration: 13 Jul 200916 Jul 2009

Conference

ConferenceInternational Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009
CountryUnited States
CityLas Vegas, NV
Period13/07/0916/07/09

Fingerprint

Benchmarking
Navigation
Image retrieval
Testing

Keywords

  • Benchmarking
  • Browsing
  • Image database navigation
  • Image databases
  • Image retrieval
  • Visualisation

Cite this

Plant, W., & Schaefer, G. (2009). Evaluation and benchmarking of image database navigation tools. In Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009 (Vol. 1, pp. 248-254). CSREA.
Plant, William ; Schaefer, Gerald. / Evaluation and benchmarking of image database navigation tools. Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009. Vol. 1 CSREA, 2009. pp. 248-254
@inproceedings{14f9f25c6b1c48b891018af2d2a719a9,
title = "Evaluation and benchmarking of image database navigation tools",
abstract = "Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for image database navigation tools. Such systems allow the browsing of large image collections in an intuitive manner enabling the user to find or annotate images effectively and efficiently. We present previous work aimed at testing such systems and propose a set of guidelines for a standardised benchmark that can be used to evaluate and compare different image browsing approaches. We argue that such a benchmark needs to be based on a standardised image database, and should include user tasks such as target and category search, and annotation as well as record the opinion of the users. The performance on each of the selected tasks needs to be quantifiable by a defined set of measures in order to allow for comparison between systems.",
keywords = "Benchmarking, Browsing, Image database navigation, Image databases, Image retrieval, Visualisation",
author = "William Plant and Gerald Schaefer",
year = "2009",
month = "12",
day = "1",
language = "English",
isbn = "978-1-6013-2119-0",
volume = "1",
pages = "248--254",
booktitle = "Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009",
publisher = "CSREA",

}

Plant, W & Schaefer, G 2009, Evaluation and benchmarking of image database navigation tools. in Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009. vol. 1, CSREA, pp. 248-254, International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009, Las Vegas, NV, United States, 13/07/09.

Evaluation and benchmarking of image database navigation tools. / Plant, William; Schaefer, Gerald.

Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009. Vol. 1 CSREA, 2009. p. 248-254.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Evaluation and benchmarking of image database navigation tools

AU - Plant, William

AU - Schaefer, Gerald

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for image database navigation tools. Such systems allow the browsing of large image collections in an intuitive manner enabling the user to find or annotate images effectively and efficiently. We present previous work aimed at testing such systems and propose a set of guidelines for a standardised benchmark that can be used to evaluate and compare different image browsing approaches. We argue that such a benchmark needs to be based on a standardised image database, and should include user tasks such as target and category search, and annotation as well as record the opinion of the users. The performance on each of the selected tasks needs to be quantifiable by a defined set of measures in order to allow for comparison between systems.

AB - Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for image database navigation tools. Such systems allow the browsing of large image collections in an intuitive manner enabling the user to find or annotate images effectively and efficiently. We present previous work aimed at testing such systems and propose a set of guidelines for a standardised benchmark that can be used to evaluate and compare different image browsing approaches. We argue that such a benchmark needs to be based on a standardised image database, and should include user tasks such as target and category search, and annotation as well as record the opinion of the users. The performance on each of the selected tasks needs to be quantifiable by a defined set of measures in order to allow for comparison between systems.

KW - Benchmarking

KW - Browsing

KW - Image database navigation

KW - Image databases

KW - Image retrieval

KW - Visualisation

UR - http://www.scopus.com/inward/record.url?scp=84864931282&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84864931282

SN - 978-1-6013-2119-0

VL - 1

SP - 248

EP - 254

BT - Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009

PB - CSREA

ER -

Plant W, Schaefer G. Evaluation and benchmarking of image database navigation tools. In Proceedings of the 2009 International Conference on Image Processing, Computer Vision, and Pattern Recognition, IPCV 2009. Vol. 1. CSREA. 2009. p. 248-254