The mosaic test: measuring the effectiveness of colour-based image retrieval

William Plant*, Jo Lumsden, Ian Nabney

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content - i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.

Original languageEnglish
Pages (from-to)695-716
Number of pages22
JournalMultimedia Tools and Applications
Volume64
Issue number3
Early online date4 Jan 2012
DOIs
Publication statusPublished - Jun 2013

Fingerprint

Image retrieval
Color
Color films
Color television
Television
Animation

Bibliographical note

The original publication is available at www.springerlink.com

Keywords

  • content-based image retrieval
  • image databases
  • Iimage retrieval
  • performance evaluation
  • query-by-colour
  • query-by-sketch

Cite this

@article{7c5ac38ea323454393fba2974672f19d,
title = "The mosaic test: measuring the effectiveness of colour-based image retrieval",
abstract = "A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content - i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.",
keywords = "content-based image retrieval, image databases, Iimage retrieval, performance evaluation, query-by-colour, query-by-sketch",
author = "William Plant and Jo Lumsden and Ian Nabney",
note = "The original publication is available at www.springerlink.com",
year = "2013",
month = "6",
doi = "10.1007/s11042-011-0951-6",
language = "English",
volume = "64",
pages = "695--716",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer",
number = "3",

}

The mosaic test : measuring the effectiveness of colour-based image retrieval. / Plant, William; Lumsden, Jo; Nabney, Ian.

In: Multimedia Tools and Applications, Vol. 64, No. 3, 06.2013, p. 695-716.

Research output: Contribution to journalArticle

TY - JOUR

T1 - The mosaic test

T2 - measuring the effectiveness of colour-based image retrieval

AU - Plant, William

AU - Lumsden, Jo

AU - Nabney, Ian

N1 - The original publication is available at www.springerlink.com

PY - 2013/6

Y1 - 2013/6

N2 - A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content - i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.

AB - A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content - i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.

KW - content-based image retrieval

KW - image databases

KW - Iimage retrieval

KW - performance evaluation

KW - query-by-colour

KW - query-by-sketch

UR - http://www.scopus.com/inward/record.url?scp=84888642498&partnerID=8YFLogxK

U2 - 10.1007/s11042-011-0951-6

DO - 10.1007/s11042-011-0951-6

M3 - Article

AN - SCOPUS:84888642498

VL - 64

SP - 695

EP - 716

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 3

ER -