Visual Objects in the Global Graph

Abstract

Here, we use the premise of an actual image-retrieval application to examine how various approaches and techniques in computer vision help to bridge the much talked about semantic gap. A lot of cross-fertilization of ideas from the world of text processing has found its way into image processing as well. When images are processed for various purposes, their low-level features usually bear no resemblance with the types of concepts used in describing them. When tasked with developing a useful image model, most strategies take a data-driven or ontological route, or a combination of both. Whatever strategy is adopted, we observe how different image contexts are used to derive some type of semantic knowledge. In this study, we provide an analysis of how an ontological model can be derived from the structural composition of clustered features that result from an image-retrieval task, especially focusing on error pairs. In other words, we explore the additional contexts in which the semantic gap can be narrowed, when the search context for images relative to a large database of features, is also narrowed. We use a small sample of games set to train and eventually test how effective our image-retrieval task can find and match an image based on its low-level features. In so doing, we had wanted to create the basis for potentially pairing these unique low-level features to a higher-level concept based on scene class, for instance. But ultimately for each image-retrieval task, we keenly recognize when errors do occur, under different object descriptor and search strategies, and particularly look out for consistent error patterns across these descriptors, based on the retrieved results from an image search. We discover an additional context for deriving semantic knowledge about the query image, providing for the basis to develop another data-driven ontological model.