A New Framework to Bridge Semantic Gap for Semantics Based Image Retrieval

Abstract

The performance of Content Based Image Retrieval (CBIR) is limited because of the Semantic Gap (SG). This motivates to extend the image retrieval process beyond low level descriptions to image semantics. Therefore, researchers proposed Semantic Based Image Retrieval (SBIR) to bridge SG. In the literature, various approaches for SBIR such as image annotation, relevance feedback, and object ontology have been proposed to bridge SG. It has been observed that not very promising results with these methods are reported in the literature. Annotation based approaches are constrained by low precision because real life images usually have a diverse set of image concepts. Relevance feedback approach is also constrained by low precision and recall because it forces to alter the query vector that causes to modify image semantics. Object ontology is a considerably good approach, but its metadata architecture appears to be very complex and suggested to use only for semantic web instead of SBIR. Moreover, this approach is reported to have low precision for conceptually diverse images. Therefore, the first contribution of our work is the performance evaluation of CBIR methods using soft computing techniques for image comparison and retrieval. In this regard, we have reported performance improvement in terms of retrieving visually similar images by employing proposed modifications in soft computing methods. However, these modifications do not translate into semantically correct retrieval of images. This leads us towards the logical conclusion of planning and development of a new SBIR framework using image concepts. By the term “image concept” we means that a set of noticeable objects and regions in an image e.g. Sky, group of person, land etc. In our first contribution we have also explored existing CBIR approaches in order to find a suitable 11 candidate (or its modified version) to be used for SBIR. The second contribution of our work is therefore to bridge the SG with a maximum tradeoff between precision and recall. The proposed framework is extensively examined by evaluating precision and recall for large segmented datasets. For the rigorous testing of our major contributions, three datasets have been opted, namely, Wang’s, COIL, and IAPR TC-12.