OWL2 Benchmarking for the Evaluation of the Knowledge Based Systems' Platforms

Abstract

Data modelling using OWL semantics for the development of a knowledge based system (KBS) has recently attracted the attention of many applications in various domains like Business, Biosciences, Health, and Digital Libraries etc. Well-known knowledge base systems platforms (KBSPs) are used for storing and querying the ontology-based applications using di erent storage formats (memory, graph, le, and database). Choosing an appropriate KBSP is considered as an important task to help domain experts to select suitable KBSP. In this research, a problem with the current state of the art evaluation benchmarks has been identi ed; the existing state of the art evaluation benchmarks are not designed to support complete OWL semantics (OWL1.1 and OWL2). The objectives of the research include; inspection of the existing evaluation benchmarks for the missing OWL semantics, construction of benchmark with complete OWL covergae, analysis of the proposed benchmark and evaluation of the KBSPs using proposed benchmark. In this research, the proposed OWL2 benchmark (OEB2) for the evaluation of the KBSPs is constructed using the foundational building blocks of the evaluation benchmark: data schema, dataset, and queryset with performance evaluation matric. The proposed work uses university ontology as case study in the construction of OEB2. The complete OWL semantics are added in the data schema of the proposed benchmark through survey of relevant ontologies, usage of WordNet senses, and addition of property characteristics through patterned queries. The dataset of the proposed benchmark is enriched with all the assertion level OWL semantics and coverage of all the property characterisitics. The coverage of OWL semantics in the queries set are covered by classifying the queries and also making them more generic. Finally, the proposed benchmark has been tested on the memory based, le based, graph based, and relational database KBSPs for the performance and scalability measures. The results show that OEB2 is able to evaluate the behaviour of di erent KBSPs with complete OWL semantics (OWL1.1 and OWL2). The reported results provides an evidence that di erent knowledge base system are suitable for di erent domains. The present work assist domain experts to choose a relevant knowledge base system based on the nature of their domain. This research also concludes with multiple directions for future research in this domain.