EXPLOITING SENSOR DATA SEMANTICS FOR SMARTPHONE-BASED LIFELOGGING: TOWARDS THE DEVELOPMENT OF DIGITAL PROSTHETIC MEMORY ON SMARTPHONES

Abstract

The paradigm of lifelogging promises the development of automatic systems for recording users' life events information digitally and develops an electronic prosthetic memory for providing complementary assistance to human biological memory. Several lifelogging systems are invented by assimilating computing and sensory technologies to capture, annotate, and retrieve lifelong information. The wearable technology has gained market traction; however, has several limitations including hard-to-work environment, number of sensors, uni-faceted, etc. Therefore, they are unable to use contextual semantics for organizing lifelog information like in human episodic memory. In addition, the large-scale adaptation of lifelogging is possible, if lifelogging functionality is integrated in devices that are already owned and maintained by users. To bridge the gap, this thesis examines smartphone technology for developing a common understanding of using smartphone as a de-facto lifelogging device. The various contextual semantics from smartphone sensors data and their potential applications in lifelogging are identified. The semantic model (i.e., ontology) is developed and tested for using the contextual semantics to organize, annotate, and relate lifelog information in similar to human episodic memory, and provide enough contextual cues to recall lifelog information like associative recall in human memory. The semantic framework is proposed to unify the research efforts by incorporating smartphone's sensors and processing capabilities, and semantic model to develop a semantically enriched digital prosthetic memory on smartphone. The proof-of-concept application called Semantic Lifelogging (SLOG) is developed to show the practicality of the proposed framework. The empirical evaluation has shown effectiveness of the methodology. In addition, future research directions are highlighted to help researchers in finding research topics.