Makhataeva, Z., Akhmetov, T., & Varol, H. A. (2023). Augmented-Reality-Based Human Memory Enhancement Using Artificial Intelligence. IEEE Transactions on Human-Machine Systems

Abstract: This work presents a human memory augmentation system that uses augmented reality (AR), computer vision (CV), and artificial intelligence to replace the internal mental representation of objects in the environment with an external augmented representation. The system consists of two components: 1) an AR headset; and 2) a computing station. The AR headset runs an application that senses the indoor environment, sends data to the computing station for processing, receives the processed data, and updates the external representation of objects using a virtual 3-D object projected into the real environment in front of the user's eyes. The computing station performs CV-based indoor environment self-localization, object detection, and object-to-location binding using first-person view data received from the AR headset. We designed a behavioral study to evaluate the usability of the system. In a pilot study with 26 participants (12 females and 14 males), we investigated human performance in an experimental task that involved remembering the positions of objects in a physical space and displaying the positions of the learned objects on the 2-D map of the space. We conducted the studies under two conditions—that is, with and without using the AR system. We investigated the usability of the system, subjective workload, and performance variables under both conditions. The results showed that the AR-based augmentation of the mental representation of objects indoors reduced cognitive load and increased performance accuracy.