THECAAV19

Abstract: The maintenance of systems that must operate over long periods of time is expensive, knowledge-intensive, and involves the precise execution of a complex sequence of tasks. Dr. Shivakumar Sastry presents an approach, based on Augmented Reality (AR) that guides the completion of such task sequences in 3D environments by providing just-in-time informatics and guidance to support the operators who carry out the maintenance tasks. We describe a novel architecture that enables the efficient storage, retrieval and update of the AR artifacts that are necessary to guide the completion of tasks. To effectively situate the informatics in the operational environment of the user, it is necessary to annotate objects and artifacts of the physical world within the field-of-view of an AR device. Instead of using the popular, computationally intensive approaches based on computer vision and image processing, Dr. Sastry uses a CAD model of the engineered system. In this approach, he transforms the gaze point of the user in the virtual world in the AR field-of-view, to the corresponding coordinates of the 3D CAD model for the system. He then exploit the details in the CAD model to localize the user in the environment and appropriately annotates the objects and artifacts of the physical world to guide the operators. He demonstrates that the 3D models required to support this approach do not have to be very detailed for carrying out macro-scale maintenance activities. The creation and maintenance of the AR artifacts to support such an approach are expensive, time-consuming and error prone. To address this challenge, he will discuss the role of intelligent methods and systems to automatically generate the AR content using available design and engineering artifacts in engineered system environments. In the future, this approach can extend to include social, emotional and behaviorial factors that impact the frontier of work at the interface of humans and machines.