Real-Time AI Assistance for Disorienting Control Tasks: Performance, Behavior, and Trust

Abstract

Spatial awareness is an important ability humans develop to use in everyday activities like walking and driving. It is an even more critical skill required in high-risk occupations, such as piloting an airplane or spacecraft. Erroneous inputs to the sensory system can lead to spatial disorientation, rendering a person unable to interpret their speed, position, and orientation with respect to other objects or the horizon. A report by the Federal Aviation Administration indicates that 367 fatal accidents in general aviation could be attributed to spatial disorientation among pilots between 2003 and 2021. An AI system situated in the problem space could monitor the vehicle's position and orientation, determine when loss of control is imminent, and provide corrective maneuvers to avoid accidents, all in real time. This work presents a first-of-a-kind AI system designed to assist humans with visual aids in disorienting continuous-control tasks, specifically the Virtual Inverted Pendulum and a navigation task in a flight simulator. An offline evaluation reveals that an AI system can select actions, per the task's specifications, objectively better than humans. The results align with the hypothesis that an AI system is not susceptible to the same sensory disruptions as humans and can therefore excel at the task and guide humans toward more accurate actions. This dissertation demonstrates, through multiple human-subject studies, that bi-directional learning can improve specific task performance metrics in both humans and AI while aligning the AI agent's resulting actions with human intuition, but sometimes at the cost of AI performance. Empirical evidence also indicates that humans prefer to trust AI systems that are more closely aligned with human behavior, and that they also trust intuitive and non-intrusive forms of assistance, even when there is no objective improvement in task performance.