Master of Science, The Ohio State University, 2021, Industrial and Systems Engineering

Capable computational technology is a necessary but not a sufficient condition for engineering a high-performing human-machine team (HMT). The interactions between human and machine agents can have substantial and surprising positive or negative effects on the overall performance of the system. This implies two requirements for robust HMT design: a set of design strategies to explicitly support joint cognitive functions and reliable techniques to evaluate how this support affects overall system performance. This study proposes simultaneous inference and data (SID) displays as a novel design technique to enable improved human-machine performance and joint activity graphs as a technique to evaluate this performance from a limited testing set. SID displays provide increased observability to computational algorithms by annotating a base data display with how the algorithm is interpreting the underlying data. Joint activity graphs compare performance of the joint HMT against the reference of the machine alone, extrapolating how each will perform outside the set of discrete testing cases. The results show SID displays enabled the HMT to significantly diverge from increasingly incorrect machine guidance, leading to substantial improvements in performance, especially when the machine guidance was wrong. Therefore, SID displays and joint activity graphs appear to be promising techniques for designing and evaluating joint cognitive systems that mitigate the negative consequences of incorrect machine guidance.

Committee: Michael Rayo (Advisor); David Woods (Committee Member) Subjects: Artificial Intelligence; Industrial Engineering; Systems Design; Systems Science