Misuse of Automation: The Impact of System Experience on Complacency and Automation Bias in Interaction with Automated Aids

Proceedings of the Human Factors and Ergonomics Society Annual MeetingPublished 1 September 2010

Juliane Reichenbach, Linda Onnasch, Dietrich Manzey

DOI10.1177/154193121005400422

Citations12

SJR score0.26

SNIP0.32

TL;DR

Results show that negative experiences, i.e., automation failures, entail stronger effects on subjective trust in automation as well as the level of complacency and automation bias than positive experiences (correct recommendations of the aid), and commission errors may be due to three different sorts of effects.

Abstract

The study investigates how complacency and automation bias effects in interaction with automated aids are moderated by system experience. Participants performed a supervisory control task supported by an aid for fault identification and management. Groups differed with respect to how long they worked with the aid until eventually an automation failure occurred, and whether this failure was the first or second one the participants were exposed to. Results show that negative experiences, i.e., automation failures, entail stronger effects on subjective trust in automation as well as the level of complacency and automation bias than positive experiences (correct recommendations of the aid). Furthermore, results suggest that commission errors may be due to three different sorts of effects: (1) a withdrawal of attention in terms of incomplete cross-checks of information, (2) an active discounting of contradictory system information, and (3) an inattentive processing of contradictory information analogue to a “looking-but-not-seeing” effect.

Keywords

PsychologyDecision SciencesHealth Professions

Advances in psychologyDevelopment of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research

14,345 Citations1988Sandra G. Hart, Lowell E. Staveland

Human Factors The Journal of the Human Factors and Ergonomics SocietyHumans and Automation: Use, Misuse, Disuse, Abuse

4,010 Citations1997Raja Parasuraman, Victor Riley

Understanding the factors associated with each of these aspects of human use of automation can lead to improved system design, effective training methods, and judicious policies and procedures involving automation use.

ErgonomicsTrust, control strategies and allocation of function in human-machine systems

1,437 Citations1992John D. Lee, Neville Moray

An experiment is reported to characterize the changes in operators' trust during an interaction with a semi-automatic pasteurization plant, and a regression model identifies the causes of changes in trust and a 'trust transfer function' is developed using time series analysis to describe the dynamics of trust.

International Journal of Aviation PsychologyPerformance Consequences of Automation-Induced 'Complacency'

844 Citations1993Raja Parasuraman, Robert M. Molloy +1 more

The effect of variations in the reliability of an automated monitoring system on human operator detection of automation failures was examined in two experiments, providing the first empirical evidence of the performance consequences of these changes.

International Journal of Human-Computer StudiesDoes automation bias decision-making?

580 Citations1999Linda J. Skitka, Kathleen L. Mosier +1 more

Compared error rates in a simulated flight task with and without a computer that monitored system states and made decision recommendations, participants in non-automated settings out-performed their counterparts with a very but not perfectly reliable automated aid on a monitoring task.

International Journal of Human-Computer StudiesMisuse of automated decision aids: Complacency, automation bias and the impact of training experience

254 Citations2008J. Elin Bahner, Anke-Dorothea Hüper +1 more

Examining the effect of a preventive training intervention which includes exposing participants to rare automation failures during training significantly decreased complacency and thus represents a suitable means to reduce this risk, even though it might not avoid it completely.

Proceedings of the Human Factors and Ergonomics Society Annual MeetingMisuse of Diagnostic Aids in Process Control: The Effects of Automation Misses on Complacency and Automation Bias

34 Citations2008J. Elin Bahner, Monika Elepfandt +1 more

The effects of misses of an automated alarm and fault diagnosis system on different manifestations of automation misuse were examined and reduced complacency towards the alarm function of the decision aid as well as omission errors but did neither affect complacence towards the aid's diagnostic function nor commission errors.

Carl von Ossiezky University of OldenburgA Case Study for Using a Cognitive Model of Learned Carelessness in Cognitive Engineering

8 Citations2005Andreas Lüdtke, Claus Möbus

A methodology for automatically predicting potential pilot errors during the design of mode-based systems for modern cockpits is developed using a cognitive model of pilot behaviour allowing to predict misconceptions in the pilot's mental procedure model based on a cognitive process called "learned carelessness".

View PDF