Questioning the AI: Informing Design Practices for Explainable AI User Experiences
An algorithm-informed XAI question bank is developed in which user needs for explainability are represented as prototypical questions users might ask about the AI, and used as a study probe to identify gaps between current XAI algorithmic work and practices to create explainable AI products.
Abstract
A surge of interest in explainable AI (XAI) has led to a vast collection of\nalgorithmic work on the topic. While many recognize the necessity to\nincorporate explainability features in AI systems, how to address real-world\nuser needs for understanding AI remains an open question. By interviewing 20 UX\nand design practitioners working on various AI products, we seek to identify\ngaps between the current XAI algorithmic work and practices to create\nexplainable AI products. To do so, we develop an algorithm-informed XAI\nquestion bank in which user needs for explainability are represented as\nprototypical questions users might ask about the AI, and use it as a study\nprobe. Our work contributes insights into the design space of XAI, informs\nefforts to support design practices in this space, and identifies opportunities\nfor future XAI work. We also provide an extended XAI question bank and discuss\nhow it can be used for creating user-centered XAI.\n