Holistic QA: Software Quality Assurance for the Machine Learning Era
An existing 1.0 QA process architecture, the Quality Assurance Machine (QAM), is proposed, to effectively manage both 1.
Abstract
The traditional software space (1.0) has seen more than fifty years of creation, testing, and delivery of deterministic software, but this tradition is being disrupted by machine learning (2.0). However, ML and traditional software are considerably different, and the nascent ML industry uses unique workflows and toolsets for both life cycles. This “one foot in each raft” scenario forces companies to support duplicate resources which are essentially doing the same thing. This paper proposes an answer to the research question: Can software 2.0 quality assurance be performed effectively using the same process as 1.0? A systematic literature review was performed and process documents from a machine learning company reviewed. 132 papers were selected and refined to 24. ML process documents supported what is an industry standard ML life cycle. While the literature review shows a gap relative to holistic QA solutions for ML products, process documents showed the answer to the research question is...perhaps, since it was determined that the typical ML life cycle can be mapped to the standard software 1.0 life cycle. This paper proposes using that mapping to extend an existing 1.0 QA process architecture, the Quality Assurance Machine (QAM), to effectively manage both 1.0 and 2.0 QA.