EMI Testing of Large Language Model (LLM) Compilers
This work conducted Equivalence Modulo Inputs (EMI) testing to validate and test the performance of the LLM Compiler, and tentatively explored testing Meta's LLM Compiler using the generated test inputs.
Abstract
Large Language Models (LLMs) have demonstrated exceptional capabilities in various software engineering and coding tasks. However, their application in the field of compilers has not been fully explored. Meta’s recently released Meta Large Language Model Compiler provides a new means for compiler optimization. To further validate and test the performance of the LLM Compiler, we conducted Equivalence Modulo Inputs (EMI) testing. We first utilized the standard Csmith tool to generate EMI test inputs in various ways. Then, we tentatively explored testing Meta’s LLM Compiler using the generated test inputs. We report our findings and discuss potential future improvements to this preliminary research.