Retrieval Augmented Generation (RAG) Based Restaurant Chatbot with AI Testability
The paper addresses the enhancement of AI chatbots through the integration of Retrieval-Augmented Generation with the Large Language Model (LLM) and focuses on the development of a restaurant chatbot that not only engages in natural-language conversations but also addresses context optimization and LLM optimization for restaurant context learning.
Abstract
Post-COVID the restaurant industry is experiencing a surge in demand, presenting a unique challenge of efficiently managing increased customer flow while ensuring seamless interactions. Chatbots have emerged as an innovative solution to meet the demand increase. The paper addresses the enhancement of AI chatbots through the integration of Retrieval-Augmented Generation (RAG) with the Large Language Model (LLM). This paper focuses on the development of a restaurant chatbot that not only engages in natural-language conversations but also addresses context optimization and LLM optimization for restaurant context learning. The approach uses a Neo4j Knowledge graph built using the restaurant data as an external source of knowledge. The graph is traversed to match the user question with appropriate answer tokens using Term Frequency - Inverse Document Frequency (TF-IDF) embeddings. The relevant tokens along with user questions are used to provide additional context to the T5 language model to provide nuanced responses to the users. This improvement is quantitatively evidenced by a Bilingual Evaluation Understudy (BLEU) score of 0.60, indicating a high level of precision in language understanding and generation. An extensive evaluation of the chatbot includes assessing AI testability on the level of words, sentences, and information. These evaluations include simulated dialogue assessments and performance analyses, with a focus on the chatbot's ability to retrieve and integrate information. Based on the AI testability evaluation, the models consistently produce more knowledgeable, diverse, and relevant answers as compared with state-of-the-art models with an average information score in the range of 0.6-0.8.