earticle

영어

Recent advances of Generative AI (GenAI) tools have transformed information retrieval by offering conversational chatbot interaction and synthesized knowledge access. However, Generative AI systems rely on static, pre-trained data that are often outdated, making them prone to generate hallucinations – fabricated and inaccurate outputs. Retrieval Augmented Generation (RAG) technology is a promising architecture that enhance AI outputs by integrating external, accurate data. While RAG’s technical performance has been widely studied, there are limited studies on user interaction with RAG and its influence on user performance in real-world tasks. This research addresses this gap and assesses the effectiveness of RAG in user outcomes. Grounded in Task-Technology Fit (TTF) theory, we employ a scenario-based experiment design using 2x2 factorial design (AI System Type x Task Complexity). Participants complete tasks of different complexities using either standard LLMs or RAG systems. User performance is assessed through information quality metrics: accuracy, completeness and relevance. Findings are expected to contribute to evaluation of practical utility of RAG tools.