원문정보
초록
영어
This study explores the practical application of large language models (LLMs) in marketing, addressing the challenges posed by their stochastic and context-agnostic nature. By employing a human-in-the-loop approach with domain experts, the study aligns LLM-generated content with specific contexts and evaluates the impact of semantic inconsistency on user engagement. In collaboration with a major South Korean TV manufacturer, the researchers conducted a randomized field experiment with 39,588 smart TV devices, testing LLM-generated persuasive messages to recommend TV content. The results demonstrate that context-relevant LLM-generated messages significantly improve click-through rates. However, semantically inconsistent messages diminish this effect. These findings underscore the need to mitigate LLMs' stochastic nature through human oversight to ensure consistent and effective user engagement.
목차
Introduction
Literature Review
Stochastic and Context-agnostic Nature of LLMs
Contextual Targeting and Priming Effect
Hypotheses Development
Method
Research Context: Smart TV Content Recommendation
Message Preparation: Human-in-the-loop Process
Experimental Design
Analysis and Results
Results of the Empirical Model
Empirical Extensions
Conclusions
Contributions
Limitations
References