earticle

영어

This study proposes an interdisciplinary framework that integrates Visual Grammar with Artificial Intelligence (AI) to quantify multimodal semantics in sign language. Focusing on the lexical item “DANGER” in Korean Sign Language (KSL) and American Sign Language (ASL), we examine how representational, interactive, and compositional meanings are manifested in 3D kinematic patterns extracted from video. The analysis provides empirical evidence of contrasting representational strategies: KSL shows an internalized Z-axis “Impact” vector toward the chest, while ASL exhibits an externalized X-axis “Friction” vector associated with contact-based motion. In addition, AI-based quantification of non-manual markers reveals a consistent face–manual timing relationship, with facial tension preceding the manual sign by 185 ms (KSL) and 245 ms (ASL), supporting the theoretical view of facial expressions as grammatical anchors in multimodal communication. By moving beyond qualitative description toward measurement-based validation, this work positions AI as a methodological instrument for Digital Sign Humanities and demonstrates how micro-temporal and micro-spatial patterns—often difficult to detect through manual observation—can be systematically linked to linguistic interpretation. Future work will extend this framework to larger datasets and additional sign languages toward a global digital atlas of sign-language kinematics.