Evaluating Hallucination in LLM with Jeju Dialect Inputs 


Vol. 14,  No. 5, pp. 363-371, May  2025
https://doi.org/10.3745/TKIPS.2025.14.5.363


PDF
  Abstract

LLM have been actively utilized in various high-stakes applications. However, as most models are trained on standard language data, they exhibit structural limitations in processing low-resource linguistic inputs such as dialects. In particular, when faced with inputs in the Jeju dialect—which significantly differs from Standard Korean in grammar, vocabulary, and expression—LLMs often fail to accurately interpret user intent and generate hallucinated content without factual basis. This study empirically investigates this issue by constructing a dataset of queries in both Standard Korean and the Jeju dialect. A quantitative comparison was conducted on commercial LLM focusing on response reliability, hallucination frequency, sensitivity to linguistic components, and model-specific response divergence. Experimental results show that dialectal input significantly increases the error rate, with hallucination being more prominent in multiple-answer formats and queries with complex verb endings. This study provides concrete evidence of LLM's heightened sensitivity to dialectal inputs and offers foundational insights for developing dialect-aware language models and designing safer generative AI systems

  Statistics
  Cite this article

[IEEE Style]

S. Gwon, J. Lee, G. Kim, H. Suh, S. Lee, "Evaluating Hallucination in LLM with Jeju Dialect Inputs," The Transactions of the Korea Information Processing Society, vol. 14, no. 5, pp. 363-371, 2025. DOI: https://doi.org/10.3745/TKIPS.2025.14.5.363.

[ACM Style]

Seokjae Gwon, Jeongho Lee, Gyeonghoe Kim, Heeyeong Suh, and Seyoung Lee. 2025. Evaluating Hallucination in LLM with Jeju Dialect Inputs. The Transactions of the Korea Information Processing Society, 14, 5, (2025), 363-371. DOI: https://doi.org/10.3745/TKIPS.2025.14.5.363.

Forms

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)

Old Journals

Indexing

All publications of TKIPS is indexed in DOI, EBSCO, Google Scholar, Crossref, and CrossCheck.

TKIPS is also selected as the Journal for Accreditation by NRF (National Research Foundation of Korea).

Related Journals