Label Differential Privacy Study for Privacy Protection in Multimodal Contrastive Learning Model 


Vol. 14,  No. 5, pp. 289-296, May  2025
https://doi.org/10.3745/TKIPS.2025.14.5.289


PDF Full-Text
  Abstract

Recent advancements in multimodal deep learning have garnered significant attention from both academia and industry due to their exceptional accuracy and ability to learn rich knowledge representations. In particular, contrastive learning based approaches have played a pivotal role in dramatically enhancing the performance of multimodal deep learning. However, the use of multiple data sources in multimodal deep learning increases the risk of inferring sensitive information through data fusion, posing a higher privacy invasion attack compared to unimodal deep learning. This challenge cannot be fully addressed by privacy preserving techniques traditionally employed in unimodal deep learning, underscoring the growing importance of privacy protection in this domain. To address this issue, previous studies have relied on trusted execution environments or strengthened security by selectively recording data classified as privacy threatening. However, these approaches face limitations such as hardware dependency, performance degradation, and accuracy issues in data classification. These shortcomings hinder scalability and usability while leaving systems vulnerable to emerging threats. In this study, we address the privacy concerns by applying the Double Randomized Response algorithm, which ensures label differential privacy during the data preparation process. As a result, we achieved 80.14% accuracy in image-table matching and classification tasks, demonstrating a balance between privacy protection and performance. This method is the first to incorporate data security considerations into multimodal deep learning models while substantiating its efficacy, marking a significant contribution to the field.

  Statistics
  Cite this article

[IEEE Style]

Y. Kim, M. Yu, Y. Lee, H. Bae, "Label Differential Privacy Study for Privacy Protection in Multimodal Contrastive Learning Model," The Transactions of the Korea Information Processing Society, vol. 14, no. 5, pp. 289-296, 2025. DOI: https://doi.org/10.3745/TKIPS.2025.14.5.289.

[ACM Style]

Youngseo Kim, Minseo Yu, Younghan Lee, and Ho Bae. 2025. Label Differential Privacy Study for Privacy Protection in Multimodal Contrastive Learning Model. The Transactions of the Korea Information Processing Society, 14, 5, (2025), 289-296. DOI: https://doi.org/10.3745/TKIPS.2025.14.5.289.

Forms

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)

Old Journals

Indexing

All publications of TKIPS is indexed in DOI, EBSCO, Google Scholar, Crossref, and CrossCheck.

TKIPS is also selected as the Journal for Accreditation by NRF (National Research Foundation of Korea).

Related Journals