• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Pan Xudong, Zhang Mi, Yan Yifan, Lu Yifan, Yang Min. Evaluating Privacy Risks of Deep Learning Based General-Purpose Language Models[J]. Journal of Computer Research and Development, 2021, 58(5): 1092-1105. DOI: 10.7544/issn1000-1239.2021.20200908
Citation: Pan Xudong, Zhang Mi, Yan Yifan, Lu Yifan, Yang Min. Evaluating Privacy Risks of Deep Learning Based General-Purpose Language Models[J]. Journal of Computer Research and Development, 2021, 58(5): 1092-1105. DOI: 10.7544/issn1000-1239.2021.20200908

Evaluating Privacy Risks of Deep Learning Based General-Purpose Language Models

Funds: This work was supported by the National Natural Science Foundation of China (61972099, U1636204, U1836213, U1836210, U1736208) and the Natural Science Foundation of Shanghai (19ZR1404800).
More Information
  • Published Date: April 30, 2021
  • Recently, a variety of Transformer-based GPLMs (general-purpose language models), including Google’s BERT (bidirectional encoder representation from transformers), are proposed in NLP (natural language processing). GPLMs help achieve state-of-the-art performance on a wide range of NLP tasks, and are applied in industrial applications. Despite their generality and promising performance, a recent research work first shows that an attacker, who has access to the textual embeddings produced by GPLMs, can infer whether the original text contains a specific keyword with high accuracy. However, the previous work has the following limitations. First, they only consider the occurrence of one sensitive word as the sensitive information to steal, which is still far from a threatening privacy violation. Besides, their attack requires several rather strict security assumptions on the attacker’s capability, e.g., the attacker knows which GPLM produces the victim’s textual embeddings. Moreover, they only consider the GPLMs designed for English texts. To address the aforementioned limitations and serve as a complement to their work, this paper proposes a more comprehensive privacy theft chain which is designed to explore whether there are even more privacy risks in general-purpose language models. Via experiments on 13 commercial GPLMs, we empirically show that an attacker can step by step infer the GPLM type behind the textual embedding with near 100% accuracy, then infer the textual length with over 70% on average and finally probe sensitive words that possibly occur in the original text, which brings useful information for the attacker to finally reconstruct the sensitive semantics. Besides, this paper also evaluates the privacy risks of three typical general-purpose language models in Chinese. The results confirm that privacy risks also exist in Chinese general-purpose language models, which calls for mitigation studies in the future.
  • Related Articles

    [1]Xie Guo, Zhang Huaiwen, Wang Le, Liao Qing, Zhang Aoqian, Zhou Zhili, Ge Huilin, Wang Zhiheng, Wu Guozheng. Acceptance and Funding Status of Artificial Intelligence Discipline Projects Under the National Natural Science Foundation of China in 2024[J]. Journal of Computer Research and Development, 2025, 62(3): 648-661. DOI: 10.7544/issn1000-1239.202550008
    [2]Chen Xuanting, Ye Junjie, Zu Can, Xu Nuo, Gui Tao, Zhang Qi. Robustness of GPT Large Language Models on Natural Language Processing Tasks[J]. Journal of Computer Research and Development, 2024, 61(5): 1128-1142. DOI: 10.7544/issn1000-1239.202330801
    [3]Zhang Mi, Pan Xudong, Yang Min. JADE-DB:A Universal Testing Benchmark for Large Language Model Safety Based on Targeted Mutation[J]. Journal of Computer Research and Development, 2024, 61(5): 1113-1127. DOI: 10.7544/issn1000-1239.202330959
    [4]Shu Wentao, Li Ruixiao, Sun Tianxiang, Huang Xuanjing, Qiu Xipeng. Large Language Models: Principles, Implementation, and Progress[J]. Journal of Computer Research and Development, 2024, 61(2): 351-361. DOI: 10.7544/issn1000-1239.202330303
    [5]Yang Yi, Li Ying, Chen Kai. Vulnerability Detection Methods Based on Natural Language Processing[J]. Journal of Computer Research and Development, 2022, 59(12): 2649-2666. DOI: 10.7544/issn1000-1239.20210627
    [6]Pan Xudong, Zhang Mi, Yang Min. Fishing Leakage of Deep Learning Training Data via Neuron Activation Pattern Manipulation[J]. Journal of Computer Research and Development, 2022, 59(10): 2323-2337. DOI: 10.7544/issn1000-1239.20220498
    [7]Pan Xuan, Xu Sihan, Cai Xiangrui, Wen Yanlong, Yuan Xiaojie. Survey on Deep Learning Based Natural Language Interface to Database[J]. Journal of Computer Research and Development, 2021, 58(9): 1925-1950. DOI: 10.7544/issn1000-1239.2021.20200209
    [8]Zheng Haibin, Chen Jinyin, Zhang Yan, Zhang Xuhong, Ge Chunpeng, Liu Zhe, Ouyang Yike, Ji Shouling. Survey of Adversarial Attack, Defense and Robustness Analysis for Natural Language Processing[J]. Journal of Computer Research and Development, 2021, 58(8): 1727-1750. DOI: 10.7544/issn1000-1239.2021.20210304
    [9]Wang Ye, Chen Junwu, Xia Xin, Jiang Bo. Intelligent Requirements Elicitation and Modeling: A Literature Review[J]. Journal of Computer Research and Development, 2021, 58(4): 683-705. DOI: 10.7544/issn1000-1239.2021.20200740
    [10]Ke Yan, Zhang Minqing, Su Tingting. A Novel Multiple Bits Reversible Data Hiding in Encrypted Domain Based on R-LWE[J]. Journal of Computer Research and Development, 2016, 53(10): 2307-2322. DOI: 10.7544/issn1000-1239.2016.20160444
  • Cited by

    Periodical cited type(5)

    1. 周军芽,吴进伟,吴广飞,张何为. 基于Bi-LSTM神经网络的短文本敏感词识别方法. 武汉理工大学学报(信息与管理工程版). 2024(02): 312-316 .
    2. 石新满,胡广林,邵鑫,赵新爽,张思慧,乔晓. 基于人工智能大语言模型技术的电网优化运行应用分析. 自动化与仪器仪表. 2024(08): 180-184 .
    3. 李卓卓,蒋雨萌. 信息隐私量表对象、指标和应用的研究与展望. 情报理论与实践. 2024(10): 41-52 .
    4. 谭九生,李猛. 人机融合智能的伦理风险及其适应性治理. 昆明理工大学学报(社会科学版). 2022(03): 37-45 .
    5. 潘旭东,张谧,杨珉. 基于神经元激活模式控制的深度学习训练数据泄露诱导. 计算机研究与发展. 2022(10): 2323-2337 . 本站查看

    Other cited types(7)

Catalog

    Article views (1072) PDF downloads (619) Cited by(12)

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return