A User Security Continuous Authentication Method for Earable Devices

Wang Yong; Xiong Yi; Yang Tianyu; Shen Yiran

doi:10.7544/issn1000-1239.202440415

Journal of Computer Research and Development > 2024 > 61(11): 2821-2834. > DOI: 10.7544/issn1000-1239.202440415 CSTR: 32373.14.issn1000-1239.202440415

Wang Yong, Xiong Yi, Yang Tianyu, Shen Yiran. A User Security Continuous Authentication Method for Earable Devices[J]. Journal of Computer Research and Development, 2024, 61(11): 2821-2834. DOI: 10.7544/issn1000-1239.202440415

Citation:

PDF (1778 KB)

A User Security Continuous Authentication Method for Earable Devices

Wang Yong^{1, 2,},
Xiong Yi¹,
Yang Tianyu^1, ,,
Shen Yiran³

1.
College of Computer Science and Technology, Harbin Engineering University, Harbin 150001
2.
Modeling and Emulation in E-Government National Engineering Laboratory (Harbin Engineering University), Harbin 150001
3.
School of Software, Shandong University, Jinan 250100

Funds: This work was supported by the National Natural Science Foundation of China (61672179), the Youth Fund Project of Humanities and Social Sciences Research of the Ministry of Education of China (20YJCZH172), the China Postdoctoral Science Foundation (2019M651262), and the Funding from the Basic Research Support Program for Outstanding Young Teachers in Provincial Undergraduate Universities in Heilongjiang Province（YQJH2023302）.

More Information

Author Bio:
Wang Yong: born in 1983. PhD, associate professor. Senior member of CCF. His main research interests include artificial intelligence, privacy computing, and Internet of things

Xiong Yi: born in 2003. Bachelor. His main research interests include smart wearable devices, intelligent Internet of things, and machine learning

Yang Tianyu: born in 2002. Bachelor. His research interests include smart wearable devices, intelligent Internet of things, and machine learning

Shen Yiran: born in 1987. PhD, professor. Member of CCF， senior member of IEEE. His main research interests include mobile computing and virtual reality
Received Date: May 30, 2024
Revised Date: September 05, 2024
Available Online: September 12, 2024

Graphical Abstract

Abstract

Abstract

Earable devices are used as typical AIoT edge sensing devices. Protecting the privacy of legitimate users and preventing illegal use has become extremely important. In response to the current user authentication methods for earable devices, which are limited by input interfaces, sensor costs, and device power consumption, resulting in insufficient security, low universality, and poor user experience, a user authentication model based on the built-in inertial measurement unit (IMU) of earable devices is proposed. This model extracts user-specific information by collecting vibration signals generated by users performing facial interaction gestures, and achieves diversified implicit continuous user authentication based on intelligent analysis of the above information. To extract accurate and reliable user-specific information, a deep neural network feature encoder based on a Siamese network is proposed, which maps gesture samples of the same user closer in the feature space and enlarges the distance between gesture samples of different users, achieving effective encoding of user-specific information. For continuous user authentication based on user-specific information, a weighted voting strategy based on the distance of the one-class support vector machine hyperplane is proposed, which can adaptively optimize the discrimination boundary to better capture the contained features and structures. The confidence level of the sample is determined based on the distance of the sample points inside and outside the hyperplane, and a weighted voting is designed for authentication. Experimental results show that the method in this paper achieves an authentication accuracy of 97.33% in a single vote, and achieves an authentication accuracy of 99.993% after seven rounds of continuous authentication, which is better than all the methods compared in this paper. It provides a smoother user experience and a higher level of security without the need for passwords, and has a high practical application value.
- AIoT,
- earable devices,
- user continuous authentication,
- deep neural network feature encoder,
- weighted voting strategy

FullText(HTML)

References (36)

References

[1]	The Gates Notes LLC. AI is about to completely change how you use computers [EB/OL]. [2024-05-27]. https://www.gatesnotes.com/AI-agents
[2]	Apple Inc. Apple AirPods [EB/OL]. [2024-05-27]. https://www.apple.com. cn/airpods/
[3]	华为终端有限公司. 华为Freebuds官网 [EB/OL]. [2024-05-27]. https://consumer.huawei.com/cn/audio/ Huawei Terminal Co., Ltd. Huawei Freebuds official website [EB/OL]. [2024-05-27]. https://consumer.huawei.com/cn/audio/
[4]	2024索尼(中国)有限公司. LinkBuds官网 [EB/OL]. [2024-05-27]. https://www.sonystyle.com.cn/products/headphone/index.html 2024 Sony China Co., Ltd. LinkBuds official website [EB/OL]. [2024-05-27]. https://www.sonystyle.com.cn/products/headphone/index. html (in Chinese)
[5]	张玉清,周威,彭安妮. 物联网安全综述[J]. 计算机研究与发展,2017,54(10):2130−2143 doi: 10.7544/issn1000-1239.2017.20170470 Zhang Yuqing, Zhou Wei, Peng Anni. Survey of Internet of things security[J]. Journal of Computer Research and Development, 2017, 54(10): 2130−2143 (in Chinese) doi: 10.7544/issn1000-1239.2017.20170470
[6]	周俊,沈华杰,林中允,等. 边缘计算隐私保护研究进展[J]. 计算机研究与发展,2020,57(10):2027−2051 doi: 10.7544/issn1000-1239.2020.20200614 Zhou Jun, Shen Huajie, Lin Zhongyun, et al. Research advances on privacy preserving in edge computing[J]. Journal of Computer Research and Development, 2020, 57(10): 2027−2051 (in Chinese) doi: 10.7544/issn1000-1239.2020.20200614
[7]	董晓蕾. 物联网隐私保护研究进展[J]. 计算机研究与发展,2015,52(10):2341−2352 doi: 10.7544/issn1000-1239.2015.20150764 Dong Xiaolei. Advances of privacy preservation in Internet of things[J]. Journal of Computer Research and Development, 2015, 52(10): 2341−2352 (in Chinese) doi: 10.7544/issn1000-1239.2015.20150764
[8]	刘奇旭,靳泽,陈灿华,等. 物联网访问控制安全性综述[J]. 计算机研究与发展,2022,59(10):2190−2211 doi: 10.7544/issn1000-1239.20220510 Liu Qixu, Jin Ze, Chen Canhua, et al. Survey on Internet of things access control security[J]. Journal of Computer Research and Development, 2022, 59(10): 2190−2211 (in Chinese) doi: 10.7544/issn1000-1239.20220510
[9]	Bromley J, Guyon I, LeCun Y, et al. Signature verification using a "Siamese" time delay neural network [C] // Proc of the 6th Int Conf on Neural Information Processing Systems (NIPS’93). San Francisco, CA: Morgan Kaufmann Publishers Inc., 1993: 737–744
[10]	王会勇,唐士杰,丁勇,等. 生物特征识别模板保护综述[J]. 计算机研究与发展,2020,57(5):1003−1021 doi: 10.7544/issn1000-1239.2020.20190371 Wang Huiyong, Tang Shijie, Ding Yong, et al. Survey on biometrics template protection[J]. Journal of Computer Research and Development, 2020, 57(5): 1003−1021 (in Chinese) doi: 10.7544/issn1000-1239.2020.20190371
[11]	Tax D M, Duin R P. Support vector data description[J]. Machine Learning, 2004, 54: 45−66 doi: 10.1023/B:MACH.0000008084.60811.49
[12]	Lissermann R, Huber J, Hadjakos A, et al. EarPut: Augmenting ear-worn devices for ear-based interaction [C] // Proc of the 26th Australian Computer-Human Interaction Conf on Designing Futures: The Future of Design. New York: Association for Computing Machinery, 2014: 300−307
[13]	Xu Xuhai, Shi Haitian, Yi Xin, et al. EarBuddy: Enabling on-face interaction via wireless earbuds [C] // Proc of the 2020 CHI Conf on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2020: 1−14
[14]	Wang Yong, Yang Tianyu, Wang Chunxiao, et al. BudsAuth: Toward gesture-wise continuous user authentication through earbuds vibration sensing[J]. IEEE Internet of Things Journal, 2024, 11(12): 22007−22020 doi: 10.1109/JIOT.2024.3380811
[15]	Qamar N, Siddiqui N, Ehatisham-ul-Haq M, et al. An approach towards position-independent human activity recognition model based on wearable accelerometer sensor[J]. Procedia Computer Science, 2020, 177: 196−203 doi: 10.1016/j.procs.2020.10.028
[16]	Lu Chris Xiaoxuan, Du Bowen, Zhao Peijun, et al. Deepauth: In-situ authentication for smartwatches via deeply learned behavioural biometrics [C] // Proc of the 2018 ACM Int Symp on Wearable Computers. New York: Association for Computing Machinery, 2018: 204−207
[17]	Li Feng, Zhao Jiayi, Yang Huan, et al. VibHead: An authentication scheme for smart headsets through vibration[J]. ACM Transactions on Sensor Networks, 2024, 20(4): 1−12
[18]	Liu Jianwei, Song Wenfan, Shen Leming, et al. Secure user verification and continuous authentication via earphone IMU[J]. IEEE Transactions on Mobile Computing, 2023, 22(11): 6755−6769
[19]	Ma D, Ferlini A, Mascolo C. OESense: Employing occlusion effect for in-ear human sensing [C] //Proc of the 19th Annual Int Conf on Mobile Systems, Applications, and Services. New York: Association for Computing Machinery, 2021: 175−187
[20]	Wang Zi, Ren Yili, Chen Yingying, et al. ToothSonic: Earable authentication via acoustic toothprint [C] // Proc of ACM Int Conf on Interactive Mobile, Wearable and Ubiquitous Technologies. New York: Association for Computing Machinery, 2022: 1−24
[21]	Wang Zi, Wang Yilin, Yang Jie. EarSlide: A secure ear wearables biometric authentication based on acoustic fingerprint [C] // Proc of ACM Int Conf on Interactive Mobile, Wearable and Ubiquitous Technologies. New York: Association for Computing Machinery, 2024: 1−29
[22]	Hu Changshuo, Ma Xiao, Ma Dong, et al. Lightweight and non-invasive user authentication on earables [C] // Proc of the 24th Int Workshop on Mobile Computing Systems and Applications. Newport Beach, CA: Association for Computing Machinery, 2023: 36−41
[23]	Li Jiao, Liu Yang, Li Zhenjiang, et al. EarPass: Continuous user authentication with in-ear PPG [C] // Proc of the 2023 ACM Int Joint Conf on Pervasive and Ubiquitous Computing & the 2023 ACM Int Symp on Wearable Computing. Cancun, Quintana Roo, Mexico: Association for Computing Machinery, 2023: 327−332
[24]	Choi S, Yim Junghwan, Jin Yincheng, et al. EarPPG: Securing your identity with your ears [C] // Proc of the 28th Int Conf on Intelligent User Interfaces. Sydney, NSW: Association for Computing Machinery, 2023: 835−849
[25]	王气洪,贾洪杰,黄龙霞,等. 联合数据增强的语义对比聚类[J]. 计算机研究与发展,2024,61(6):1511−1524 doi: 10.7544/issn1000-1239.202220995 Wang Qihong, Jia Hongjie, Huang Longxia, et al. Semantic contrastive clustering with federated data augmentation[J]. Journal of Computer Research and Development, 2024, 61(6): 1511−1524 (in Chinese) doi: 10.7544/issn1000-1239.202220995
[26]	黄学坚,马廷淮,王根生. 基于样本内外协同表示和自适应融合的多模态学习方法[J]. 计算机研究与发展,2024,61(5):1310−1324 doi: 10.7544/issn1000-1239.202330722 Huang Xuejian, Ma Tinghuai, Wang Gensheng. Multimodal learning method based on intra- and inter-sample cooperative representation and adaptive fusion[J]. Journal of Computer Research and Development, 2024, 61(5): 1310−1324 (in Chinese) doi: 10.7544/issn1000-1239.202330722
[27]	杜金明,孙媛媛,林鸿飞,等. 融入知识图谱和课程学习的对话情绪识别[J]. 计算机研究与发展,2024,61(5):1299−1309 doi: 10.7544/issn1000-1239.202220951 Du Jinming, Sun Yuanyuan, Lin Hongfei, et al. Conversational emotion recognition incorporating knowledge graph and curriculum learning[J]. Journal of Computer Research and Development, 2024, 61(5): 1299−1309 (in Chinese) doi: 10.7544/issn1000-1239.202220951
[28]	包涵,王意洁. 低跨云数据中心修复流量的纠删码的快速构造方法[J]. 计算机研究与发展,2023,60(10):2418−2439 doi: 10.7544/issn1000-1239.202220580 Bao Han, Wang Yijie. A fast construction method of the erasure code with small cross-cloud data center repair traffic[J]. Journal of Computer Research and Development, 2023, 60(10): 2418−2439 (in Chinese) doi: 10.7544/issn1000-1239.202220580
[29]	Ganaie M A, Hu M, Malik A K, et al. Ensemble deep learning: A review[J]. Engineering Applications of Artificial Intelligence, 2022, 115: 105−151
[30]	Zhao Langcheng, Lyu Rui, Lin Qi, et al. mmArrhythmia: Contactless arrhythmia detection via mmWave sensing [C] // Proc of the 24th ACM Int Conf on Mobile Computing and Networking. New York: Association for Computing Machinery, 2024: 1−25
[31]	Wang Yuexin, Zheng Jie, Wang Danni, et al. Multi-objective planning model based on the soft voting ensemble learning algorithm [C] // Proc of the 2023 IEEE Int Conf on Electrical, Automation and Computer Engineering (ICEACE). Piscataway: IEEE, 2023: 1104−1107
[32]	郭虎升,张洋,王文剑. 面向不同类型概念漂移的两阶段自适应集成学习方法[J]. 计算机研究与发展,2024,61(7):1799−1811 doi: 10.7544/issn1000-1239.202330452 Guo Husheng, Zhang Yang, Wang Wenjian. Two-stage adaptive ensemble learning method for different types of concept drift[J]. Journal of Computer Research and Development, 2024, 61(7): 1799−1811 (in Chinese) doi: 10.7544/issn1000-1239.202330452
[33]	Xie Yadong, Li Fan, Wu Yue, et al. User authentication on farable devices via bone-conducted occlusion sounds[J]. IEEE Transactions on Dependable and Secure Computing, 2024, 21(4): 3704−3718 doi: 10.1109/TDSC.2023.3335368
[34]	Lee S, Choi W, Lee D H. The vibration knows who you are! A further analysis on usable authentication for smartwatch users[J]. Computers & Security, 2023, 125(C): 103−040
[35]	Wang Zi, Yang Jie. Ear wearable (earable) user authentication via acoustic toothprint [J/OL]. 2022 [2024-05-27]. https://api.semanticscholar.org/CorpusID:248218666
[36]	Xu Xiangyu, Yu Jiadi, Chen Yingying, et al. TouchPass: Towards behavior-irrelevant on-touch user authentication on smartphones leveraging vibrations [C] //Proc of the 26th Annual Int Conf on Mobile Computing and Networking (MobiCom’20). New York: Association for Computing Machinery, 2020: 1−13

[1]	Zhou Junzuo, Yi Jiangyan, Tao Jianhua, Ren Yong, Wang Tao. Mel Spectrogram and Squeeze-Excitation-Weighted Quantization for Neural Speech Codec[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440329
[2]	Wu Jinjin, Liu Quan, Chen Song, Yan Yan. Averaged Weighted Double Deep Q-Network[J]. Journal of Computer Research and Development, 2020, 57(3): 576-589. DOI: 10.7544/issn1000-1239.2020.20190159
[3]	Fan Zhengguang, Qu Dan, Yan Honggang, Zhang Wenlin. Joint Acoustic Modeling of Multi-Features Based on Deep Neural Networks[J]. Journal of Computer Research and Development, 2017, 54(5): 1036-1044. DOI: 10.7544/issn1000-1239.2017.20160031
[4]	Wang Liang, Wang Weiping, Meng Dan. Privacy Preserving Data Publishing via Weighted Bayesian Networks[J]. Journal of Computer Research and Development, 2016, 53(10): 2343-2353. DOI: 10.7544/issn1000-1239.2016.20160465
[5]	Ding Zhaoyun, Zhou Bin, Jia Yan, Wang Xiang. Detecting Spammers with a Bidirectional Vote Algorithm Based on Statistical Features in Microblogs[J]. Journal of Computer Research and Development, 2013, 50(11): 2336-2348.
[6]	Zhang Xiang, Deng Zhaohong, Wang Shitong, Choi Kupsze. Maximum Entropy Relief Feature Weighting[J]. Journal of Computer Research and Development, 2011, 48(6): 1038-1048.
[7]	Shen Derong, Ma Ye, Nie Tiezheng, Kou Yue, and Yu Ge. A Query Relaxation Strategy Applied in a Deep Web Data Integration System[J]. Journal of Computer Research and Development, 2010, 47(1): 88-95.
[8]	Liu He, Liu Dayou, Pei Zhili, Gao Ying. A Feature Weighting Scheme for Text Categorization Based on Feature Importance[J]. Journal of Computer Research and Development, 2009, 46(10): 1693-1703.
[9]	Fu Yunqing, Wang Songjian, and Wu Zhongfu. A Routing Protocol of Wireless Mesh Network Based on Weighted Link State[J]. Journal of Computer Research and Development, 2009, 46(1): 137-143.
[10]	Huang Dongping, Liu Duo, and Dai Yiqi. Weighted Threshold Secret Sharing[J]. Journal of Computer Research and Development, 2007, 44(8): 1378-1382.