Multi-Task Attention Network for Contactless Sleep Monitoring

Li Siheng; Jin Beihong; Zhang Fusang; Wang Zhi; Ma Junqi; Su Chang; Ren Xiaoyong; Liu Haiqin

doi:10.7544/issn1000-1239.202440389

Journal of Computer Research and Development > 2024 > 61(11): 2739-2753. > DOI: 10.7544/issn1000-1239.202440389 CSTR: 32373.14.issn1000-1239.202440389

Li Siheng, Jin Beihong, Zhang Fusang, Wang Zhi, Ma Junqi, Su Chang, Ren Xiaoyong, Liu Haiqin. Multi-Task Attention Network for Contactless Sleep Monitoring[J]. Journal of Computer Research and Development, 2024, 61(11): 2739-2753. DOI: 10.7544/issn1000-1239.202440389

Citation:

PDF (3932 KB)

Multi-Task Attention Network for Contactless Sleep Monitoring

Li Siheng^{1, 2, 3,},
Jin Beihong^{1, 2, 3, ,},
Zhang Fusang^{1, 2, 3},
Wang Zhi^{1, 2, 3},
Ma Junqi^{1, 2, 3},
Su Chang^{1, 2, 3},
Ren Xiaoyong⁴,
Liu Haiqin⁴

1.
Technology Center of Software Engineering, Institute of Software, Chinese Academy of Sciences, Beijing 100190
2.
Key Laboratory of System Software(Chinese Academy of Sciences)and State Key Laboratory of Computer Science(Institute of Software, Chinese Academy of Sciences), Beijing 100190
3.
University of Chinese Academy of Sciences, Beijing 100049
4.
The Second Affiliated Hospital of Xi’an Jiaotong University, Xi’an 710000

Funds: This work was supported by the National Natural Science Foundation of China (62072450, 62172394) and the Basic Research Program of ISCAS (ISCAS-JCMS-202305).

More Information

Author Bio:
Li Siheng: born in 1997. PhD candidate. Student member of CCF. His main research interests include mobile and pervasive computing, data mining, wireless sensing, and mobile health

Jin Beihong: born in 1967. PhD, professor, PhD supervisor. Distinguished member of CCF. Her main research interests include pervasive computing and machine learning, particularly as applied to healthcare sensing and recommender systems

Zhang Fusang: born in 1987. PhD, associate professor. Senior member of CCF. His main research interests include mobile and pervasive computing, smart IoT, wireless sensing, and mobile health

Wang Zhi: born in 1998. PhD candidate. Student member of CCF. His main research interests include mobile and pervasive computing, data mining, wireless sensing, and mobile health

Ma Junqi: born in 1999. PhD candidate. Student member of CCF. His main research interests include mobile and pervasive computing, wireless sensing, and indoor localization

Su Chang: born in 1999. PhD candidate. Student member of CCF. His main research interests include mobile and pervasive computing, wireless sensing, and indoor localization

Ren Xiaoyong: born in 1970. PhD, professor, PhD supervisor, chief physician. His main research interests include research on the pathological mechanism and clinical comorbidity of sleep apnea, and clinical and basic research on head, neck tumors and pharyngeal voice diseases

Liu Haiqin: born in 1975. Bachelor, associate senior technician. Her main research interests include clinical assessment of sleep breathing disorders, sleep monitoring, and therapy of non-invasive positive pressure ventilation
Received Date: May 30, 2024
Revised Date: August 20, 2024
Available Online: August 29, 2024

Graphical Abstract

Abstract

Abstract

Sleep takes up nearly one-third of a person’s day and is closely related to human health. Since the durations and transitions over different sleep stages during sleep directly affect a person’s sleep quality, identifying sleep stages has become the most basic and important task in sleep monitoring. However, sleep disorders occurring in sleep can lead to complex sleep structures, thereby increasing the difficulty of classifying sleep stages. Most of the existing contactless solutions for sleep stage classification lack a sufficient understanding of the complexity in sleep structure, ignoring the relationship between sleep stage and sleep disorder. Therefore, these solutions fail to achieve great performance in patients with sleep disorders. In this paper, we propose a sleep monitoring system that focuses on predicting sleep stages from ultra-wideband (UWB) signals. We design a sequence prediction model that combines an attention-based sequence encoder and a contrastive learning module to extract the temporal progression of sleep and improve the generalizability of the encoder. Particularly, the key to our approach is a multi-task fine-tuning strategy that incorporates sleep disorder information into sleep staging to reduce the interference of sleep disorders with sleep stage prediction. We conduct extensive experiments on 110 subjects, including healthy individuals and patients with different severities of sleep disorders. The experimental results demonstrate that the performance of our model is superior to the baseline methods.
- contactless sensing,
- ultra-wideband (UWB),
- deep neural networks,
- multi-task learning,
- sleep stage classification,
- sleep disorder

FullText(HTML)

References (38)

References

[1]	Faraut B, Boudjeltia K Z, Vanhamme L, et al. Immune, inflammatory and cardiovascular consequences of sleep restriction and recovery[J]. Sleep Medicine Reviews, 2012, 16(2): 137−149
[2]	Chandrasekaran B, Fernandes S, Davis F. Science of sleep and sports performance–a scoping review[J]. Science & Sports, 2020, 35(1): 3−11
[3]	Guo Xiaofan, Zheng Liqiang, Wang Jun, et al. Epidemiological evidence for the link between sleep duration and high blood pressure: A systematic review and meta-analysis[J]. Sleep Medicine, 2013, 14(4): 324−332
[4]	Khot S P, Morgenstern L B. Sleep and stroke[J]. Stroke, 2019, 50(6): 1612−1617
[5]	谭思静,齐征. 如何解救被低质量睡眠“困”住的年轻人[N/OL]. 中国青年报. 2024-03-22(005). [2024-05-27]. https://zqb.cyol.com/html/2024-03/22/nw.D110000zgqnb_20240322_2-05.htm Tan Sijing, Qi Zheng. How to rescue young people trapped by poor quality sleep[N/OL]. China Youth Daily. 2024-03-22(005). [2024-05-27]. https://zqb.cyol.com/html/2024-03/22/nw.D110000zgqnb_20240322_2-05.htm (in Chinese)
[6]	News in Health. Good sleep for good health[EB/OL]. [2024-05-27]. https://newsinhealth.nih.gov/2021/04/good-sleep-good-health
[7]	Schulz H. Rethinking sleep analysis: Comment on the AASM manual for the scoring of sleep and associated events[J]. Journal of Clinical Sleep Medicine, 2008, 4(2): 99−103
[8]	Zhai Bing, Guan Yu, Catt M, et al. Ubi-SleepNet: Advanced multimodal fusion techniques for three-stage sleep classification using ubiquitous sensing[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2021, 5(4): 1−33
[9]	Huang Guanjie, Yuan Ye, Cao Guohong, et al. Accsleepnet: An axis-aware hybrid deep fusion model for sleep stage classification using wrist-worn accelerometer data[C]//Proc of 2022 IEEE Int Conf on Bioinformatics and Biomedicine. Piscataway, NJ: IEEE, 2022: 1005−1012
[10]	Yin Wenpeng, Kann K, Yu Mo, et al. Comparative study of CNN and RNN for natural language processing[J]. arXiv preprint, arXiv: 1702.01923, 2017
[11]	Bengio Y, Frasconi P, Simard P. The problem of learning long-term dependencies in recurrent networks[C]//Proc of IEEE Int Conf on Neural Networks. Piscataway, NJ: IEEE, 1993: 1183−1188
[12]	Bianchi M T, Cash S S, Mietus J, et al. Obstructive sleep apnea alters sleep stage transition dynamics[J]. PLoS One, 2010, 5(6): e11356
[13]	Zhao Mingmin, Yue Shichao, Katabi D, et al. Learning sleep stages from radio signals: A conditional adversarial architecture[C]//Proc of the 34th Int Conf on Machine Learning. Cambridge, MA: MIT, 2017: 4100−4109
[14]	Zhai Qian, Tang Tingyu, Lu Xiaoling, et al. Machine learning-enabled noncontact sleep structure prediction[J]. Advanced Intelligent Systems, 2022, 4(5): 2100227
[15]	Korkalainen H, Aakko J, Nikkonen S, et al. Accurate deep learning-based sleep staging in a clinical population with suspected obstructive sleep apnea[J]. IEEE Journal of Biomedical and Health Informatics, 2019, 24(7): 2073−2081
[16]	Zhang Feng, Wu Chenshu, Wang Beibei, et al. SMARS: Sleep monitoring via ambient radio signals[J]. IEEE Transactions on Mobile Computing, 2019, 20(1): 217−231
[17]	Rahman T, Adams A T, Ravichandran R V, et al. Dopplesleep: A contactless unobtrusive sleep sensing system using short-range doppler radar[C]//Proc of the 2015 ACM Int Joint Conf on Pervasive and Ubiquitous Computing. New York: ACM, 2015: 39−50
[18]	Jia Ziyu, Lin Youfang, Wang Jing, et al. GraphSleepNet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification[C]//Proc of the 29th Int Joint Conf on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, 2021: 1324−1330
[19]	Biswal S, Sun H, Goparaju B, et al. Expert-level sleep scoring with deep neural networks[J]. Journal of the American Medical Informatics Association, 2018, 25(12): 1643−1650
[20]	Jia Haohui, Yang Ziwei, Gao Pei, et al. Automatic sleep staging via frequency-wise spiking neural networks[C]//Proc of 2022 IEEE Int Conf on Bioinformatics and Biomedicine. Piscataway, NJ: IEEE, 2022: 1028−1033
[21]	Chen Jingrui, Li Yujie, Xiao Jing, et al. MSF-sleepnet: Multi-stream fusion network with contrastive learning for sleep stage classification[C]// Proc of 2022 IEEE Int Conf on Bioinformatics and Biomedicine. Piscataway, NJ: IEEE, 2022: 1493−1496
[22]	Advanced Brain Monitoring. Sleep profiler: In-home, patient applied objective assessment of sleep architecture & continuity[EB/OL]. [2024-05-27]. https://www.advancedbrainmonitoring.com/products/sleep-profiler
[23]	Chang Liqiong, Lu Jiaqi, Wang Ju, et al. SleepGuard: Capturing rich sleep information using smartwatch sensing data[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2018, 2(3): 1−34
[24]	Han Feiyu, Yang Panlong, Feng Yuanhao, et al. EarSleep: In-ear acoustic-based physical and physiological activity recognition for sleep stage detection[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2024, 8(2): 1−31
[25]	Zhai Bing, Perez-Pozuelo I, Clifton E A D, et al. Making sense of sleep: Multimodal sleep stage classification in a large, diverse population using movement and cardiac sensing[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2020, 4(2): 1−33
[26]	De Zambotti M, Goldstone A, Claudatos S, et al. A validation study of Fitbit Charge 2^TM compared with polysomnography in adults[J]. Chronobiology International, 2018, 35(4): 465−476
[27]	Choe E K, Consolvo S, Watson N F, et al. Opportunities for computing technologies to support healthy sleep behaviors[C]//Proc of the SIGCHI Conf on Human Factors in Computing Systems. New York: ACM, 2011: 3053−3062
[28]	Gu Weixi, Shangguan Longfei, Yang Zheng, et al. Sleep hunter: Towards fine grained sleep stage tracking with smartphones[J]. IEEE Transactions on Mobile Computing, 2015, 15(6): 1514−1527
[29]	Yu Bohan, Wang Yuxiang, Niu Kai, et al. WiFi-sleep: Sleep stage monitoring using commodity Wi-Fi devices[J]. IEEE Internet of Things Journal, 2021, 8(18): 13900−13913
[30]	Yim D H, Cho S H. An equidistance multi-human detection algorithm based on noise level using mono-static IR-UWB radar system[C]//Proc of the 2014 Int Conf on Future Communication, Information and Computer Science. Boca Raton, FL: CRC Press, 2014: 131−134
[31]	Mushtaq R. Augmented dickey fuller test[J/OL]. 2011 [2024-05-27]. http://dx. doi.org/10.2139/ssrn.1911068
[32]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Cambridge, MA: MIT, 2017: 5998−6008
[33]	Kang Wangcheng, McAuley J. Self-attentive sequential recommendation[C]//Proc of 2018 IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2018: 197−206
[34]	Wang Feng, Liu Huaping. Understanding the behaviour of contrastive loss[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 2495−2504
[35]	Wang Tongzhou, Isola P. Understanding contrastive representation learning through alignment and uniformity on the hypersphere[C]//Proc of the 37th Int Conf on Machine Learning. Cambridge, MA: MIT, 2020: 9929−9939
[36]	Sateia M J. International classification of sleep disorders[J]. Chest, 2014, 146(5): 1387−1394
[37]	Tang Hongyan, Liu Junning, Zhao Ming, et al. Progressive layered extraction (PLE): A novel multi-task learning (MTL) model for personalized recommendations[C]//Proc of the 14th ACM Conf on Recommender Systems. New York: ACM, 2020: 269−278
[38]	马新宇,范意兴,郭嘉丰,等. 关于短文本匹配的泛化性和迁移性的研究分析[J]. 计算机研究与发展,2022,59(1):118−126 doi: 10.7544/issn1000-1239.20200626 Ma Xinyu, Fan Yixing, Guo Jiafeng, et al. An empirical investigation of generalization and transfer in short text matching[J]. Journal of Computer Research and Development, 2022, 59(1): 118−126 (in Chinese) doi: 10.7544/issn1000-1239.20200626

[1]	Zhou Yuanding, Gao Guopeng, Fang Yaodong, Qin Chuan. Perceptual Authentication Hashing with Image Feature Fusion Based on Window Self-Attention[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202330669
[2]	Gao Wei, Chen Liqun, Tang Chunming, Zhang Guoyan, Li Fei. One-Time Chameleon Hash Function and Its Application in Redactable Blockchain[J]. Journal of Computer Research and Development, 2021, 58(10): 2310-2318. DOI: 10.7544/issn1000-1239.2021.20210653
[3]	Wu Linyang, Luo Rong, Guo Xueting, Guo Qi. Partitioning Acceleration Between CPU and DRAM: A Case Study on Accelerating Hash Joins in the Big Data Era[J]. Journal of Computer Research and Development, 2018, 55(2): 289-304. DOI: 10.7544/issn1000-1239.2018.20170842
[4]	Jiang Jie, Yang Tong, Zhang Mengyu, Dai Yafei, Huang Liang, Zheng Lianqing. DCuckoo: An Efficient Hash Table with On-Chip Summary[J]. Journal of Computer Research and Development, 2017, 54(11): 2508-2515. DOI: 10.7544/issn1000-1239.2017.20160795
[5]	Wang Wendi, Tang Wen, Duan Bo, Zhang Chunming, Zhang Peiheng, Sun Ninghui. Parallel Accelerator Design for High-Throughput DNA Sequence Alignment with Hash-Index[J]. Journal of Computer Research and Development, 2013, 50(11): 2463-2471.
[6]	Yuan Xinpan, Long Jun, Zhang Zuping, Luo Yueyi, Zhang Hao, and Gui Weihua. Connected Bit Minwise Hashing[J]. Journal of Computer Research and Development, 2013, 50(4): 883-890.
[7]	Qin Chuan, Chang Chin Chen, Guo Cheng. Perceptual Robust Image Hashing Scheme Based on Secret Sharing[J]. Journal of Computer Research and Development, 2012, 49(8): 1690-1698.
[8]	Ding Zhenhua, Li Jintao, Feng Bo. Research on Hash-Based RFID Security Authentication Protocol[J]. Journal of Computer Research and Development, 2009, 46(4): 583-592.
[9]	Li Zhiqiang, Chen Hanwu, Xu Baowen, Liu Wenjie. Fast Algorithms for Synthesis of Quantum Reversible Logic Circuits Based on Hash Table[J]. Journal of Computer Research and Development, 2008, 45(12): 2162-2171.
[10]	Liu Ji. One-Way Hash Function based on Integer Coupled Tent Maps and Its Performance Analysis[J]. Journal of Computer Research and Development, 2008, 45(3): 563-569.