Citation: | Cai Huayang, Huang Xing, Liu Genggeng. Control Logic Routing for Continuous-Flow Microfluidic Biochips Based on Deep Reinforcement Learning[J]. Journal of Computer Research and Development, 2025, 62(4): 950-962. DOI: 10.7544/issn1000-1239.202440034 |
With the advancement of electronic design automation, continuous-flow microfluidic biochips have become one of the most promising platforms for biochemical experiments. This chip manipulates fluid samples in milliliters or nanoliters by utilizing internal microvalves and microchannels, and thus automatically performs basic biochemical experiments, such as mixing and detection. To achieve the correct bioassay function, the microvalves deployed inside the chip are usually managed by a multiplexer-based control logic, and valves receive control signals from a core input through the control channel for accurate switching. Since biochemical reactions typically require high sensitivity, the length of control paths connecting each valve needs to be reduced to ensure immediate signal propagation, and thus to reduce the signal propagation delay. In addition, to reduce the fabrication cost of chips, a vital issue to be addressed in the logic architecture design is how to effectively reduce the total channel length within the control logic. To address the above issues, we propose a deep reinforcement learning-based control logic routing algorithm to minimize the signal propagation delay and total control channel length, thereby automatically constructing an efficient control channel network. The algorithm employs the dueling deep Q-network architecture as the agent of the deep reinforcement learning framework to evaluate the tradeoff between signal propagation delay and total channel length. Besides, the diagonal channel routing is implemented for the first time for control logic, thus fundamentally improving the efficiency of valve switching operations and reducing the fabrication cost of the chip. The experimental results demonstrate that the proposed algorithm can effectively construct a high-performance and low-cost control logic architecture.
[1] |
Huang Xing, Ho T, Guo Wenzhong, et al. Computer-aided design techniques for flow-based microfluidic lab-on-a-chip systems[J]. ACM Computing Surveys, 2021, 54(5): 1−29
|
[2] |
Liu Genggeng, Huang Hongbin, Chen Zhisheng, et al. Design automation for continuous-flow microfluidic biochips: A comprehensive review[J]. Integration, 2022, 82: 48−66
|
[3] |
Hu Xu, Chen Zhen, Chen Zhisheng, et al. Architectural synthesis of continuous-flow microfluidic biochips with connection pair optimization[J]. Electronics, 2024, 13(2): 247 doi: 10.3390/electronics13020247
|
[4] |
Einav S, Gerber D, Bryson P D, et al. Discovery of a hepatitis C target and its pharmacological inhibitors by microfluidic affinity analysis[J]. Nature Biotechnology, 2008, 26(9): 1019−1027 doi: 10.1038/nbt.1490
|
[5] |
Hong J W, Chen Yan, Anderson W F, et al. Molecular biology on a microfluidic chip[J]. Journal of Physics: Condensed Matter, 2006, 18(18): 691−701 doi: 10.1088/0953-8984/18/18/S14
|
[6] |
Unger M A, Chou H P, Thorsen T, et al. Monolithic microfabricated valves and pumps by multilayer soft lithography[J]. Science, 2000, 288(5463): 113−116 doi: 10.1126/science.288.5463.113
|
[7] |
Thorsen T, Maerkl S J, Quake S R. Microfluidic large-scale integration[J]. Science, 2002, 298(5593): 580−584 doi: 10.1126/science.1076996
|
[8] |
Huang Xing, Ho T, Chakrabarty K, et al. Timing-driven flow-channel network construction for continuous-flow microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019, 39(6): 1314−1327
|
[9] |
Araci I E, Quake S R. Microfluidic very large scale integration (mVLSI) with integrated micromechanical valves[J]. Lab on a Chip, 2012, 12(16): 2803−2806 doi: 10.1039/c2lc40258k
|
[10] |
Fidalgo L M, Maerkl S J. A software-programmable microfluidic device for automated biology[J]. Lab on a Chip, 2011, 11(9): 1612−1619 doi: 10.1039/c0lc00537a
|
[11] |
Lim Y C, Kouzani A Z, Duan W. Lab-on-a-chip: A component view[J]. Microsystem Technologies, 2010, 16: 1995−2015 doi: 10.1007/s00542-010-1141-6
|
[12] |
Tseng T M, Li Mengchu, Freitas D N, et al. Columba 2.0: A co-layout synthesis tool for continuous-flow microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017, 37(8): 1588−1601
|
[13] |
Yang Kailin, Yao Hailong, Ho T, et al. AARF: Any-angle routing for flow-based microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018, 37(12): 3042−3055 doi: 10.1109/TCAD.2018.2789356
|
[14] |
Wang Qin, Zuo Shiliang, Yao Hailong, et al. Hamming-distance-based valve switching optimization for control-layer multiplexing in flow-based microfluidic biochips[C]//Proc of the 22nd Asia and South Pacific Design Automation Conf. Piscataway, NJ: IEEE, 2017: 524−529
|
[15] |
Wang Qin, Xu Yue, Zuo Shiliang, et al. Pressure-aware control layer optimization for flow-based microfluidic biochips[J]. IEEE Transactions on Biomedical Circuits and Systems, 2017, 11(6): 1488−1499 doi: 10.1109/TBCAS.2017.2766210
|
[16] |
Zhu Ying, Huang Xing, Li Bing, et al. Multicontrol: Advanced control-logic synthesis for flow-based microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019, 39(10): 2489−2502
|
[17] |
Huang Xing, Cai Huayang, Guo Wenzhong, et al. Control-logic synthesis of fully programmable valve array using reinforcement learning[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024, 43(1): 277−290 doi: 10.1109/TCAD.2023.3309740
|
[18] |
Liang Siyuan, Li Mengchu, Tseng T M, et al. CoMUX: Combinatorial-coding-based high-performance microfluidic control multiplexer design[C/OL]//Proc of the 41st ACM Int Conf on Computer-Aided Design. New York: ACM, 2022 [2024-06-19]. https://dl.acm.org/doi/abs/10.1145/3508352.3549353
|
[19] |
Chen Zhisheng, Huang Xing, Guo Wenzhong, et al. Physical synthesis of flow-based microfluidic biochips considering distributed channel storage[C]//Proc of Design, Automation & Test in Europe Conf & Exhibition. Piscataway, NJ: IEEE, 2019: 1525−1530
|
[20] |
Hu Kai, Dinh T A, Ho T, et al. Control-layer routing and control-pin minimization for flow-based microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2016, 36(1): 55−68
|
[21] |
曾俊杰,秦龙,徐浩添,等,基于内在动机的深度强化学习探索方法综述[J]. 计算机研究与发展,2023,60(10):2359−2382
Zeng Junjie, Qin Long, Xu Haotian, et al. Exploration approaches in deep reinforcement learning based on intrinsic motivation: A review[J]. Journal of Computer Research and Development, 2023, 60(10): 2359−2382 (in Chinese)
|
[22] |
Sutton R S, Barto A G. Reinforcement learning: An introduction[M]//Adaptive Computation and Machine Learning. Cambridge, MA: MIT, 2018
|
[23] |
唐楚哲,王肇国,陈海波. 机器学习方法赋能系统软件:挑战、实践与展望[J]. 计算机研究与发展,2023,60(5):964−973
Tang Chuzhe, Wang Zhaoguo, Chen Haibo. Empowering system software with machine learning methods: Challenges, practice, and prospects[J]. Journal of Computer Research and Development, 2023, 60(5): 964−973 (in Chinese)
|
[24] |
Kober J, Bagnell J A, Peters J. Reinforcement learning in robotics: A survey[J]. The International Journal of Robotics Research, 2013, 32(11): 1238−1274 doi: 10.1177/0278364913495721
|
[25] |
Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529−533 doi: 10.1038/nature14236
|
[26] |
Wang Ziyu, Schaul T, Hessel M, et al. Dueling network architectures for deep reinforcement learning[C]//Proc of the 33rd Int Conf on Machine Learning. Cambridge, MA: MIT, 2016: 1995−2003
|
[27] |
Liu Chunfeng, Huang Xing, Li Bing, et al. DCSA: Distributed channel-storage architecture for flow-based microfluidic biochips[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020, 40(1): 115−128
|
[28] |
Marcus J S, Anderson W F, Quake S R. Microfluidic single-cell mRNA isolation and analysis[J]. Analytical Chemistry, 2006, 78(9): 3084−3089 doi: 10.1021/ac0519460
|
[29] |
Van Hasselt H, Guez A, Silver D. Deep reinforcement learning with double q-learning[C]//Proc of the 30th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2016: 2094−2100
|
[30] |
Mnih V, Kavukcuoglu K, Silver D, et al. Playing Atari with deep reinforcement learning[J]. arXiv preprint, arXiv:1312.560, 2013
|
[1] | Li Song, Cao Wenqi, Hao Xiaohong, Zhang Liping, Hao Zhongxiao. Collective Spatial Keyword Query Based on Time-Distance Constrained and Cost Aware[J]. Journal of Computer Research and Development, 2025, 62(3): 808-819. DOI: 10.7544/issn1000-1239.202330815 |
[2] | Wang Kaifan, Xu Yinan, Yu Zihao, Tang Dan, Chen Guokai, Chen Xi, Gou Lingrui, Hu Xuan, Jin Yue, Li Qianruo, Li Xin, Lin Jiawei, Liu Tong, Liu Zhigang, Wang Huaqiang, Wang Huizhe, Zhang Chuanqi, Zhang Fawang, Zhang Linjuan, Zhang Zifei, Zhang Ziyue, Zhao Yangyang, Zhou Yaoyang, Zou Jiangrui, Cai Ye, Huan Dandan, Li Zusong, Zhao Jiye, He Wei, Sun Ninghui, Bao Yungang. XiangShan Open-Source High Performance RISC-V Processor Design and Implementation[J]. Journal of Computer Research and Development, 2023, 60(3): 476-493. DOI: 10.7544/issn1000-1239.202221036 |
[3] | Ren Hao, Liu Baisong, Sun Jinyang, Dong Qian, Qian Jiangbo. A Time and Relation-Aware Graph Collaborative Filtering for Cross-Domain Sequential Recommendation[J]. Journal of Computer Research and Development, 2023, 60(1): 112-124. DOI: 10.7544/issn1000-1239.202110545 |
[4] | Zhang Tong, Feng Jiaqi, Ma Yanying, Qu Siyuan, Ren Fengyuan. Survey on Traffic Scheduling in Time-Sensitive Networking[J]. Journal of Computer Research and Development, 2022, 59(4): 747-764. DOI: 10.7544/issn1000-1239.20210203 |
[5] | Cui Yuanning, Li Jing, Shen Li, Shen Yang, Qiao Lin, Bo Jue. Duration-HyTE: A Time-Aware Knowledge Representation Learning Method Based on Duration Modeling[J]. Journal of Computer Research and Development, 2020, 57(6): 1239-1251. DOI: 10.7544/issn1000-1239.2020.20190253 |
[6] | Zheng Xiao, Gao Han, Wang Xiujun, Qin Feng. Contact Duration Aware Cooperative Data Caching in Mobile Opportunistic Networks[J]. Journal of Computer Research and Development, 2018, 55(2): 338-345. DOI: 10.7544/issn1000-1239.2018.20160929 |
[7] | Wang Chong, Lü Yinrun, Chen Li, Wang Xiuli, Wang Yongji. Survey on Development of Solving Methods and State-of-the-Art Applications of Satisfiability Modulo Theories[J]. Journal of Computer Research and Development, 2017, 54(7): 1405-1425. DOI: 10.7544/issn1000-1239.2017.20160303 |
[8] | Chen Huangke, Zhu Jianghan, Zhu Xiaomin, Ma Manhao, Zhang Zhenshi. Resource-Delay-Aware Scheduling for Real-Time Tasks in Clouds[J]. Journal of Computer Research and Development, 2017, 54(2): 446-456. DOI: 10.7544/issn1000-1239.2017.20151123 |
[9] | Zhou Hang, Huang Zhiqiu, Zhu Yi, Xia Liang, Liu Linyuan. Real-Time Systems Contact Checking and Resolution Based on Time Petri Net[J]. Journal of Computer Research and Development, 2012, 49(2): 413-420. |
[10] | Zhou Hang, Huang Zhiqiu, Hu Jun, Zhu Yi. Real-Time System Resource Conflict Checking Based on Time Petri Nets[J]. Journal of Computer Research and Development, 2009, 46(9): 1578-1585. |