DNN Inference Acceleration via Heterogeneous IoT Devices Collaboration
-
摘要: 深度神经网络(deep neural network, DNN)已经广泛应用于各种智能应用，如图像和视频识别.然而，由于DNN任务计算量大，资源受限的物联网(Internet of things, IoT)设备难以本地单独执行DNN推断任务.现有云协助方法容易受到通信延迟无法预测和远程服务器性能不稳定等因素的影响.一种非常有前景的方法是利用IoT设备协作实现分布式、可扩展DNN任务推断.然而，现有工作仅研究IoT设备同构情况下的静态拆分策略.因此，迫切需要研究如何在能力异构且资源受限的IoT设备间自适应地拆分DNN任务，协作执行任务推断.上述研究问题面临2个重要挑战:1)DNN任务多层推断延迟难以准确预测;2)难以在异构动态的多设备环境中实时智能调整协作推断策略.为此，首先提出细粒度可解释的多层延迟预测模型.进一步，利用进化增强学习(evolutionary reinforcement learning, ERL)自适应确定DNN推断任务的近似最优拆分策略.实验结果表明：该方法能够在异构动态环境中实现显著DNN推断加速.
-
关键词:
- 深度神经网络推断加速 /
- 异构设备协作 /
- 进化增强学习 /
- 多层预测模型 /
- 拆分策略
Abstract: Deep neural networks (DNNs) have been intensively deployed in a variety of intelligent applications (e.g., image and video recognition). Nevertheless, due to DNNs’ heavy computation burden, resource-constrained IoT devices are unsuitable to locally execute DNN inference tasks. Existing cloud-assisted approaches are severely affected by unpredictable communication latency and unstable performance of remote servers. As a countermeasure, it is a promising paradigm to leverage collaborative IoT devices to achieve distributed and scalable DNN inference. However, existing works only consider homogeneous IoT devices with static partition. Thus, there is an urgent need for a novel framework to adaptively partition DNN tasks and orchestrate distributed inference among heterogeneous resource-constrained IoT devices. There are two main challenges in this framework. First, it is difficult to accurately profile the DNNs’ multi-layer inference latency. Second, it is difficult to learn the collaborative inference strategy adaptively and in real-time in the heterogeneous environments. To this end, we first propose an interpretable multi-layer prediction model to abstract complex layer parameters. Furthermore, we leverage the evolutionary reinforcement learning (ERL) to adaptively determine the near-optimal partitioning strategy for DNN inference tasks. Real-world experiments based on Raspberry Pi are implemented, showing that our proposed method can significantly accelerate the inference speed in dynamic and heterogeneous environments. -
-
期刊类型引用(9)
1. 霍纬纲，侯振环. 基于多尺度卷积自注意力的多维时间序列预测. 计算机工程与设计. 2023(04): 1250-1258 . 
百度学术
2. 董红斌，韩爽，付强. 基于AR与DNN联合模型的地理传感器时间序列预测. 计算机科学. 2023(11): 41-48 . 百度学术
3. 许丹丹，徐洋，张思聪，付子爔. 基于DCNN-GRU模型的XSS攻击检测方法. 计算机应用与软件. 2022(02): 324-329 . 百度学术
4. 刘琳岚，肖庭忠，舒坚，牛明晓. 基于门控循环单元的链路质量预测. 工程科学与技术. 2022(06): 51-58 . 百度学术
5. 吴蕾，曾慧平，王海威. 网络非平稳流量多尺度时间序列预测数学建模. 计算机仿真. 2021(08): 356-359+434 . 百度学术
6. 罗佩，袁景凌，陈旻骋，盛德明. 面向教学资源的均值惩罚随机森林非平稳时序预测方法. 小型微型计算机系统. 2021(10): 2089-2094 . 百度学术
7. 张冬梅，李金平，李江，余想，宋凯旋. 基于门控权重单元的多变量时间序列预测. 湖南大学学报(自然科学版). 2021(10): 105-112 . 百度学术
8. 朱海浩，祝永新，汪辉. 基于深度置信网络的多变量时间序列分类方法. 计算机仿真. 2021(12): 262-266 . 百度学术
9. 杜圣东，李天瑞，杨燕，王浩，谢鹏，洪西进. 一种基于序列到序列时空注意力学习的交通流预测模型. 计算机研究与发展. 2020(08): 1715-1728 . 本站查看
其他类型引用(24)
计量
- 文章访问数: 1340
- HTML全文浏览量: 6
- PDF下载量: 591
- 被引次数: 33
下载:
