Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Toward distributed, global, deep learning using IoT devices

Sudharsan, Bharath, Patel, Pankesh, Breslin, John, Ali, Muhammad Intizar, Mitra, Karan, Dustdar, Schahram, Rana, Omer ORCID: https://orcid.org/0000-0003-3597-2646, Jayaraman, Prem Prakash and Ranjan, Rajiv 2021. Toward distributed, global, deep learning using IoT devices. IEEE Internet Computing 25 (3) , pp. 6-12. 10.1109/MIC.2021.3053711

[thumbnail of Toward_Distributed_Global_Deep_Learning_Using_IoT_Devices.pdf] PDF - Published Version
Available under License Creative Commons Attribution.

Download (438kB)

Abstract

Deep learning (DL) using large scale, high-quality IoT datasets can be computationally expensive. Utilizing such datasets to produce a problem-solving model within a reasonable time frame requires a scalable distributed training platform/system. We present a novel approach where to train one DL model on the hardware of thousands of mid-sized IoT devices across the world, rather than the use of GPU cluster available within a data center. We analyze the scalability and model convergence of the subsequently generated model, identify three bottlenecks that are: high computational operations, time consuming dataset loading I/O, and the slow exchange of model gradients. To highlight research challenges for globally distributed DL training and classification, we consider a case study from the video data processing domain. A need for a two-step deep compression method, which increases the training speed and scalability of DL training processing, is also outlined. Our initial experimental validation shows that the proposed method is able to improve the tolerance of the distributed training process to varying internet bandwidth, latency, and Quality of Service metrics.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Additional Information: This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0
Publisher: Institute of Electrical and Electronics Engineers
ISSN: 1089-7801
Date of First Compliant Deposit: 22 July 2021
Last Modified: 23 May 2023 19:57
URI: https://orca.cardiff.ac.uk/id/eprint/142732

Citation Data

Cited 7 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics