Multiview Summarization and Activity Recognition Meet Edge Computing in IoT Environments

Simple item page
dc.contributor.authorHussain, Tanveer
dc.contributor.authorMuhammad, Khan
dc.contributor.authorUllah, Amin
dc.contributor.authorSer, Javier Del
dc.contributor.authorGandomi, Amir H.
dc.contributor.authorSajjad, Muhammad
dc.contributor.authorBaik, Sung Wook
dc.contributor.authorDe Albuquerque, Victor Hugo C.
dc.contributor.institutionIA
dc.date.accessioned2024-09-10T14:45:02Z
dc.date.available2024-09-10T14:45:02Z
dc.date.issued2021-06-15
dc.descriptionPublisher Copyright: © 2014 IEEE.
dc.description.abstractMultiview video summarization (MVS) has not received much attention from the research community due to inter-view correlations and views' overlapping, etc. The majority of previous MVS works are offline, relying on only summary, and require additional communication bandwidth and transmission time, with no focus on foggy environments. We propose an edge intelligence-based MVS and activity recognition framework that combines artificial intelligence with Internet of Things (IoT) devices. In our framework, resource-constrained devices with cameras use a lightweight CNN-based object detection model to segment multiview videos into shots, followed by mutual information computation that helps in a summary generation. Our system does not rely solely on a summary, but encodes and transmits it to a master device using a neural computing stick for inter-view correlations computation and efficient activity recognition, an approach which saves computation resources, communication bandwidth, and transmission time. Experiments show an increase of 0.4 unit in F -measure on an MVS Office dateset and 0.2% and 2% improved accuracy for UCF-50 and YouTube 11 datesets, respectively, with lower storage and transmission times. The processing time is reduced from 1.23 to 0.45 s for a single frame and optimally 0.75 seconds faster MVS. A new dateset is constructed by synthetically adding fog to an MVS dateset to show the adaptability of our system for both certain and uncertain IoT surveillance environments.en
dc.description.sponsorshipManuscript received March 30, 2020; revised June 21, 2020 and August 29, 2020; accepted September 10, 2020. Date of publication September 29, 2020; date of current version June 7, 2021. This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (MSIT, Development of AI-Convergence Technologies for Smart City Industry Productivity Innovation) under Grant 2019-0-00136. (Corresponding author: Sung Wook Baik.) Tanveer Hussain, Amin Ullah, and Sung Wook Baik are with Sejong University, Seoul 143-747, South Korea (e-mail: tanveer445@ieee.org; aminullah@ieee.org; sbaik@sejong.ac.kr).
dc.description.statusPeer reviewed
dc.format.extent11
dc.identifier.citationHussain , T , Muhammad , K , Ullah , A , Ser , J D , Gandomi , A H , Sajjad , M , Baik , S W & De Albuquerque , V H C 2021 , ' Multiview Summarization and Activity Recognition Meet Edge Computing in IoT Environments ' , IEEE Internet of Things Journal , vol. 8 , no. 12 , 9208765 , pp. 9634-9644 . https://doi.org/10.1109/JIOT.2020.3027483
dc.identifier.doi10.1109/JIOT.2020.3027483
dc.identifier.issn2327-4662
dc.identifier.urihttps://hdl.handle.net/11556/5189
dc.identifier.urlhttp://www.scopus.com/inward/record.url?scp=85107497489&partnerID=8YFLogxK
dc.language.isoeng
dc.relation.ispartofIEEE Internet of Things Journal
dc.relation.projectIDInstitute for Information and Communications Technology Promotion, IITP
dc.relation.projectIDMinistry of Science and ICT, South Korea, MSIT, 2019-0-00136
dc.rightsinfo:eu-repo/semantics/openAccess
dc.subject.keywordsActivity recognition
dc.subject.keywordsInternet of Things (IoT)
dc.subject.keywordsdeep autoencoder
dc.subject.keywordsdeep learning
dc.subject.keywordsmultiview video summarization (MVS)
dc.subject.keywordssequential learning
dc.subject.keywordsvideo data analytics
dc.subject.keywordsvideo summarization
dc.subject.keywordsSignal Processing
dc.subject.keywordsInformation Systems
dc.subject.keywordsHardware and Architecture
dc.subject.keywordsComputer Science Applications
dc.subject.keywordsComputer Networks and Communications
dc.titleMultiview Summarization and Activity Recognition Meet Edge Computing in IoT Environmentsen
dc.typejournal article
Files