Artificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Data

Simple item page
dc.contributor.authorUllah, Waseem
dc.contributor.authorUllah, Amin
dc.contributor.authorHussain, Tanveer
dc.contributor.authorMuhammad, Khan
dc.contributor.authorHeidari, Ali Asghar
dc.contributor.authorDel Ser, Javier
dc.contributor.authorBaik, Sung Wook
dc.contributor.authorDe Albuquerque, Victor Hugo C.
dc.contributor.institutionIA
dc.date.accessioned2024-09-10T13:45:02Z
dc.date.available2024-09-10T13:45:02Z
dc.date.issued2022-04
dc.descriptionPublisher Copyright: © 2021 Elsevier B.V.
dc.description.abstractIn the last few years, visual sensors are deployed almost everywhere, generating a massive amount of surveillance video data in smart cities that can be inspected intelligently to recognize anomalous events. In this work, we present an efficient and robust framework to recognize anomalies from surveillance Big Video Data (BVD) using Artificial Intelligence of Things (AIoT). Smart surveillance is an important application of AIoT and we propose a two-stream neural network in this direction. The first stream comprises instant anomaly detection that is functional over resource-constrained IoT devices, whereas second phase is a two-stream deep neural network allowing for detailed anomaly analysis, suited to be deployed as a cloud computing service. Firstly, a self-pruned fine-tuned lightweight convolutional neural network (CNN) classifies the ongoing events as normal or anomalous in an AIoT environment. Upon anomaly detection, the edge device alerts the concerned departments and transmits the anomalous frames to cloud analysis center for their detailed evaluation in the second phase. The cloud analysis center resorts to the proposed two-stream network, modeled from the integration of spatiotemporal and optical flow features through the sequential frames. Fused features flow through a bi-directional long short-term memory (BD-LSTM) layer, which classifies them into their respective anomaly classes, e.g., assault and abuse. We perform extensive experiments over benchmarks built on top of the UCF-Crime and RWF-2000 datasets to test the effectiveness of our framework. We report a 9.88% and 4.01% increase in accuracy when compared to state-of-the-art methods evaluated over the aforementioned datasets.en
dc.description.sponsorshipThis work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1A2B5B01070067 ). All authors have read and agreed to the submitted version of the manuscript.
dc.description.statusPeer reviewed
dc.format.extent12
dc.identifier.citationUllah , W , Ullah , A , Hussain , T , Muhammad , K , Heidari , A A , Del Ser , J , Baik , S W & De Albuquerque , V H C 2022 , ' Artificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Data ' , Future Generation Computer Systems , vol. 129 , pp. 286-297 . https://doi.org/10.1016/j.future.2021.10.033
dc.identifier.doi10.1016/j.future.2021.10.033
dc.identifier.issn0167-739X
dc.identifier.urihttps://hdl.handle.net/11556/5120
dc.identifier.urlhttp://www.scopus.com/inward/record.url?scp=85121546312&partnerID=8YFLogxK
dc.language.isoeng
dc.relation.ispartofFuture Generation Computer Systems
dc.relation.projectIDNational Research Foundation of Korea, NRF
dc.relation.projectIDMinistry of Science and ICT, South Korea, MSIT, 2019R1A2B5B01070067
dc.rightsinfo:eu-repo/semantics/restrictedAccess
dc.subject.keywordsAnomaly detection
dc.subject.keywordsAnomaly recognition
dc.subject.keywordsSurveillance videos
dc.subject.keywordsTwo-stream network
dc.subject.keywordsSoftware
dc.subject.keywordsHardware and Architecture
dc.subject.keywordsComputer Networks and Communications
dc.subject.keywordsSDG 11 - Sustainable Cities and Communities
dc.titleArtificial Intelligence of Things-assisted two-stream neural network for anomaly detection in surveillance Big Video Dataen
dc.typejournal article
Files