Hostname: page-component-8448b6f56d-gtxcr Total loading time: 0 Render date: 2024-04-19T17:58:29.083Z Has data issue: false hasContentIssue false
  • English
  • Français

A survey and taxonomy on intelligent surveillance from a system perspective

Published online by Cambridge University Press:  25 April 2018

Radu-Casian Mihailescu ,
Paul Davidsson ,
Ulrik Eklund  and
Jan A. Persson
Radu-Casian Mihailescu
Affiliation:
Department of Computer Science, Malmö University, Internet of Things and People Research Center, 205 06 Malmö, Sweden e-mail: radu.c.mihailescu@mah.se, paul.davidsson@mah.se, ulrik.eklund@mah.se, jan.a.persson@mah.se
Paul Davidsson
Affiliation:
Department of Computer Science, Malmö University, Internet of Things and People Research Center, 205 06 Malmö, Sweden e-mail: radu.c.mihailescu@mah.se, paul.davidsson@mah.se, ulrik.eklund@mah.se, jan.a.persson@mah.se
Ulrik Eklund
Affiliation:
Department of Computer Science, Malmö University, Internet of Things and People Research Center, 205 06 Malmö, Sweden e-mail: radu.c.mihailescu@mah.se, paul.davidsson@mah.se, ulrik.eklund@mah.se, jan.a.persson@mah.se
Jan A. Persson
Affiliation:
Department of Computer Science, Malmö University, Internet of Things and People Research Center, 205 06 Malmö, Sweden e-mail: radu.c.mihailescu@mah.se, paul.davidsson@mah.se, ulrik.eklund@mah.se, jan.a.persson@mah.se
Get access Rights & Permissions [Opens in a new window]

Abstract

Recent proliferation of surveillance systems is mostly attributed to advances in both image-processing techniques and hardware enhancement of smart cameras, as well as the ubiquity of sensor-driven architectures. Owing to these capabilities, new aspects are coming to the forefront. This paper addresses the current state-of-the-art and provides researchers with an overview of existing surveillance solutions, analyzing their properties as a system and drawing attention to relevant challenges when developing, deploying and managing them. Also, some of the more prominent application domains are highlighted here. In an effort to understand the development of the advanced solutions, based on their most distinctive characteristics, we propose a taxonomy for surveillance systems to help classify them and reveal gaps in existing research. We conclude by identifying promising future research lines.

Type
Principles and Practice of Multi-Agent Systems
Information
The Knowledge Engineering Review , Volume 33 , 2018 , e4
Copyright
© Cambridge University Press, 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aguilar-Ponce, R., Kumar, A., Tecpanecatl-Xihuitl, J. L. & Bayoumi, M. 2007. A network of sensor-based framework for automated visual surveillance. Journal of Network and Computer Applications 30(3), 1244–1271.Google Scholar
Ahmadi, A., Mitchell, E., Richter, C., Destelle, F., Gowing, M., O’Connor, N. & Moran, K. 2015. Toward automatic activity classification and movement assessment during a sports training session. Internet of Things Journal, IEEE 2(1), 23–32.Google Scholar
Bellifemine, F., Caire, G., Poggi, A. & Rimassa, G. 2008. JADE: a software framework for developing multi-agent applications. Lessons learned. Information and Software Technology 50(12), 10–21.Google Scholar
Bicocchi, N., Fontana, D. & Zambonelli, F. 2014. A self-aware, reconfigurable architecture for context awareness. In IEEE Symposium on Computers and Communication (ISCC), 1–7.Google Scholar
Biswas, P. K., Qi, H. & Xu, Y. 2008. Mobile-agent-based collaborative sensor fusion. Information Fusion 9(3), 399–411.Google Scholar
Braubach, L., Lamersdorf, W. & Pokahr, A. 2003. Jadex: implementing a BDI-infrastructure for JADE agents. Search of Innovation (Special Issue on JADE) 3, 76–85.Google Scholar
Brown, B., Wei, W., Ozburn, R., Kumar, M. & Cohen, K. 2015. Surveillance for intelligent emergency response robotic aircraft (SIERRA)- VTOL aircraft for emergency response. In AIAA Infotech. American Institute of Aeronautics and Astronautics.Google Scholar
Brutzer, S., Hoferlin, B. & Heidemann, G. 2011. Evaluation of background subtraction techniques for video surveillance. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1937–1944.Google Scholar
Bustamante, A. L., Molina, J. M. & Patricio, M. A. 2014. A practical approach for active camera coordination based on a fusion-driven multi-agent system. International Journal of Systems Science 45(4), 741–755.Google Scholar
Byun, J. S. & Park, S. 2011. Development of a self-adapting intelligent system for building energy saving and context-aware smart services. IEEE Transactions on Consumer Electronics 57(1), 90–98.Google Scholar
Camplani, M. & Salgado, L. 2011. Scalable software architecture for on-line multi-camera video processing. In Proceedings of SPIE, 7871, 6–21.Google Scholar
Carchiolo, V., Longheu, A., Malgeri, M., Mangioni, G. & Torrisi, D. 2010. Pervasive home security: an intelligent domotics application. In Intelligent Distributed Computing IV, Studies in Computational Intelligence 315, M. Essaaidi, M. Malgeri & C. Badica (eds). Springer Berlin Heidelberg, 155–164.Google Scholar
Castanedo, F., Garcia, J., Patricio, M. & Molina, J. 2008. A multi-agent architecture to support active fusion in a visual sensor network. In Second ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), 1–8.Google Scholar
Chen, S.-L., Chang, S.-K. & Chen, Y.-Y. 2015. Development of a multisensor embedded intelligent home environment monitoring system based on digital signal processor and Wi-Fi. International Journal of Distributed Sensor Networks 2015, 11.Google Scholar
Cho, Y., Lim, S. O. & Yang, H. S. 2010. Collaborative occupancy reasoning in visual sensor network for scalable smart video surveillance. IEEE Transactions on Consumer Electronics 56(3), 1997–2003.Google Scholar
Christensen, E., Curbera, F., Meredith, G. & Weerawarana, S. 2001. Web Service Definition Language (WSDL). Technical report, http://www.w3.org/TR/wsdl.Google Scholar
Collins, R., Lipton, A., Fujiyoshi, H. & Kanade, T. 2001. Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE 89(10), 1456–1477.Google Scholar
Compton, M., Barnaghi, P., Bermudez, L., Garca-Castro, R., Corcho, O., Cox, S., Graybeal, J., Hauswirth, M., Henson, C., Herzog, A., Huang, V., Janowicz, K., Kelsey, W. D., Phuoc, D. L., Lefort, L., Leggieri, M., Neuhaus, H., Nikolov, A., Page, K., Passant, A., Sheth, A. & Taylor, K. 2012. The SSN ontology of the W3C semantic sensor network incubator group. Web Semantics: Science, Services and Agents on the World Wide Web 17, 25–32.Google Scholar
Davidsson, P. & Boman, M. 2000. Saving energy and providing value added services in intelligent buildings: a MAS approach. In Agent Systems, Mobile Agents, and Applications, Lecture Notes in Computer Science 1882, D. Kotz & F. Mattern (eds). Springer Berlin Heidelberg, 166–177.Google Scholar
Davidsson, P. & Boman, M. 2005. Distributed monitoring and control of office buildings by embedded agents. Information Sciences 171(4), 293–307.Google Scholar
Fernández-De-Alba, J. M., Fuentes-Fernández, R. & Pavón, J. 2015. Architecture for management and fusion of context information. Information Fusion 21, 100–113.Google Scholar
Findeisen, M., Meinel, L., Richter, J. & Hirtz, G. 2015. An omnidirectional stereo sensor for human behavior analysis in complex indoor environments. In 2015 IEEE International Conference on Consumer Electronics (ICCE), 17–19.Google Scholar
Gascuena, J. M., Castillo, J. C., Navarro, E. & Fernandez-Caballero, A. 2014. Engineering the development of systems for multisensory monitoring and activity interpretation. International Journal of Systems Science 45(4), 728–740.Google Scholar
Hachem, S., Teixeira, T. & Issarny, V. 2011. Ontologies for the internet of things. In Proceedings of the 8th Middleware Doctoral Symposium MDS ‘11, 3:1–3:6. ACM.Google Scholar
Haesevoets, R., Van Eylen, B., Weyns, D., Helleboogh, A., Holvoet, T. & Joosen, W. 2007. Context-driven dynamic organizations applied to coordinated monitoring of traffic jams. In Proceedings of Workshop Engineering Environment-Mediated Multiagent Systems, 126–143.Google Scholar
Hoffmann, M., Wittke, M., Hahner, J. & Muller-Schloer, C. 2008. Spatial partitioning in self-organizing smart camera systems. IEEE Journal of Selected Topics in Signal Processing 2(4), 480–492.Google Scholar
Hu, W., Tan, T., Wang, L. & Maybank, S. 2004. A survey on visual surveillance of object motion and behaviors. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 34(3), 334–352.Google Scholar
Hull, R., Kumar, B., Lieuwen, D., Patel-Schneider, P. F., Sahuguet, A., Varadarajan, S. & Vyas, A. 2005. Improving user experience through rule-based service customization. International Journal of Cooperative Information Systems 14(4), 469–502.Google Scholar
Jovanovic, M. & Rinner, B. 2007. Middleware for dynamic reconfiguration in distributed camera systems. In Fifth Workshop on Intelligent Solutions in Embedded Systems, 139–150.Google Scholar
Khajenasiri, I., Patti, E., Jahn, M., Acquaviva, A., Verhelst, M., Macii, E. & Gielen, G. 2014. Design and implementation of a multi-standard event-driven energy management system for smart buildings. In IEEE 3rd Global Conference on Consumer Electronics (GCCE), 20–21.Google Scholar
Ko, T. 2008. A survey on behavior analysis in video surveillance for homeland security applications. In 37th IEEE Applied Imagery Pattern Recognition Workshop, 1–8.Google Scholar
Krahnstoever, N., Yu, T., nam Lim, S., Patwardhan, K. & Tu, P. 2008. Collaborative real-time control of active cameras in large scale surveillance systems. In Proceedings Workshop on Multicamera and Multi-modal Sensor Fusion Algorithms and Applications, 26, 1–12.Google Scholar
Lassila, O. & Swick, R. R. 1999. Resource Description Framework (RDF) Model and Syntax Specification. Technical report, W3C. ttp://www.w3.org/TR/REC-rdf-syntax/.Google Scholar
Li, X., Lu, R., Liang, X., Shen, X., Chen, J. & Lin, X. 2011. Smart community: an internet of things application. IEEE Communications Magazine 49(11), 68–75.Google Scholar
Longheu, A., Carchiolo, V., Malgeri, M., Mangioni, G., Longheu, R., Carchiolo, V. & Malgeri, M., I, V. A. D. 2012. An intelligent and pervasive surveillance system for home security. International Journal Computer Communications 7(2), 312–324.Google Scholar
Meinel, L., Findeisen, M., Hes, M., Apitzsch, A. & Hirtz, G. 2014. Automated real-time surveillance for ambient assisted living using an omnidirectional camera. In IEEE International Conference on Consumer Electronics (ICCE), 396–399.Google Scholar
Meisels, A. 2007. Distributed Search by Constrained Agents: Algorithms, Performance, Communication. Advanced Information and Knowledge Processing. Springer.Google Scholar
Mihailescu, R., Persson, J. A., Davidsson, P. & Eklund, U. 2016. Towards collaborative sensing using dynamic intelligent virtual sensors. In Intelligent Distributed Computing X – Proceedings of the 10th International Symposium on Intelligent Distributed Computing – IDC 2016, 217–226. 10–12 October.Google Scholar
Monari, E., Voth, S. & Kroschel, K. 2008. An object- and task-oriented architecture for automated video surveillance in distributed sensor networks. In IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance (AVSS ‘08), 339–346.Google Scholar
Nilsson, F. 2008. Intelligent Network Video: Understanding Modern Video Surveillance Systems. CRC Press.Google Scholar
Nguyen, N. T., Venkatesh, S., West, G. & Bui, H. H. 2003. Multiple camera coordination in a surveillance system. ACTA Automatica Sinica 29, 408–422.Google Scholar
Nguyen, T. A., Raspitzu, A. & Aiello, M. 2014. Ontology-based office activity recognition with applications for energy savings. Journal of Ambient Intelligence and Humanized Computing 5(5), 667–681.Google Scholar
Oliver, N., Horvitz, E. & Garg, A. 2002. Layered representations for human activity recognition. In Proceedings Fourth IEEE International Conference on Multimodal Interfaces, 3–8.Google Scholar
Oliver, N., Rosario, B. & Pentland, A. 2000. A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 831–843.Google Scholar
Park, H. D., Min, O.-G. & Lee, Y.-J. 2017. Scalable architecture for an automated surveillance system using edge computing. Journal of Supercomputing 73(3), 926–939.Google Scholar
Patti, E., Acquaviva, A., Jahn, M., Pramudianto, F., Tomasi, R., Rabourdin, D., Virgone, J. & Macii, E. 2014. Event-driven user-centric middleware for energy-efficient buildings and public spaces. IEEE Systems Journal 10(3), 1137–1146.Google Scholar
Pavon, J., Gomez-Sanz, J., Fernndez-Caballero, A. & Valencia-Jimenez, J. J. 2007. Development of intelligent multisensor surveillance systems with agents. Robotics and Autonomous Systems 55(12), 892–903.Google Scholar
Piette, F., Dinont, C., Seghrouchni, A. E. F. & Taillibert, P. 2015. Deployment and configuration of applications for ambient systems. In Proceedings of the 6th International Conference on Ambient Systems, Networks and Technologies, ANT-2015, Procedia Computer Science, 27, 373–380.Google Scholar
Power, P. W. & Schoonees, J. A. 2002. Understanding background mixture models for foreground segmentation. In Proceedings Image and Vision Computing New Zealand, 267–271.Google Scholar
Prud’hommeaux, E. & Seaborne, A. 2006. SPARQL Query Language for RDF, Technical report. http://www.w3.org/TR/rdf-sparql-query/.Google Scholar
Qian, H., Wu, X. & Xu, Y. 2011. Intelligent Surveillance Systems, 51. Springer Science & Business Media.Google Scholar
Quaritsch, M., Kreuzthaler, M., Rinner, B., Bischof, H. & Strobl, B. 2007. Autonomous multicamera tracking on embedded smart cameras. EURASIP Journal on Embedded Systems 2007(1), 35–45.Google Scholar
Rao, A. S. & Georgeff, M. P. 1995. BDI agents: from theory to practice. in Proceedings of the First International Conference on Multi-Agent Systems (ICMAS), 312–319. MIT Press.Google Scholar
Rothkrantz, L. 2013. Crisis management using multiple camera surveillance systems. In Proceedings of the 10th International Conference on Information Systems for Crisis Response and Management, 617–626.Google Scholar
Ryoo, M. S. & Aggarwal, J. K. 2008. Recognition of high-level group activities based on activities of individual members. In Proceedings of the 2008 IEEE Workshop on Motion and Video Computing, 1–8.Google Scholar
Silva, R., Arakaki, J., Junqueira, F., Filho, D. S. & Miyagi, P. 2012. Modeling of active holonic control systems for intelligent buildings. Automation in Construction 25, 20–33.Google Scholar
Snidaro, L., García, J. & Llinas, J. 2015. Context-based information fusion. Information Fusion 25(C), 16–31.Google Scholar
Sobhani, F., Kahar, N. F. & Zhang, Q. 2015. An ontology framework for automated visual surveillance system. In Proceeding of the 13th International Workshop on Content-Based Multimedia Indexing (CBMI), 1–7.Google Scholar
Soldatos, J., Pandis, I., Stamatis, K., Polymenakos, L. & Crowley, J. L. 2007. Agent based middleware infrastructure for autonomous context-aware ubiquitous computing services. Computer Communications 30(3), 577–591.Google Scholar
Stavropoulos, T. G., Vrakas, D., Vlachava, D. & Bassiliades, N. 2012. BOnSAI: a smart building ontology for ambient intelligence. In Proceedings of the Second International Conference on Web Intelligence, Mining and Semantics, WIMS ‘12, 30:1–30:12. ACM.Google Scholar
Town, C. 2007. Multi-sensory and multi-modal fusion for sentient computing. International Journal of Computer Vision 71(2), 235–253.Google Scholar
Valera, M. & Velastin, S. 2005. Intelligent distributed surveillance systems: a review. IEEE Vision, Image and Signal Processing 152(2), 192–204.Google Scholar
Veeraraghavan, A., Chellappa, R. & Roy-Chowdhury, A. 2006. The function space of an activity. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, 959–968.Google Scholar
Velipasalar, S., Schlessman, J., Chen, C.-Y., Wolf, W. & Singh, J. 2006. SCCS: a scalable clustered camera system for multiple object tracking communicating via message passing interface. In IEEE International Conference on Multimedia and Expo, 28, 277–280.Google Scholar
Vicaire, P., He, T., Cao, Q., Yan, T., Zhou, G., Gu, L., Luo, L., Stoleru, R., Stankovic, J. A. & Abdelzaher, T. F. 2009. Achieving long-term surveillance in vigilnet. ACM Transactions on Sensor Networks 5(1), 9:1–9:39.Google Scholar
Vogler, C. & Metaxas, D. 1999. Parallel hidden Markov models for American sign language recognition. In The Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, 1, 116–122.Google Scholar
W3C OWL Working Group 2009. OWL 2 Web Ontology Language Document Overview, Technical report, W3C. http://www.w3.org/TR/2009/REC-owl2-overview-20091027/.Google Scholar
Weinland, D., Ronfard, R. & Boyer, E. 2011. A survey of vision-based methods for action representation, segmentation and recognition. Computer Vision and Image Understanding 115(2), 224–241.Google Scholar
Winkler, T. & Rinner, B. 2014. Security and privacy protection in visual sensor networks: a survey. ACM Computing Surveys 47(1), 2:1–2:42.Google Scholar
Woolridge, M. 2009. Introduction to Multiagent Systems, 2nd edition. John Wiley & Sons Inc.Google Scholar
Yan, L., Chao, L., Ke Wei, P. & Tao, S. 2015. Crowd counting using wireless infrared distance sensors for indoor environments. In Informatics and Communication Technologies for Societal Development, E. B. Rajsingh, A. Bhojan & J. D. Peter (eds). Springer, 1–8.Google Scholar
Yi, P., Iwayemi, A. & Zhou, C. 2011. Developing ZigBee deployment guideline under WiFi interference for smart grid applications. IEEE Transactions on Smart Grid 2(1), 110–120.Google Scholar
Yilmaz, A., Javed, O. & Shah, M. 2006. Object tracking: a survey. ACM Computing Surveys 38(4), 1–45.Google Scholar
Yong, C., Qiao, B., Wilson, D., Wu, M., Clements-Croome, D., Liu, K., Egan, R. & Guy, C. 2007. Coordinated management of intelligent pervasive spaces. In Proceedings of the 5th IEEE.Google Scholar
Yu, T., Sekar, V., Seshan, S., Agarwal, Y. & Xu, C. 2015. Handling a trillion (unfixable) aws on a billion devices: rethinking network security for the internet-of-things. In Proceedings of the 14th ACM Workshop on Hot Topics in Networks, HotNets-XIV, 5:1–5:7. ACM.Google Scholar
Zhang, D., Gatica-Perez, D., Bengio, S. & McCowan, I. 2006. Modeling individual and group actions in meetings with layered HMMs. IEEE Transactions on Multimedia 8(3), 509–520.Google Scholar
Zhu, F., Mutka, M. W. & Ni, L. M. 2005. Service discovery in pervasive computing environments. IEEE Pervasive Computing 4(4), 81–90.Google Scholar