Hostname: page-component-5b777bbd6c-v47t2 Total loading time: 0 Render date: 2025-06-11T11:34:25.644Z Has data issue: false hasContentIssue false
  • English
  • Français

A probabilistic logic programming event calculus

Published online by Cambridge University Press:  22 May 2014

ANASTASIOS SKARLATIDIS ,
ALEXANDER ARTIKIS ,
JASON FILIPPOU  and
GEORGIOS PALIOURAS
ANASTASIOS SKARLATIDIS
Affiliation:
Institute of Informatics and Telecommunications, NCSR Demokritos, Athens, Greece (e-mail: anskarl@iit.demokritos.gr, a.artikis@iit.demokritos.gr, jfilip@iit.demokritos.gr, paliourg@iit.demokritos.gr) Department of Digital Systems, University of Piraeus, Piraeus, Greece
ALEXANDER ARTIKIS
Affiliation:
Institute of Informatics and Telecommunications, NCSR Demokritos, Athens, Greece (e-mail: anskarl@iit.demokritos.gr, a.artikis@iit.demokritos.gr, jfilip@iit.demokritos.gr, paliourg@iit.demokritos.gr)
JASON FILIPPOU
Affiliation:
Institute of Informatics and Telecommunications, NCSR Demokritos, Athens, Greece (e-mail: anskarl@iit.demokritos.gr, a.artikis@iit.demokritos.gr, jfilip@iit.demokritos.gr, paliourg@iit.demokritos.gr) University of Maryland, College Park, MD, USA
GEORGIOS PALIOURAS
Affiliation:
Institute of Informatics and Telecommunications, NCSR Demokritos, Athens, Greece (e-mail: anskarl@iit.demokritos.gr, a.artikis@iit.demokritos.gr, jfilip@iit.demokritos.gr, paliourg@iit.demokritos.gr)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

We present a system for recognising human activity given a symbolic representation of video content. The input of our system is a set of time-stamped short-term activities (STA) detected on video frames. The output is a set of recognised long-term activities (LTA), which are pre-defined temporal combinations of STA. The constraints on the STA that, if satisfied, lead to the recognition of an LTA, have been expressed using a dialect of the Event Calculus. In order to handle the uncertainty that naturally occurs in human activity recognition, we adapted this dialect to a state-of-the-art probabilistic logic programming framework. We present a detailed evaluation and comparison of the crisp and probabilistic approaches through experimentation on a benchmark dataset of human surveillance videos.

Keywords

event recognitionevent pattern matchingEvent CalculusProbLog
Type
Regular Papers
Information
Theory and Practice of Logic Programming , Volume 15 , Issue 2: Probability, Logic and Learning , March 2015 , pp. 213 - 245
Copyright
Copyright © Cambridge University Press 2014 

References

Allen, J. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM 26, 11, 832–843.CrossRefGoogle Scholar
Artikis, A., Sergot, M. and Paliouras, G. 2010. A logic programming approach to activity recognition. In Proceedings of ACM Workshop on Events in Multimedia. ACM, New York, NY, USA, 3–8.CrossRefGoogle Scholar
Artikis, A., Sergot, M. and Paliouras, G. 2012. Run-time composite event recognition. In Proceedings of International Conference on Distributed Event-Based Systems (DEBS). Bry, F., Paschke, A., Eugster, P. T., Fetzer, C. and Behrend, A., Eds. ACM, New York, NY, USA, 69–80.CrossRefGoogle Scholar
Artikis, A., Skarlatidis, A., Portet, F. and Paliouras, G. 2012. Logic-based event recognition. Knowledge Engineering Review 27, 469–506.CrossRefGoogle Scholar
Biswas, R., Thrun, S. and Fujimura, K. 2007. Recognizing activities with multiple cues. In Proceedings of Workshop on Human Motion, Elgammal, A. M., Rosenhahn, B. and Klette, R., Eds. LNCS, vol. 4814. Springer, 255–270.Google Scholar
Brand, M., Oliver, N. and Pentland, A. 1997. Coupled hidden Markov models for complex action recognition. In Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR). Plummer, D. and Tonvick, I., Eds. IEEE Computer Society, Los Alamitos, CA, USA, 994–999.CrossRefGoogle Scholar
Brendel, W., Fern, A. and Todorovic, S. 2011. Probabilistic event logic for interval-based event recognition. In Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 3329–3336.Google Scholar
Bruynooghe, M., Mantadelis, T., Kimmig, A., Gutmann, B., Vennekens, J., Janssens, G. and De Raedt, L. 2010. ProbLog technology for inference in a probabilistic first order logic. In Proceedings of European Conference on Artificial Intelligence (ECAI). Coelho, H., Studer, R. and Wooldridge, M., Eds. IOS Press, 719–724.Google Scholar
Bryant, R. 1986. Graph-based algorithms for Boolean function manipulation. IEEE Transactions on Computers 35, 8, 677–691.CrossRefGoogle Scholar
Cugola, G. and Margara, A. 2011. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys 44, 3.Google Scholar
Dousson, C. and Maigat, P. L. 2007. Chronicle recognition improvement using temporal focusing and hierarchisation. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI). Veloso, M. M., Ed. 324–329.Google Scholar
Fierens, D., den Broeck, G. V., Thon, I., Gutmann, B. and De Raedt, L. 2011. Inference in probabilistic logic programs using weighted CNF's. In Proceedings of International Conference on Uncertainty in Artificial Intelligence (UAI). Cozman, F. G. and Pfeffer, A., Eds. AUAI Press, 211–220.Google Scholar
Gibson, J. 1979. The ecological approach to visual perception. Houghton Mifflin, Boston.Google Scholar
Ginsberg, M. L. 1990. Bilattices and modal operators. Journal of Logic and Computation 1, 1–41.CrossRefGoogle Scholar
Gong, S. and Xiang, T. 2003. Recognition of group activities using dynamic probabilistic networks. In Proceedings of International Conference on Computer Vision. IEEE Computer Society, Los Alamitos, CA, USA, 742–749.Google Scholar
Hakeem, A. and Shah, M. 2007. Learning, detection and representation of multi-agent events in videos. Artificial Intelligence 171, 8–9, 586–605.CrossRefGoogle Scholar
Helaoui, R., Niepert, M. and Stuckenschmidt, H. 2011. Recognizing interleaved and concurrent activities: A statistical-relational approach. In Proceedings of International Conference on Pervasive Computing and Communications. IEEE Press, New York, NY, USA, 1–9.Google Scholar
Hongeng, S. and Nevatia, R. 2003. Large-scale event detection using semi-Hidden Markov Models. In Proceedings of International Conference on Computer Vision. IEEE Computer Society, Los Alamitos, CA, USA, 1455–1462.Google Scholar
Kembhavi, A., Yeh, T. and Davis, L. S. 2010. Why did the person cross the road (there)? scene understanding using probabilistic logic models and common sense reasoning. In Proceedings of European Conference on Computer Vision (ECCV). Daniilidis, K., Maragos, P. and Paragios, N., Eds. Springer Berlin Heidelberg, 693–706.Google Scholar
Kimmig, A., Demoen, B., De Raedt, L., Costa, V. S. and Rocha, R. 2011. On the implementation of the probabilistic logic programming language ProbLog. Theory and Practice of Logic Programming 11, 235–262.CrossRefGoogle Scholar
Kosmopoulos, D., Antonakaki, P., Valasoulis, K., Kesidis, A. and Perantonis, S. 2008. Human behaviour classification using multiple views. In Proceedings of Hellenic Conference on Artificial Intelligence, vol. 5138. Darzentas, J., Vouros, G., Vosinakis, S. and Arnellos, A., Eds. Springer Berlin Heidelberg.Google Scholar
Kowalski, R. and Sergot, M. 1986. A logic-based calculus of events. New Generation Computing 4, 1, 67–96.CrossRefGoogle Scholar
Lafferty, J. D., McCallum, A. and Pereira, F. C. N. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of International Conference on Machine Learning (ICML), Brodley, C. E. and Danyluk, A. P., Eds. Morgan Kaufmann, San Francisco, CA, USA, 282–289.Google Scholar
Liao, L., Fox, D. and Kautz, H. 2007. Hierarchical conditional random fields for GPS-based activity recognition. Robotics Research 28, 487–506.CrossRefGoogle Scholar
Luckham, D. 2002. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley Professional, Indianapolis, Indiana, USA.Google Scholar
Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J. and De Raedt, L. 2012. Learning relational affordance models for robots in multi-object manipulation tasks. In Proceedings of International Conference on Robotics and Automation (ICRA). IEEE Computer Society, Los Alamitos, CA, USA, 4373–4378.Google Scholar
Morariu, V. I. and Davis, L. S. 2011. Multi-agent event recognition in structured scenarios. In Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 3289–3296.Google Scholar
Murphy, K. 2002. Dynamic Bayesian Networks: Representation, Inference and Learning. PhD thesis, University of California.Google Scholar
Natarajan, P. and Nevatia, R. 2007. Hierarchical multi-channel semi Hidden Markov Models. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2562–2567.Google Scholar
Rabiner, L. and Juang, B. 1986. An introduction to Hidden Markov Models. ASSP Magazine 3, 1, 4–16.CrossRefGoogle Scholar
Richardson, M. and Domingos, P. 2006. Markov logic networks. Machine Learning 62, 1–2, 107–136.CrossRefGoogle Scholar
Sadilek, A. and Kautz, H. 2012. Location-based reasoning about complex multi-agent behavior. Journal of Artificial Intelligence Research 43, 87–133.CrossRefGoogle Scholar
Selman, J., Amer, M., Fern, A. and Todorovic, S. 2011. PEL-CNF: Probabilistic event logic conjunctive normal form for video interpretation. In Computer Vision Workshops. IEEE Computer Society, Los Alamitos, CA, USA, 680–687.Google Scholar
Shet, V., Harwood, D. and Davis, L. 2005. VidMAP: Video monitoring of activity with Prolog. In Proceedings of International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE Computer Society, Los Alamitos, CA, USA, 224–229.Google Scholar
Shet, V., Neumann, J., Ramesh, V. and Davis, L. 2007. Bilattice-based logical reasoning for human detection. In Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 1–8.Google Scholar
Siskind, J. 2001. Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic. Journal of Artificial Intelligence Research 15, 31–90.CrossRefGoogle Scholar
Skarlatidis, A., Paliouras, G., Vouros, G. A. and Artikis, A. 2011. Probabilistic event calculus based on Markov logic networks. In RuleML America. Olken, F., Palmirani, M. and Sottara, D., Eds. Springer Berlin Heidelberg, 155–170.Google Scholar
Tran, S. D. and Davis, L. S. 2008. Event modeling and recognition using Markov logic networks. In Proceedings of European Conference on Computer Vision (ECCV). Forsyth, D. A., Torr, P. H. S. and Zisserman, A., Eds. Springer Berlin Heidelberg, 610–623.Google Scholar
Vail, D., Veloso, M. and Lafferty, J. 2007. Conditional random fields for activity recognition. In Proceedings of International Conference on Autonomous Agents and Multiagent Systems (AAMAS). ACM, New York, NY, USA, 1–8.Google Scholar
Valiant, L. G. 1979. The complexity of enumeration and reliability problems. SIAM Journal on Computing 8, 410–421.CrossRefGoogle Scholar
Wang, J. and Domingos, P. 2008. Hybrid Markov logic networks. In Proceedings of Conference on Artificial Intelligence (AAAI). Fox, D. and Gomes, C. P., Eds. AAAI Press, Menlo Park, California, USA, 1106–1111.Google Scholar
Wu, T., Lian, C. and Hsu, J. 2007. Joint recognition of multiple concurrent activities using factorial conditional random fields. In Proceedings of AAAI Workshop on Plan, Activity, and Intent Recognition. AAAI Press, Menlo Park, California, USA.Google Scholar