Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-24T01:44:25.748Z Has data issue: false hasContentIssue false
  • English
  • Français

Managing caching strategies for stream reasoning with reinforcement learning

Published online by Cambridge University Press:  21 September 2020

CARMINE DODARO Open the ORCID record for CARMINE DODARO [Opens in a new window] ,
THOMAS EITER Open the ORCID record for THOMAS EITER [Opens in a new window] ,
PAUL OGRIS Open the ORCID record for PAUL OGRIS [Opens in a new window]  and
KONSTANTIN SCHEKOTIHIN Open the ORCID record for KONSTANTIN SCHEKOTIHIN [Opens in a new window]
CARMINE DODARO
Affiliation:
Department of Mathematics and Computer Science, University of Calabria, Italy (e-mail: dodaro@mat.unical.it)
THOMAS EITER
Affiliation:
Institute of Logic and Computation, KBS Group, Vienna University of Technology, Austria, (e-mail: eiter@kr.tuwien.ac.at)
PAUL OGRIS
Affiliation:
Alpen-Adria-Universität, Klagenfurt, Austria, (e-mail: paul.ogris@aau.at, konstantin.schekotihin@aau.at)
KONSTANTIN SCHEKOTIHIN
Affiliation:
Alpen-Adria-Universität, Klagenfurt, Austria, (e-mail: paul.ogris@aau.at, konstantin.schekotihin@aau.at)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Efficient decision-making over continuously changing data is essential for many application domains such as cyber-physical systems, industry digitalization, etc. Modern stream reasoning frameworks allow one to model and solve various real-world problems using incremental and continuous evaluation of programs as new data arrives in the stream. Applied techniques use, e.g., Datalog-like materialization or truth maintenance algorithms to avoid costly re-computations, thus ensuring low latency and high throughput of a stream reasoner. However, the expressiveness of existing approaches is quite limited and, e.g., they cannot be used to encode problems with constraints, which often appear in practice. In this paper, we suggest a novel approach that uses the Conflict-Driven Constraint Learning (CDCL) to efficiently update legacy solutions by using intelligent management of learned constraints. In particular, we study the applicability of reinforcement learning to continuously assess the utility of learned constraints computed in previous invocations of the solving algorithm for the current one. Evaluations conducted on real-world reconfiguration problems show that providing a CDCL algorithm with relevant learned constraints from previous iterations results in significant performance improvements of the algorithm in stream reasoning scenarios.

Keywords

Stream reasoningASPReinforcement learningCaching strategies
Type
Original Article
Information
Theory and Practice of Logic Programming , Volume 20 , Issue 5: 36th International Conference on Logic Programming Special Issue I , September 2020 , pp. 625 - 640
Check for updates
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2020. Published by Cambridge University Press

References

Adams, E. N. 1984. Optimizing preventive service of software products. IBM J. Res. Dev. 28, 1, 2–14.Google Scholar
Alviano, M., Dodaro, C., Faber, W., Leone, N., and Ricca, F. 2013. WASP: A native ASP solver based on constraint learning. In LPNMR. 54–66.Google Scholar
Alviano, M., Dodaro, C., Leone, N., and Ricca, F. 2015. Advances in WASP. In LPNMR. 40–54.Google Scholar
Anantharam, V., Varaiya, P., and Walrand, J. 1987. Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-Part I: I.I.D. rewards. IEEE Trans. on Automatic Control 32, 11, 968–976.Google Scholar
Aschinger, M., Drescher, C., Friedrich, G., Gottlob, G., Jeavons, P., Ryabokon, A., and Thorstensen, E. 2011. Optimization methods for the partner units problem. In CPAIOR. 4–19.Google Scholar
Audemard, G. and Simon, L. 2009. Predicting learnt clauses quality in modern SAT solvers. In IJCAI. 399–404.Google Scholar
Audemard, G. and Simon, L. 2018. On the glucose SAT solver. Int. J. Artif. Intell. Tools 27, 1, 1–25.Google Scholar
Bazoobandi, H. R., Beck, H., and Urbani, J. 2017. Expressive stream reasoning with laser. In ISWC. 87–103.Google Scholar
Beck, H., Bierbaumer, B., Dao-Tran, M., Eiter, T., Hellwagner, H., and Schekotihin, K. 2017. Stream reasoning-based control of caching strategies in CCN routers. In ICC. IEEE, 1–6.Google Scholar
Beck, H., Dao-Tran, M., and Eiter, T. 2015. Answer update for rule-based stream reasoning. In IJCAI. AAAI Press, 2741–2747.Google Scholar
Beck, H., Dao-Tran, M., and Eiter, T. 2018. LARS: A logic-based framework for analytic reasoning over streams. Artif. Intell. 261, 16–70.CrossRefGoogle Scholar
Beck, H., Eiter, T., and Folie, C. 2017. Ticker: A system for incremental asp-based stream reasoning. TPLP 17, 5-6, 744–763.Google Scholar
Calimeri, F., Ianni, G., Pacenza, F., Perri, S., and Zangari, J. 2019. Incremental answer set programming with overgrounding. Theory Pract. Log. Program. 19, 5-6, 957–973.CrossRefGoogle Scholar
de Kleer, J. 1986. An assumption-based TMS. Artif. Intell. 28, 2, 127–162.Google Scholar
Doyle, J. 1979. A truth maintenance system. Artif. Intell. 12, 3, 231–272.Google Scholar
Eiter, T., Ogris, P., and Schekotihin, K. 2019. A distributed approach to LARS stream reasoning (system paper). Theory Pract. Log. Program. 19, 5-6, 974–989.Google Scholar
Gai, Y., Krishnamachari, B., and Jain, R. 2012. Combinatorial Network Optimization With Unknown Variables: Multi-Armed Bandits With Linear Rewards and Individual Observations. IEEE/ACM Transactions on Networking 20, 5, 1466–1478.Google Scholar
Gaschnig, J. 1979. Performance measurement and analysis of certain search algorithms. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, USA.Google Scholar
Gebser, M., Grote, T., Kaminski, R., Obermeier, P., Sabuncu, O., and Schaub, T. 2012. Stream reasoning with answer set programming: Preliminary report. In KR. AAAI Press.Google Scholar
Gebser, M., Kaminski, R., Kaufmann, B., and Schaub, T. 2019. Multi-shot ASP solving with clingo. Theory Pract. Log. Program. 19, 1, 27–82.Google Scholar
Gelfond, M. and Lifschitz, V. 1988. The stable model semantics for logic programming. In ICLP/SLP. MIT Press, 1070–1080.Google Scholar
Gent, I. P., Jefferson, C., and Nightingale, P. 2017. Complexity of n-queens completion. J. Artif. Intell. Res. 59, 815–848.Google Scholar
Gomes, C. P., Selman, B., and Kautz, H. A. 1998. Boosting combinatorial search through randomization. In AAAI/IAAI. AAAI Press/The MIT Press, 431–437.Google Scholar
Hehenberger, P., Vogel-Heuser, B., Bradley, D., Eynard, B., Tomiyama, T., and Achiche, S. 2016. Design, modelling, simulation and integration of cyber physical systems: Methods and applications. Comput. Ind. 82, 273–289.Google Scholar
Huang, J. 2007. The effect of restarts on the efficiency of clause learning. In IJCAI. 2318–2323.Google Scholar
Kaufmann, B., Leone, N., Perri, S., and Schaub, T. 2016. Grounding and solving in answer set programming. AI Magazine 37, 3, 25–32.Google Scholar
Nadel, A. and Ryvchin, V. 2012. Efficient SAT solving under assumptions. In SAT. 242–255.Google Scholar
Pipatsrisawat, K. and Darwiche, A. 2007. A lightweight component caching scheme for satisfiability solvers. In SAT. 294–299.Google Scholar
Ratasich, D., Khalid, F., Geissler, F., Grosu, R., Shafique, M., and Bartocci, E. 2019. A roadmap toward the resilient internet of things for cyber-physical systems. IEEE Access 7, 13260–13283.Google Scholar
Rossi, D. and Rossini, G. 2012. On sizing CCN content stores by exploiting topological information. In INFOCOM Workshops. IEEE, 280–285.Google Scholar
Silva, J. P. M. and Sakallah, K. A. 1996. Conflict analysis in search algorithms for satisfiability. In ICTAI. IEEE Computer Society, 467–469.Google Scholar
Sutton, R. S. 1995. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In NIPS. MIT Press, 1038–1044.Google Scholar
Sutton, R. S. and Barto, A. G. 2018. Reinforcement Learning: An Introduction, 2nd ed.Google Scholar
Swift, T. and Warren, D. S. 2012. XSB: extending prolog with tabled logic programming. Theory Pract. Log. Program. 12, 1-2, 157–187.Google Scholar