Skip to main content

What you always wanted to know about the deterministic part of the International Planning Competition (IPC) 2014 (but were too afraid to ask)

  • Mauro Vallati (a1), Lukáš Chrpa (a2) (a3) and Thomas L. Mccluskey (a4)

The International Planning Competition (IPC) is a prominent event of the artificial intelligence planning community that has been organized since 1998; it aims at fostering the development and comparison of planning approaches, assessing the state-of-the-art in planning and identifying new challenging benchmarks. IPC has a strong impact also outside the planning community, by providing a large number of ready-to-use planning engines and testing pioneering applications of planning techniques.

This paper focusses on the deterministic part of IPC 2014, and describes format, participants, benchmarks as well as a thorough analysis of the results. Generally, results of the competition indicates some significant progress, but they also highlight issues and challenges that the planning community will have to face in the future.

Hide All
Alcázar, V., Veloso, M. M. & Borrajo, D. 2011. Adapting a rapidly-exploring random tree for automated planning. In Proceedings of the Fourth Annual Symposium on Combinatorial Search, SOCS.
Baier, J. A. & McIlraith, S. A. 2008. Planning with preferences. AI Magazine 29(4), 2536.
Balint, A., Belov, A., Diepold, D., Gerber, S., Järvisalo, M. & Sinz, C. 2012. SAT challenge 2012.
Balint, A., Belov, A., Heule, M. J. & Järvisalo, M. 2013. SAT competition 2013.
Belov, A., Diepold, D., Heule, M. J. & Järvisalo, M. 2014. SAT competition 2014.
Bylander, T. 1996. A probabilistic analysis of prepositional strips planning. Artificial Intelligence 81(1), 241271.
Calimeri, F., Gebser, M., Maratea, M. & Ricca, F. 2016. Design and results of the fifth answer set programming competition. Artificial Intelligence 231, 151181.
Cenamor, I., De La Rosa, T. & Fernández, F. 2012. Mining IPC-2011 results. In WS-IPC 2012.
Chen, Y., Wah, B. W. & Hsu, C.-W. 2006. Temporal planning using subgoal partitioning and resolution in sgplan. Journal of Artificial Intelligence Research 26, 323369.
Chrpa, L., McCluskey, T., Vallati, M. & Vaquero, T. 2017. The fifth international competition on knowledge engineering for planning and scheduling: summary and trends. AI Magazine 38(1), 104106.
Domshlak, C., Hoffmann, J. & Katz, M. 2015. Red-black planning: a new systematic approach to partial delete relaxation. Artificial Intelligence 221, 73114.
Dréo, J., Savéant, P., Schoenauer, M. & Vidal, V. 2011. Divide-and-evolve: the marriage of Descartes and Darwin. In Proceedings of the 7th International Planning Competition (IPC).
Edelkamp, S., Kissmann, P. & Torralba, Á. 2015. Bdds strike back (in AI planning). In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.
Fawcett, C., Vallati, M., Hutter, F., Hoffmann, J., Hoos, H. H. & Leyton-Brown, K. 2014. Improved features for runtime prediction of domain-independent planners. In Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling, ICAPS.
Fox, M. & Long, D. 2003. PDDL2.1: an extension to PDDL for expressing temporal planning domains. Journal of Artificial Intelligence Research 20, 61124.
Gerevini, A., Saetti, A. & Serina, I. 2003. Planning through stochastic local search and temporal action graphs. Journal of Artificial Intelligence Research 20, 239290.
Gerevini, A., Saetti, A. & Vallati, M. 2014. Planning through automatic portfolio configuration: the pbp approach. Journal of Artificial Intelligence Research 50, 639696.
Gerevini, A. E., Haslum, P., Long, D., Saetti, A. & Dimopoulos, Y. 2009. Deterministic planning in the fifth international planning competition: Pddl3 and experimental evaluation of the planners. Artificial Intelligence 173(5), 619668.
Gulić, M., Olivares, R. & Borrajo, D. 2016. Using automated planning for traffic signals control. PROMET-Traffic&Transportation 28(4), 383391.
Haslum, P. 2011. Computing genome edit distances using domain-independent planning. In ICAPS'11 Scheduling and Planning Applications Workshop (SPARK).
Helmert, M. 2003. Complexity results for standard benchmark domains in planning. Artificial Intelligence 143(2), 219262.
Helmert, M. 2006. The fast downward planning system. Journal of Artificial Intelligence Research 26, 191246.
Helmert, M. & Domshlak, C. 2011. LM-cut: optimal planning with the landmark-cut heuristic. In IPC 2011 Planner Abstracts.
Helmert, M. & Röger, G. 2008. How good is almost perfect? In Proceedings of AAAI, 944–949.
Helmert, M., Röger, G. & Karpas, E. 2011. Fast downward stone soup: a baseline for building planner portfolios. In ICAPS 2011 Workshop on Planning and Learning, 28–35.
Hoffmann, J. 2003. The Metric-FF planning system: translating ‘ignoring delete lists’ to numeric state variables. Journal Artificial Intelligence Research 20, 291341.
Hoffmann, J. 2011. Analyzing search topology without running any search: on the connection between causal graphs and h+. Journal of Artificial Intelligence Research 41, 155229.
Hoffmann, J., Edelkamp, S., Thiébaux, S., Englert, R., dos, F., Liporace, S. & Trüg, S. 2006. Engineering benchmarks for planning: the domains used in the deterministic part of IPC-4. Journal of Artificial Intelligence Research 26, 453541.
Hoffmann, J. & Nebel, B. 2001. The FF planning system: fast plan generation through heuristic search. Journal of Artificial Intelligence Research 14, 253302.
Howe, A. E. & Dahlman, E. 2002. A critical assessment of benchmark comparison in planning. Journal of Artificial Intelligence Research 17(1), 133.
Howey, R., Long, D. & Fox, M. 2004. Val: automatic plan validation, continuous effects and mixed initiative planning using PDDL. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI-04), 294–301. IEEE.
Hurley, B. & O'Sullivan, B. 2015. Statistical regimes and runtime prediction. In International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press.
Kanefsky, B. & Taylor, W. 1991. Where the really hard problems are. In Proceedings of IJCAI 91, 163–169.
Lipovetzky, N. & Geffner, H. 2011. Searching for plans with carefully designed probes. In Proceedings of ICAPS.
Long, D. & Fox, M. 2003. The 3rd international planning competition: results and analysis. Journal of Artificial Intelligence Research 20, 159.
López, C. L., Celorrio, S. J. & Olaya, Á. G. 2015. The deterministic part of the seventh international planning competition. Artificial Intelligence 223, 82119.
Matloob, R. & Soutchanski, M. 2016. Exploring organic synthesis with state-of-the-art planning techniques. In Proceedings of ICAPS ’16 Scheduling and Planning Applications Workshop (SPARK).
Nakhost, H. & Müller, M. 2010. Action elimination and plan neighborhood graph search: two algorithms for plan improvement. In The Twentieth International Conference on Automated Planning and Scheduling (ICAPS), 121–128.
Nissim, R., Hoffmann, J. & Helmert, M. 2011. The merge-and-shrink planner: bisimulation-based abstraction for optimal planning. In IPC 2011 Planner Abstracts, 106–107.
Núñez, S., Borrajo, D. & López, C. L. 2015. Automatic construction of optimal static sequential portfolios for AI planning and beyond. Artificial Intelligence 226, 75101.
Ramirez, M., Lipovetzky, N. & Muise, C. 2014. Lightweight automated planning toolkit. Technical report, http://lapkt. org.
Richter, S. & Westphal, M. 2010. The LAMA planner: guiding cost-based anytime planning with landmarks. Journal Artificial Intelligence Research 39, 127177.
Richter, S., Westphal, M. & Helmert, M. 2011. Lama 2008 and 2011. In International Planning Competition, 117–124.
Rintanen, J. 2004. Phase transitions in classical planning: an experimental study. In ICAPS 2004, 101–110.
Rintanen, J. 2012. Engineering efficient planners with SAT. In Proceedings of ECAI, 684–689.
Rizzini, M., Fawcett, C., Vallati, M., Gerevini, A. E. & Hoos, H. 2015. Portfolio methods for optimal planning: an empirical analysis. In Proceedings of the IEEE International Conference on Tools with Artificial Intelligence (ICTAI-15). IEEE.
Rossi, F., Van Beek, P. & Walsh, T. 2006. Handbook of constraint programming. Elsevier.
Thimm, M., Villata, S., Cerutti, F., Oren, N., Strass, H. & Vallati, M. 2016. Summary report of the first international competition on computational models of argumentation. AI Magazine 37, 102.
Torralba, A. & Alcázar, V. 2013. Constrained symbolic search: on mutexes, BDD minimization and more. In Sixth Annual Symposium on Combinatorial Search (SoCS).
Torralba, Á., Edelkamp, S. & Kissmann, P. 2013. Transition trees for cost-optimal symbolic planning. In Twenty-Third International Conference on Automated Planning and Scheduling (ICAPS).
Valenzano, R. A., Nakhost, H., Müller, M., Schaeffer, J. & Sturtevant, N. R. 2012. Arvandherd: parallel planning with a portfolio. In ECAI, 786–791.
Vallati, M., Chrpa, L., Grzes, M., McCluskey, T., Roberts, M. & Sanner, S. 2015a. The 2014 international planning competition: progress and trends. AI Magazine 36, 9098.
Vallati, M., Chrpa, L. & Kitchin, D. E. 2015b. Portfolio-Based Planning: State of the Art, Common Practice and Open Challenges. AI Communications 28, 717733.
Vallati, M., Chrpa, L. & McCluskey, T. L. 2014. The 2014 IPC: description of participating planners of the deterministic track.
Vallati, M., Hutter, F., Chrpa, L. & McCluskey, T. L. 2015c. On the effective configuration of planning domain models. In International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press.
Vallati, M. & Vaquero, T. 2015. Towards a protocol for benchmark selection in IPC. In The 4th Workshop of the International Planning Competition.
Wilcoxon, F. & Wilcox, R. A. 1964. Some Rapid Approximate Statistical Procedures. American Cyanamid Co.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The Knowledge Engineering Review
  • ISSN: 0269-8889
  • EISSN: 1469-8005
  • URL: /core/journals/knowledge-engineering-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed