This part of the book is devoted to acting, planning, and learning with operational models of actions expressed with a hierarchical task-oriented representation. Operational models are valuable for acting. They allow for detailed descriptions of complex actions handling dynamic environments with exogenous events. The representation relies on hierarchical refinement methods that describe alternative ways to handle tasks and react to events. A method can be any complex algorithm, decomposing a task into subtasks and primitive actions. Subtasks are refined recursively. Actions trigger the execution of sensory-motor procedures in closed loops that query and change the world stochastically.

Task and motion planning (TAMP) problems combine abstract causal relations from preconditions to effects with computational geometry, kinematics, and dynamics. This chapter is about the integration of planning for motion/manipulation with planning for abstract actions. It introduces the main sampling-based algorithms for motion planning. Manipulation planning is subsequently introduced. A few approaches specific to TAMP are then presented.

This chapter is about planning approaches with explicit time in the descriptive and operational models of actions, as well as in the models of the expected evolution of the world not caused by the actor. It describes a planning algorithm that handles durative and concurrent activities with respect to a predicted dynamics. Section 17.1 presents a knowledge representation for modeling actions and tasks with temporal variables using temporal refinement methods. Temporal plans and planning problems are defined as chronicles, i.e., collections of assertions and tasks with explicit temporal constraints. A planning algorithm with temporal refinement methods is developed in Section 17.2. The basic techniques for managing temporal and domain constraints are then presented in Section 17.3.

This chapter is about two key aspects of learning with deterministic models: learning heuristics to speed up the search for a solution plan and the automated synthesis of the model itself. We discuss how to learn heuristics for exploring parts of the search space that are more likely to lead to solutions. We then address the problem of how to learn a deterministic model, with a focus on learning action schemas.

The motivations for acting and planning with probabilistic models are about handling uncertainty in a quantitative way, with optimal or near-optimal decisions. The future is never entirely and precisely predictable. Uncertainty can be due to exogenous events in the environment, from nature and other actors, to noisy sensing and information gathering actions, to possible failures and outcomes of imprecise or intrinsically nondeterministic actions. Models are necessarily incomplete. Knowledge about open environments is partial. Part of what may happen can be only be modeled with uncertainty. Even in closed predictable environments, complete deterministic models may be too complex to develop. The three chapters in Part III tackle acting, planning, and learning in a probabilistic setting.

This chapter is about planning techniques for solving MDP problems. It presents algorithms that seeks optimal or near-optimal solution policies for a domain. Most of the chapter is focused on indefinite-horizon goal reachability domains that have positive costs and a safe solution; they may have dead ends, but those are avoidable. The chapter presents dynamic programming algorithms, heuristics search methods and their heuristics, linear programming methods, and online and Monte Carlo tree search techniques.

In probabilistic models, an action can have several possible outcomes that are not equally likely; their distribution can be estimated relying on statistics of past observations. The purpose is to act optimally with respect to an optimization criterion of the estimated likelihood of action effects and their cost. The usual formal probabilistic models are Markov decision processes (MDPs). An MDP is a nondeterministic state-transition system with a probability distribution and a cost distribution. The probability distribution defines how likely it is to get to a state 𝑠′ when an action 𝑎 is performed in a state 𝑠. The chapter presents MDPs in flat then structured state-space representations. Section 8.3 covers modeling issues of a probabilistic domain with MDPs and variants such as the stochastic shortest path model (SSP) or the constrained MDP (C-MDP) model. Section 8.4 focuses on acting with MDPs. Partially observable MDPs and other extended models are discussed in Section 8.5.

This chapter is about a refinement acting engine (RAE) used on a hierarchical task-oriented representation. It relies on an expressive, general-purpose language that offers rich programming control structures for online decision-making. A collection of refinement methods describes alternative ways to handle tasks and react to events. A method can be any complex algorithm, decomposing a task into subtasks and primitive actions. Subtasks are refined recursively. Nondeterministic actions trigger sensory-motor procedures that query and change the world nondeterministically. We assume that the methods are manually specified and that RAE chooses the appropriate method for the task and context at hand heuristically.

The recent developments of large language models (LLMs) and their extension in multimodal foundation models have introduced new perspectives in AI. An LLM is basically a very large neural net trained as a statistical predictor of the likely continuation of a sequence of words. LLMs have excellent competencies over a broad set of NLP tasks. Additionally, LLMs demonstrate the emergence of deliberation capabilities for reasoning, common sense, problem solving, code writing, and planning. These abilities have not been designed for in LLMs. They are unexpected and remain to a large extent poorly understood. Although error-prone and imperfect, they open up promising perspectives for acting, planning, and learning, which are presented in this chapter.

This chapter is about domain-independent classical-planning algorithms, which until recently were the most widely studied class of AI planning algorithms. The chapter classifies and describes a variety of forward search, backward search, and plan-space planning algorithms, as well as heuristics for guiding the algorithms.

This chapter sets the foundation for the next two chapters. It introduces the reader to robotics platforms for the development of acting, planning, and learning functions. The study of motion is based on classical mechanics for the modeling of forces and their effects on mouvements. Robotics builds on this knowledge to master computational motion, navigation, and manipulation over different types of devices and environments. Robotic devices are informally introduced in the following section. Motion problems and the metric representations with continuous state variables needed for geometric, kinematic, and dynamic operational models are then presented. Section 20.3 introduces localization and navigation problems, followed by a section on manipulation problems and their representations.

This chapter is about representing HTN planning domains and solving HTN planning problems. Several of the formal definitions require the same "classical planning" restrictions as in Part I, but most practical HTN implementations loosen or drop several of these restrictions. We first discuss ways to represent and solve planning problems in which there is a totally ordered sequence of tasks to accomplish. We then generalize to allow partially ordered tasks and describe ways to combine classical planning and HTN planning. Finally, we briefly discuss heuristic functions, expressivity, and computational complexity.

The chapters in Part II are about algorithms for planning, acting, and learning using hierarchical task networks (HTNs). HTNs can describe ways to perform complex tasks without the overhead of searching through a large state space, how to avoid situations where unanticipated events are likely to cause bad outcomes, and how to recover when unanticipated events occur.

This chapter discusses several ways for actors to use HTN domain models. These include a way to use HTN methods for purely reactive acting, some simple ways for an actor to make use of an HTN planner, and some ways to repair HTN plans when unexpected events occur during acting.