Models for TAMP problems are complex and challenging to develop. The high-dimensional sensory-motor space and the required integration of metric and symbolic state variables augment the challenges. Machine learning addresses these challenges at both the acting level and the planning level. But ML in robotics faces specific problems: lack of massive data; experiments needed for RL are scarce, very expensive, and difficult to reproduce; realistic sensory-motor simulators remain computationally costly; and expert human input for RL, e.g., for specifying or shaping reward functions or giving advices, is scarce and costly. The functions learned tend to be narrow: transfer of learned behaviors and models across environments and tasks is challenging. This chapter presents approaches for learning reactive sensory-motor skills using deep RL algorithms and methods for learning heuristics to guide a TAMP planner avoiding computation on unlikely feasible movements.

Acting with robots and sensory-motor devices demands the combined capabilities of reasoning both on abstract actions and on concrete motion and manipulation steps. In the robotics literature, this is referred to as "task-aware planning," i.e., planning beyond motion and manipulation. In the AI literature, it is referred to as "combined task and motion planning" (TAMP). This class of TAMP problems, which includes task, motion, and manipulation planning, is the topic of this part. The challenge in TAMP is the integration of symbolic models for task planning with metric models for motion and manipulation. This part introduces the representations and techniques for achieving and controlling motion, navigation, and manipulation actions in robotics. It discusses motion and manipulation planning algorithms, and their integration with task planning in TAMP problems. It covers learning for the combined task and motion-manipulation problems.

Acting, planning and learning are critical cognitive functions for an autonomous actor. Other functions, such as perceiving, monitoring, and goal reasoning, are also needed and can be essential in many applications. This chapter briefly surveys a few such functions and their links to acting, planning, and learning. Section 24.1 discusses perceiving and information gathering: how to model and control perception actions in order to recognize the state of the world and detect objects, events, and activities relevant to the actor while performing its tasks. It discusses semantic mapping and anchoring sensor data to symbols. Section 24.2 is about monitoring, that is, detecting and interpreting discrepancies between predictions and observations, anticipating what needs be monitored, and controlling monitoring actions. Goal reasoning in Section 24.3 is about assessing the relevance of current goals, from observed evolutions, failures, and opportunities to achieve a higher-level assigned mission.

AI acting systems, or actors – which may be embodied in physical devices such as robots or in abstract procedures such as web-based service agents – require several cognitive functions, three of which are acting, planning, and learning. Acting is more than just the sensory-motor execution of low-level commands: there is a need to decide how to perform a task, given the context and changes in the environment. Planning involves choosing and organizing actions that can achieve a goal and is done using abstract actions that the agent will need to decide how to perform. Learning is important for acquiring knowledge about expected effects, which actions to perform and how to perform them, and how to plan; and acting and planning can be used to aid learning. This chapter introduces the scientific and technical challenges of developing these three cognitive functions and the ethical challenge of doing such development responsibly.

This chapter delves into the theory and application of reversible Markov Chain Monte Carlo (MCMC) algorithms, focusing on their role in Bayesian inference. It begins with the Metropolis–Hastings algorithm and explores variations such as component-wise updates, and the Metropolis-Adjusted Langevin Algorithm (MALA). The chapter also discusses Hamiltonian Monte Carlo (HMC) and the importance of scaling MCMC methods for high-dimensional models or large datasets. Key challenges in applying reversible MCMC to large-scale problems are addressed, with a focus on computational efficiency and algorithmic adjustments to improve scalability.

In this chapter, we propose different approaches to planning with nondeterministic models. We describe three techniques for planning with nondeterministic state transition systems: And/Or graph search (Section 12.1), planning based on determinization techniques (Section 12.2), and planning via symbolic model checking (Section 12.3). We then present techniques for planning by synthesis of input/output automata (Section 12.4). We finally briefly discuss techniques for behavior tree generation (Section 12.5).

This chapter is about planning with hierarchical refinement methods. A plan guides the acting engine RAE with informed choices about the best methods for the task and context at hand. We consider an optimizing planner to find methods maximizing a utility function. In principle, the planner may rely on an exact dynamic programming optimization procedure. An approximation approach is more adapted to the online guidance of an actor. We describe a Monte Carlo tree search planner, called UPOM, parameterized for rollout depth and number of rollouts. It relies on a heuristic function for estimating the remainder of a rollout when the depth is bounded. UPOM is an anytime planner used in a receding horizon manner. This chapter relies on chapters 8, 9, and 14. It presents refinement planning domains and outlines the approach. Section 15.2 proposes utility functions and an optimization procedure. The planner is developed in Section 15.3.