Contents
\newmdenv

[tikzsetting=draw=lightgray,fill=lightgray,fill opacity=0.2]highlights

Corso di Dottorato di Ricerca in Informatica e Automazione

[.5cm] xxix ciclo

Timeline-based Planning and Execution with Uncertainty:

[.1cm] Theory, Modeling Methodologies and Practice

Dottorando:   Alessandro Umbrico __________________ Docenti guida:   Prof.ssa Marta Cialdea Mayer __________________   Dr. Andrea Orlandini __________________ Coordinatore:   Prof. Stefano Panzieri __________________

\setstretch

1.2

{abstracts}

Automated Planning is one of the main research field of Artificial Intelligence since its beginnings. Research in Automated Planning aims at developing general reasoners (i.e. planners) capable of automatically solve complex problems. Broadly speaking, planners rely on a general model characterizing the possible states of the world and the actions that can be performed in order to change the status of the world. Given a model and an initial known state, the objective of a planner is to synthesize a set of actions needed to achieve a particular goal state. The classical approach to planning roughly corresponds to the description given above. However, many planning techniques have been introduced in the literature relying on different formalisms and making different assumptions on the features of the model of the world. The timeline-based approach is a particular planning paradigm capable of integrating causal and temporal reasoning within a unified solving process. This approach has been successfully applied in many real-world scenarios although a common interpretation of the related planning concepts is missing. Indeed, there are significant differences among the existing frameworks that apply this technique. Each framework relies on its own interpretation of timeline-based planning and therefore it is not easy to compare these systems. Thus, the objective of this work is to investigate the timeline-based approach to planning by addressing several aspects ranging from the semantics of the related planning concepts to the modeling and solving techniques. Specifically, the main contributions of this PhD work consist of: (i) the proposal of a formal characterization of the timeline-based approach capable of dealing with temporal uncertainty; (ii) the proposal of a hierarchical modeling and solving approach; (iii) the development of a general purpose framework for planning and execution with timelines; (iv) the validation of this approach in real-world manufacturing scenarios.

Acknowledgements.
I would like to express my sincere gratitude to my advisors Prof. Marta Cialdea Mayer and Dr. Andrea Orlandini for their continuous support of my Ph.D study and related research, for their patience, motivation and knowledge. Their guidance helped me in all the time of research and writing of this thesis. I could not have imagined having better advisors and mentors for my Ph.D study. My sincere thanks also goes to Prof. Joachim Hertzberg from Osnabruck University who provided me the opportunity to join his team as an intern. It has been a great professional and personal experience for me. I thank my fellow labmates and all my friends in the Institute of Cognitive Science and Technology of National Research Council of Italy. Without their precious support, insightful comments and encouragement it would not be possible to conduct this research. In particular, I am grateful to Dr. Amedeo Cesta, for enlightening me the first glance of research, and to Dr. Stefano Borgo, for introducing me into the world of ontologies. Last but not least, I would like to thank my parents and my wife for giving me all the love that I need.

Chapter \thechapter Introduction

"Maybe the only significant difference between a really smart simulation and a human being was the noise they made when you punched them."

- Terry Pratchett, The Long Earth

\lettrine

[lines=2]Artificial Intelligence (AI) is the field of Computer Science that deals with the development of techniques that aim at endowing machines with some sort of intelligence. There are different research fields in AI that characterize intelligence in different ways and realize different types of intelligent machines accordingly. Broadly speaking, "the term artificial intelligence is applied when a machine mimics cognitive functions that humans associate with other human minds, such as learning and problem solving" as stated in [Russell and Norvig, 2003]. Automated Planning is one of the core fields of AI since its beginnings. Its research objective is to endow a machine (an artificial agent) with the capability of autonomously carry out complex tasks. From a practical point of view, this is a key enabling feature in application scenarios where direct human involvement is neither possible nor safe, e.g. space mission or deep sea exploration. Moreover, the recent and continuous improvements of robotic platforms with respect to reliability and efficiency represent a great opportunity for deploying AI-based techniques in even more common application scenarios (e.g. domestic care, manufacturing, rescue missions).

A planner is a general problem solver able to automatically synthesize a set of actions that allow an agent to achieve some objectives (e.g. explore and gather scientific data about an unknown environment or accomplish some complex task within the production process of a factory). A planning system usually relies on a model which represents a general description of the world. The model characterizes the environment the agent is supposed to operate in, and the agent capabilities in terms of the actions the agent can perform to interact with the environment. The classical approach to planning relies on a logical characterization of the model that focuses on the causal aspects of the problem to solve. States consist of sets of atoms asserting known facts and properties about the world, e.g. the position of a robot or an object in the environment. Actions encode transitions between states by specifying preconditions and effects. Preconditions specify a set of conditions that must be true (i.e. atoms) in order to apply the action in a particular state. Effects specify conditions that become true (i.e. positive effects) or false (i.e. negative effects) after the application of the action. For example, the action of moving an object from an initial location to a destination location can be applied to all states in which the object to move is located at initial location. The states resulting from the application of the action are all those states in which the object is located at the destination location. A planning goal usually consists of a logical formula representing the goal state to achieve, e.g. the state in which all objects of the domain are in a desired location. Thus, given an initial state containing a set of known facts about the world (e.g. the initial locations of the objects), the planning system must synthesize a set of actions needed to reach the goal state.

STRIPS [Fikes and Nilsson, 1971] is one of the first planning system introduced into the literature whose language inspired the PDDL [Mcdermott et al., 1998] which is the standard modeling language for planning. Many planning systems have been developed, e.g. SATplan [Kautz and Selman, 1992], FF [Hoffmann and Nebel, 2011] or LPG [Gerevini and Serina, 2002], that rely on the classical planning formalism and PDDL language. These planners have also shown relevant solving capabilities on toy problems during the International Planning Competitions. However, from a practical point of view, the classical planning formalism makes strong assumptions on the features of the problems to model. These assumptions limit the capabilities of classical planners to address real-world problems. For example, classical planning paradigms use an implicit representation of time which does not allow planners to deal with concurrency, temporal constraints or durative actions that are crucial in real-world scenarios. Consequently, several extensions to classical planning have been introduced into the literature in order to overcome these limitations and address more realistic problems. These extensions lead to the definition of several planning paradigms that relax different assumptions of the classical approach. Temporal planning represents the class of planning paradigms that introduce an explicit representation of time into the modeling language.

The timeline-based approach is a particular temporal planning paradigm introduced in early 90’s with HSTS [Muscettola, 1994] which has been successfully applied to solve many real-world problems (in space-like contexts mainly). This approach takes inspiration from classical control theory and is characterized by a more practical than logical view of planning. The timeline-based approach focuses on the temporal behavior of a system and the related features that must be controlled. Specifically, a complex system (e.g. an exploration rover) is modeled by identifying a set of relevant features that must be controlled over time within a known temporal horizon (e.g. the wheeled base of the robot or the communication facility). The control process consists in the synthesis of a set of temporal behaviors (i.e. the timelines) that describe how the modeled features evolve over time. The main advantage of planning with timelines consists in the representation approach, which allows the planner to deal with time and temporal constraints while building a plan. Namely, the timeline-based representation fosters a hybrid solving procedure by means of which it is possible to integrate planning and scheduling in a unified reasoning mechanism. In general, hybrid reasoning is essential to effectively address real world problems. Indeed, the key factor influencing the successful application of planning technologies to real-world problems is the capability of simultaneously dealing with different aspects of the problem like causality, time, resources, concurrency or uncertainty at solving time.

Despite the practical success of timeline-based approach, formal frameworks characterizing this formalism have been proposed only recently. There are several planning systems that have been introduced into the literature, e.g. EUROPA [Barreiro et al., 2012], IxTeT [Ghallab and Laruelle, 1994], APSI-TRF [Fratini et al., 2011], each of which applies its own interpretation of timeline-based planning. Moreover, developed Planning and Scheduling (P&S) applications are strictly connected to the specific context they have been designed for. Thus, existing timeline-based applications are hard to adapt to different problems. In general, there is a lack of methodology in modeling and solving timeline-based problems. Given the elements that compose a particular domain, it is not easy to design a suited model in order to ralize effettive P&S solutions. In addition, different systems apply different solving approaches and generate plans with different features. Thus, it is not simple to compare different timeline-based systems and it is even more difficult to compare such systems with other existing approaches.

Contribution

The objective of this work is to investigate timeline-based planning by taking into account several aspects ranging from the semantics of the main planning concepts to the modeling and solving approach. Thus the contribution of the work involves (i) the proposal of a formal characterization of the timeline-based approach which takes into account also temporal uncertainty, (ii) the proposal of a hierarchical modeling and solving approach, (iii) the development of a general-purpose framework for planning and execution with timelines (EPSL - Extensible Planning and Scheduling Library), which complies with the proposed formalization and implements the proposed hierarchical solving procedure and lastly (iv) the validation of EPSL and the envisaged approach to timeline-based planning in real-world manufacturing scenarios.

The proposed formalization defines a clear semantics of concepts like timelines, timeline-based plans and state variables, representing the basic building blocks of a planning domain. In particular, the formalization takes into account the controllability properties in order to model the temporal uncertainty concerning the uncontrollable features of a domain. This is particularly relevant for real-world problems, where not all the features of the domain are controllable with respect to the point of view of the artificial agent. Namely, the environment has uncontrollable dynamics that may affect the behavior of the system to control and the outcome of its operations (e.g. the visibility of the ground station for the communication operations of a satellite). The timeline-based plans, generated according to the proposed formalization, contain information about the uncertainty of the domain that can be analyzed to characterize the robustness of the plan with respect to its execution. There are several works in this field [Vidal and Fargier, 1999, Morris et al., 2001, Cesta et al., 2010] aiming at analyzing the plan in order to understand if, given the possible evolutions of the uncontrollable features of the domain, it is possible to complete the execution of the plan. With respect to planning, it is important to leverage the controllability information about the domain during the solving process in order to generate plans with some desired controllability properties (if possible) and therefore, have some information regarding their executability.

Given an agent to control, the proposed modeling approach follows a hierarchical specification of the domain which is similar to HTN planing [Georgievski and Aiello, 2015]. Specifically the approach proposes a functional characterization of the agent at different levels of abstraction. Broadly speaking, a primitive level characterizes the functional behavior of the physical/logical elements composing the agent in terms of commands they can directly manage over time. Functional levels model complex functions/operations the agent could perform over time by leveraging its components. Namely, functional levels model complex activities (i.e. complex tasks) the agent can perform by combining the available commands (i.e. primitive tasks). Domain rules, like methods in HTN planning, describe the operational constraints that allow the agent to implement tasks. They specify hierarchical decomposition of complex tasks in sets of constraints between primitive tasks. The resulting hierarchical structure encodes specific knowledge about the domain that the planning system can leverage during solving. Specifically, this work introduces search heuristics that leverages the hierarchical structure of the planning domain to support the plan generation process.

The EPSL framework complies with both the formalization and hierarchical modeling/solving approaches presented. EPSL is the major result of this study. It realizes a uniform framework for planning and execution with timelines under uncertainty. From the planning point of view, EPSL implements a hierarchical solving approach which is capable of dealing with temporal uncertainty during plan generation. Specifically, the solving procedure leverages information about the temporal uncertainty of the planning domain in order to generate plans with some properties characterizing their robustness with respect to the execution in the real-world. From the execution point of view, EPSL executes the timeline of a plan by taking into account the controllability properties of the related values and adapting the plans to the unexpected behaviors of the environment. EPSL planning and execution capabilities have been successfully applied to real-world manufacturing scenarios showing the effectiveness of the proposed approach.

Outline

Chapter id1 provides a brief description of the background of Automated Planning in AI by describing the classical approaches to planning, the limit of these approaches in solving real-world problems and how they have been improved in order to address more realistic problems. Chapter id1 provides a more detailed overview of the timeline-based planning approach and the related state of the art prior to this study. In particular, this chapter describes some of the most relevant timeline-based systems introduced into the literature (EUROPA, IxTeT and APSI-TRF) together with a brief description of the temporal formalisms this kind of systems usually relies on. Chapter id1 enters into the details of the contribution of the study by describing the proposed formalization of the timeline-based approach and the related controllability problem. Chapter id1 describes EPSL its structure and the implemented hierarchical modeling and solving approach. Chapter id1 presents a relevant extension of EPSL that allows the framework to execute plans while managing temporal uncertainty. This chapter also describe the deployment of EPSL to an interesting real-world manufacturing scenario of Human-Robot Collaboration (HRC). In particular, HRC applications represent well-suited contexts to leverage the EPSL capabilities of dealing with temporal uncertainty at planning and execution time. Chapter id1 presents another interesting application of the EPSL framework and its integration with semantic technologies for realizing an extended plan-based control architecture. Specifically, the chapter presents a flexible control architecture, called KBCL (Knowledge-based Control Loop), which has been applied to a real-world scenario for controlling reconfigurable manufacturing systems. KBCL aims at realizing a flexible control process able to dynamically adapt the control model to the different situations that may affect the capabilities of the system. KBCL investigates the integration and the correlations of ontological analysis and knowledge processing with the timeline-based planning approach. Finally chapter id1 draws some conclusions by providing an assessment of the achieved results and illustrates some of the most relevant open points that must be addressed in the near future.

Chapter \thechapter Planning in Artificial Intelligence

\lettrine

[lines=2]Planning is one of the most relevant research field of AI since its beginnings. The objective of a planning system is to automatically solve a problem by synthesizing a set of operations (i.e. a plan) needed to reach a desired goal (i.e. a desired state or configuration). There are many practical field like robotics or manufacturing where planning technologies have provided a significant contribution. Let us consider, for example, planetary exploration rovers that must operate in a context where direct human control is not possible. In such a context, planning technologies provide the rover with the autonomy needed for navigating an unknown environment and gathering scientific data to communicate.

There are different ways to describe the fundamental elements of a planning system. Such differences have lead to different planning paradigms ranging from those addressing fully observable, deterministic, static and discrete environments, to those that deal with partially observable stochastic environments. This chapter does not aim at presenting a complete background on all the planning technologies and systems that have been introduced into the literature. Thus, after a brief overview of some classical approaches to planning in section 1, section 2 explains the limits of these planning paradigms and the improvements needed to address real-world problems. Finally, section 3 focuses on a particular class of planning paradigms (i.e. Temporal Planning) which extends the classical approach by introducing an explicit representation of time.

1 Classical Planning

Broadly speaking, a planning system is a general problem solver whose aim is to synthesize a set of operations that, given an initial state, allow the system to reach a desired goal state. The reasoning process relies on a model which represents a general description of the problem to solve. The model provides a representation of the environment in terms of the possible states of the world and the actions the system can perform to interact with the environment. Thus, a planning process starts from an initial state and iteratively moves to other states by applying the available actions until a desired goal state is reached.

An example of a simple planning problem is represented by the Vacuum World problem described in [Russell and Norvig, 2003]. The problem consists of a set of rooms that can be either clean or dirty, and a vacuum cleaner which can move between (adjacent) rooms and clean the room the vacuum is located in. In this regard, a state of the world describes the set of rooms that compose the environment, their connections (i.e. whether two rooms are adjacent or not), their states (i.e. whether the rooms are clean or not), and the current room of the vacuum cleaner. The goal state is the state of the world where all rooms are clean. The initial state describes the status of all the rooms and the particular room the vacuum cleaner is initially located in.

Figure 1 shows the state space for the Vacuum World problem with two adjacent rooms. The state space can be seen as a directed graph where the possible states of the world are the nodes and actions are the (directed) edges connecting two states of the world. Let us consider, for example, the state in Figure 1 where both the rooms are dirty and the cleaner is located in the room on the left. The execution of action R (i.e. move right) leads to the state where both the rooms are dirty and the cleaner is located in the room on the right. Similarly, the execution of action S (i.e. clean/suck) leads to the state where the cleaner has not changed its position, the room on the left is clean and the room on the right is dirty.

Given a state space like the one depicted in Figure 1 and a known initial state, the planning process must find a sequence of actions needed to reach the state where both rooms are clean. The Vacuum World problem described above is very simple because states are fully observable (e.g. it is always possible to know whether a room is clean or dirty), actions are deterministic (i.e. there is not uncertainty about the effects of actions) and the search space is small. Thus, the planning process must simply find a path on the graph (i.e. the search space) connecting the initial state with the goal state. However, planning problems are not always fully observable or deterministic and typically entail huge search spaces that cannot be explicitly represented. A more compact and expressive representation/description of planning problems is needed and therefore several modeling languages and planning paradigms have been introduced into the literature.

1.1 Strips

STRIPS (STanford Research Institute Problem Solver) [Fikes and Nilsson, 1971] is one of the first automated planner and language used in AI. The STRIPS modeling language has represented the basic formalism for many planning paradigms that have been introduced successively. The formalism relies on the first-order predicate calculus to represent the space of world models the planning system must search in order to find a particular world model (i.e. a state), where a desired goal formula is achieved. A world model consists of a set of clauses, i.e. formulas of first-order predicate logic that describe a particular situation concerning the environment and the agent. For example, considering a robotic planning problem the related world models will contain a set of formulas concerning the position of the robot and all objects of the environment. Operators are particular transition functions that allow the planning system to move from one world model to others. It is supposed that for each world model there exists at least one operator which could be applied to "transform" the related world model into another. Thus the resulting problem solver must find the appropriate composition of operators that transform an initial world model to a "final" world model which satisfies a goal condition (i.e. a particular logical formula).

The problem space of a STRIPS problem is defined by the initial world model, the set of available operators and the goal states. Operators are grouped by schema which models a set of instances of applicable operators. Let us consider for example the operator goto, used for moving a robot between two points on a floor. In such a case, there is a distinct operator for each pair of points of the floor. Therefore it is more convenient to group all these possible instances into an operator schema goto(m, n) parametrized by the initial and final positions (m and n respectively). Specifically, an operator schema describes the effects and the conditions under which the operator is applicable. Effects specify the list of formulas that must be added to the model (the add list) and a list of formulas that must be removed (the delete list). Let us consider the example described in [Fikes and Nilsson, 1971] concerning a operator push(k, m, n) which models the action of pushing an object k from m to n. Such an operation can be modeled by the code below where ATR(m) is a predicate stating that the robot is at location m, and AT(k, m) is a predicate stating that the object k is at location m.

{highlights}
push(k, m,n)
precondition:   ATR(m) $$\land$$ AT(k, m)
delete list:   ATR(m), AT(k, m)


1.2 Pddl

The Problem Domain Description Language (PDDL) is an action-based language introduced in [Mcdermott et al., 1998] for the AIPS-98 planning competition. PDDL relies on the STRIPS formalism and aims at defining a standard syntax for expressing planning domains. An early design decision was to separate the description of parametrized actions of the domain from the description of the objects, initial conditions and goals that characterize problem instances. Thus the domain description defines the general rules and behaviors that characterize as specific application scenario/context. Given a domain description, a problem description instantiates a planning problem in terms of specific type and number of objects, initial conditions and goals. In this way, a particular domain description can be used to define many different problem descriptions. PDDL defines parametrized actions by using variables denoting elements of a particular problem instance. Indeed, variables are instantiated to objects of the specific problem description when actions are grounded for applications. Preconditions and effects of actions are logical propositions constructed from predicates, arguments (i.e. objects from a problem instance) and logical connectivities. Moreover, PDDL extends the expressive power of STRIPS formalism by including the ability to descirbe structured object types, specify types for action parameters, specify actions with negative preconditions and conditional effects, as well as introduce the use of quantification in expressing both pre- and post- conditions. The code below shows an example of a simple PDDL action which allows a rover to move between two locations.

{highlights}
(: move
:parameters (?r - rover ?from ?to - location)
:precondition (and (at ?r ?from)
(path ?from ?to))
:effect (and (not (at ?r ?from))
(at ?r ?to))
)


The action has one parameter denoting the particular rover which is moving, and two other parameters denoting the specific locations the rover moves from and to. Action preconditions specify the conditions that must hold to apply actions. An instance of the action can be applied if the rover, the action refers to, is at the starting location (i.e. the location denoted by variable ) and there exists a path connecting the starting location with the destination (i.e. the location denoted by variable ). Action effects specify the state resulting from the application of the action. Thus, once the action has been applied, the rover denoted by variable , is no longer at location (negative effect) but is at location denoted by the variable . Note that no temporal information is associated with action descriptions. Therefore, effects of actions become valid (i.e. true) as soon as actions are applied. Namely, actions in PDDL are instantaneous and there is not an explicit representation of time.

1.3 Htn

{highlights}
(:method
(transport-person ?p ?c2)
; precondition
(and (at ?p ?c1)
(aircraft ?a)
(at ?a ?c3)
(different ?c1 ?c3))
(:ordered
(move-aircraft ?a ?c1)
(board ?p ?a ?c1)
(move-aircraft ?a ?c2)
(debark ?p ?a ?c2))
)


The block of code above shows an simple example of a SHOP2 method defined in [Nau et al., 2003], for a simplified versione of the ZenoTravel domain of the AIPS-2002 Planning Competition. The method describes how to transport a person by aircraft from a location to another location in the case that the aircraft is not located at . The keyword concerns task decomposition and specifies the order the planner must follow to expand subtasks. Thus, first the aircraft moves to location , then the aircraft boards the person , then the aircraft moves to location and finally the aircraft debarks the person .

2 Planning in the Real-World

The modeling features of classical planning approaches rely on a set of assumptions that make strong simplifications of the problems to address with respect to real-world scenarios. Indeed, classical planning mainly deals with static, fully observable and deterministic domains. It means that given any state of the environment and a particular action, it is possible to know exactly which is the next state of the system. Such an assumption does not hold in real-world contexts where the environment may be partially observable and something may be either unknown or unpredictable. In such a case the planning system should be able to handle the uncertainty of the domain and find a sequence of actions that still reach the goal state.

Let us consider again for example, the Vacuum World domain of Figure 1, where the environment described is fully observable. At any state it is possible to know where the vacuum cleaner is located or it is possible to know exactly whether a room is clean or dirty. Similarly, the actions of the vacuum cleaner are deterministic and therefore, the state resulting from the application of an action is known. Let us consider, for example, the state where both rooms are dirty and the vacuum cleaner is located in the left room. If the the Suck operation is applied to this state, then the (only) successor state is the one with left room clean, the right room dirty and the vacuum cleaner still located in the left room. Such a simple problem can be made more "realistic" and more challenging if one or more assumptions are removed. Let us suppose to remove the assumption about the full observability of the environment and that it is not possible to know whether the rooms are clean or dirty. In such a case, it is necessary to find a sequence of actions that, independently from the actual state of the rooms, allows the system to reach a state where certainly both rooms are clean. Moreover, classical planning approaches have an implicit representation of time. Actions are supposed to be instantaneous, which means that the effects of an action become true as soon as the action is applied. States and/or goals are not supposed to have a temporal extension such that they hold only for a limited temporal interval, or that they must be achieved within a known temporal bound. Again this is a significant simplification in real-world contexts where time, temporal constraints (e.g. deadlines for goal achievement) and concurrency (e.g. a limit on the number of jobs that a machine can perform simultaneously) represent strong requirements that must be satisfied by plans.

There are several PDDL-based planning systems e.g. SATplan [Kautz and Selman, 1992], FF [Hoffmann and Nebel, 2011], LPG [Gerevini and Serina, 2002] or LAMA [Richter and Westphal, 2010], that have shown excellent solving capabilities during the International Planning Competitions. However, all the assumptions described above limit the expressivity of classical planning systems and their efficacy to address real-world problems. Consequently several planning approaches have been developed, with the intention of overcoming these limitations by removing one or more of the simplifying assumptions described above. In particular, this work focuses on Temporal Planning which represents the "class" of planning approaches capable of representing information and constraints that concern the temporal features of the domain. These kind of systems realizes problem solvers that make both planning and scheduling decisions during the solving process. Timeline-based planning belongs to this class of planning techniques and it will be further discussed in the next chapter. The following sections provide a brief description of the key modeling features of Temporal Planning, a brief description of PDDL2.1 [Fox and Long, 2003], the temporal extension of PDDL, and other hybrid approaches that present some common features with timeline-based planning like ANML [Smith et al., 2008], FAPE [Dvorák et al., 2014] and CHIMP [Stock et al., 2015].

3 Temporal Planning

The primary distinct characteristic of temporal planning paradigms is that they synthesize plans by combining causal reasoning with reasoning about time and resources. They overcome the traditional division between planning and scheduling technologies. In this context, planning is intended as the generation of a system behaviour that satisfies certain desired conditions over a prefixed temporal horizon. Therefore, planning is not only the process of deciding which actions to perform in order to satisfy some desired conditions, but also deciding when to execute these actions in order to obtain some desired behavior of the system. Indeed, temporal planning systems try to integrate planning and scheduling in a unified solving process.

3.1 pddl2.1

PDDL2.1 [Fox and Long, 2003] has been designed to allow PDDL-based systems to model and solve more realistic domains by introducing the capability of dealing with time. There are several planning systems that rely on this language, e.g. OPTIC [Benton et al., 2012], COLIN [Coles et al., 2012] POPF [Coles et al., 2010], which also maintains backward compatibility with PDDL. Existing PDDL domains are valid PDDL2.1 domains and valid PDDL plans are valid PDDL2.1 plans. A relevant contribution of PDDL2.1 is the introduction of discretized durative actions with temporally annotated conditions and effects. Conditions and effects must be temporally annotated in order to specify when a particular proposition must hold. Specifically, a proposition (i.e. a condition or an effect) can hold at the start of the interval of the action (i.e. the time point at which the action is applied), at the end of the interval (i.e. the time point at which the effects of the action are asserted) or over the entire interval (i.e. invariant over the duration of the action). The annotation of an effect specifies whether the related effect of the action is instantaneous (i.e. the effect becomes true as soon as the action is applied) or delayed (i.e. the effect becomes true when the action finishes). The code below shows a simple example of a durative action for loading a truck from the Dock-Worker Robots domain described in [Ghallab et al., 2004].

{highlights}
(:durative-action load-truck
:parameters (?t - truck)
(?l - location)
(?o - cargo)
(?c - crane)
:duration (= ?duration 5)
:condition (and
(at start (at ?t ?l))
(at start (at ?o ?l))
(at start (empty ?c))
(over all (at ?t ?l))
(at end (holding ?c ?o)))
:effect (and
(at end (in ?o ?t))
(at start (holding ?c ?o))
(at start (not (at ?o ?l)))
(at end (not (holding ?c ?o))))
)


Invariant conditions of a durative action hold over the entire duration of the action and are specified by means of the over all keyword (see the code above). It is worth observing that, the over all keyword excludes the start point and the end point of the action interval which is considered as an open temporal interval. If a particular preposition must hold at the start, at the end and also during the entire duration of the action, it must be specified with three temporal constraints, i.e. (at start p), (over all p) and (at end p).

3.2 Hybrid Planning approaches

There are other languages and planning frameworks that integrate causal and temporal reasoning without directly extending PDDL. An interesting planning language is the Action Notation Modeling Language (ANML) [Smith et al., 2008]. ANML has been introduced as an alternative to existing (temporal) planning languages like PDDL2.1, the IxTeT language or NDDL (the EUROPA planning language). Broadly speaking ANML represents an high-level language whose aim is to uniformly support generative and HTN planning models and provide a clear and well-defined semantics compatible with PDDL family of languages. ANML relies on a strong notation of action and state, provides constructs for expressing common forms of action conditions and effects, supports rich temporal constraints and uses a variable/value representation.

{highlights}
action Navigate (location from, to) {
duration := 5 ;
[all] { arm == stowed ;
position == from :-> to ;
batterycharge :consumes 2.0
}
}


The code above shows an example of an high-level navigation action for a rover expressed in ANML. The action has two location parameters of type from and to and a fixed duration (5 time units). The temporal qualifier [all] means that the related statements (i.e. the statements contained by the adjacent block of code) are valid all along the duration of the action. Specifically, the first statement specifies that the arm of the rover is stowed over the entire action. The second statement specifies that the position of the rover is from at the start of the action (a condition), the position is undefined during action execution, and the position is to at the end of the action (an effect). The last statement specifies the amount of energy consumed by the action.

The Flexible Acting and Planning Environment (FAPE) is a recently introduced planning framework [Dvorák et al., 2014] which extends HTN planning with temporal reasoning by implementing the ANML language. Another recent planner worth to be considered is CHIMP [Stock et al., 2015]. CHIMP relies on its own modeling language and extends HTN planning domain representation with temporal representation by leveraging the functionalities of the meta-csp [Mansouri and Pecora, 2014].

Chapter \thechapter Timeline-based Planning in a Nutshell

\lettrine

[lines=2]The timeline-based approach is a Temporal Planning paradigm introduced in early 90’s [Muscettola, 1994], which takes inspiration from classical control theory. The main distinct factor is the centrality of time in the representation formalism. Unlike classical approaches, timeline-based planning puts time to the center of the solving approach by dealing with concurrency, temporal constraints and flexible durations. Timeline-based planning realizes a sort of hybrid representation and reasoning framework which allows a solver to "easily" interleave planning and scheduling decisions. This hybrid view of planning is one of the key characteristic for successfully addressing real-world problems. Timeline-based solvers have been successfully applied in real-world contexts (especially in space-like contexts) [Muscettola et al., 1992], [Jonsson et al., 2000a], [Cesta et al., 2007].

The world model of a timeline-based application is characterized by a set of features that must be controlled over time in order to realize a complex behavior/task of a particular system to control. A complex system (e.g. a planetary exploration rover) is modeled by identifying a set of features that are relevant from the control perspective (e.g. the stereo camera or the communication facility). Each feature is modeled in terms of the values it may assume over time and their related temporal durations. The temporal evolution of a feature is represented as a timeline which consists of an ordered sequence of valued temporal intervals, usually called tokens. These tokens describe the behavior of the feature within a given temporal horizon. In addition to the description of the features, the model may also specify domain rules that allow to further constrain the temporal behaviors of the features through temporal constraints. Such rules are necessary to achieve high-level goals (e.g. take and communicate pictures of a target) by coordinating different features properly. For example, a rule may require that a particular value of a feature occurs during a known temporal interval or that a token of a timeline must always occur before a particular token of another timeline. Thus, a timeline-based plan consists of a set of timelines and that must satisfy all the temporal constraints of the domain in order to be valid.

In timeline-based planning, unlike classical planning, there is not a clear distinction between states and actions. A valued temporal interval may represent either an action or a state the related feature must perform or assume over a particular temporal interval. Similarly planning goals do not represent simply states or conditions that must be achieved. Rather, a planning goal may be either a value that a particular feature is supposed to assume during a certain temporal interval, or a complex task (e.g. take a picture of a target) that must be performed within a given time. The solving process acts on an initial set of partially specified timelines representing the initial known facts about the world. The process completes the behaviors of these timelines by iteratively adding values and temporal constraints according to desired requirements (including planning goals). Thus, timeline-based planners realize a behavior-based approach to planning, whose focus is on constraining the temporal evolutions of the system rather than synthesizing a sequence of actions that allow to achieve a desired goal state.

There are several timeline-based systems that have been introduced into literature and successfully applied to real-world problems (especially in space-like contexts). EUROPA [Barreiro et al., 2012] developed by NASA, IxTeT [Ghallab and Laruelle, 1994] developed at LAAS-CNRS, and APSI-TRF [Fratini et al., 2011] developed for Esa, represent some of the most known existing frameworks in this field. The next sections provide a brief description of the most relevant features of these timeline-based planning frameworks.

4 Europa

The EUROPA framework [Barreiro et al., 2012] relies on Constraint-based Temporal Planning (CBTP) [Frank and Jonsson, 2003] which is a Temporal Planning formalism successfully applied in many space application contexts by NASA. The CBTP modeling approach focuses on the temporal behaviour of the system we want to control and not just on the causality relationships. Therefore, a complex system (e.g. a planetary exploration rover) is modelled by identifying a set of relevant components that can independently evolve over time.

A component models a physical or logical feature of the system to be controlled by specifying a (finite) set of mutually exclusive activities the related feature may assume over time. An activity is an atomic formula of the form:

 A(x1,...,xn,stA,etA,δ)

where (i) is a predicate representing a particular condition of the world, (ii) are numerical or symbolic parameters of activities, (iii) and are temporal variables representing respectively the activity start and end times and (iv) is an interval representing lower and upper bounds of activity’s duration.

Let be the set of activities. In CBTP formalism a component is defined by a subset of , where activities represent possible states or actions of the component . Components statically describe the possible temporal evolutions of the elements of the system. However, it is necessary to specify additional constraints in order to coordinate system’s element and guarantee the overall system safeness. CBTP considers two types of constraints: (i) Codesignation Constraints can impose equalities or inequalities between the parameters of activities; (ii) Temporal Constraints can model temporal constraints between activities by expressing either interval-based or point-based temporal predicates.

In general, CBTP models temporal constraints by extending the qualitative temporal interval relationships defined in [Allen, 1983] with quantitative information. Namely, the basic temporal relations between intervals are enriched with metric information, i.e. lower and upper bounds of the distances between temporal intervals. For example, the relation A before [10, 20] B states that the interval A must precede interval B not less than 10 time units and not more than 20 time units.

The causal and temporal constraints of the system are modeled by means of dedicated rules, called compatibilities, that specify interactions between a particular activity of a component and other activities that can either belong to the same component (internal compatibility) or to a different component (external compatibility). Compatibilities describe how a particular activity (the master) is related to other activities (the slave) by specifying a set of codesignation and/or temporal constraints that must be satisfied in order to build valid plans. Conditional compatibilities can be defined by means of guard constraints that "extend" the conditions under which the related compatibility can be applied. If the guard constraints of a compatibility are satisfied, then the related temporal and/or codesignation constraints can be applied. Given a set of activities , the compatibility for an activity is defined as

 C[Ai]:G(→y)→T(Ai,Aj,...,Ak)∧P(→xi,→xj,...,→xk)

where (i) is a conjunction of guard constraints, (ii) is a conjunction of temporal constraints involving the activities and (iii) is a conjunction of codesignation constraints on variable and variables in .

If a compatibility specifies different constraints according to the different values a particular guard variable may assume then, represents a disjunctive compatibility. Given an activity , a configuration rule for is a conjunction of compatibilities and it is defined as

 R[A(→x,st,etA,δ)]=C1[A]∧...∧Cn[A]

The code below shows some compatibilities and configuration rules for a classical planning problem concerning the control of a planetary exploration rover.

{highlights}
R[Unstow()] = {
[meets Place(rock) $$\land$$ met_by Stowed()]
}

R[Place($$rock_b$$)]   =   {
[meets Use(inst, $$rock_u$$) $$\land$$ ($$rock_u=rock_b$$)] $$\land$$
[met_by Unstow()] $$\land$$
[contained_by MobilitySystem.At($$rock_a$$) $$\land$$ ($$rock_a=rock_b$$)]
}

R[Use(inst, $$rock_b$$)] = {
[$$\gamma$$ = 0 $$\to$$ meets Stow()
$$\gamma$$ = 1 $$\to$$ meets Place($$rock_p$$ $$\land$$ ($$rock_p\neq\ rock_b$$)] $$\land$$
[met_by Place($$rock_p$$) $$\land$$ ($$rock_p=rock_b$$)] $$\land$$
[contained_by MobilitySystem.At($$rock_a$$) $$\land$$ ($$rock_a=rock_b$$)]
}


Given the elements described above, a planning domain is defined by a set of components , a set of activities associated with each component and a set of evolution rules , the domain contains an evolution rule for each activity . The CBTP planning process aims at building a valid description of the temporal behaviors of the components within a temporal horizon where goal activities are scheduled at proper times. Thus, a planning problem consists of a planning horizon and an initial configuration which (partially) describes the behaviors of the components. A solution plan is represented by a temporal execution trace which specifies for each time point, the activity the components are supposed to execute.

5 IxTeT

IxTeT [Ghallab and Laruelle, 1994] is a temporal planning system which tries to integrate plan generation and scheduling into the same planning process. Some of the most important features of the IxTeT planning paradigm are: (i) an explicit representation of time with different types of metric constraints between time points; (ii) a powerful representation of the world through multi-valued attributes; (iii) the management of a large range of resource types (unsharable, sharable, consumable and producible); (iv) a task formalism allowing for the representation of complex macro-operators.

Properties of the world are described by a set of multi-valued state attributes and a set of resource attributes. A state attribute describes a particular feature of the domain as a key-mapping from some finite domains into a finite range (the value of the attribute). The code below shows an example of a domain feature modeling the possible location of a robot.

{highlights}
attribute position(?robot) {
?robot $$\in$$ {robot1, robot2};
?value $$\in$$ {RoomM, LabRoom1, LabRoom2};
}


A resource is defined as any substance, or set of objects whose cost or availability induces constraints on the actions that use them. So a resource can be either a single item with unit capacity (i.e. an unsharable resource) or an aggregate resource that can be shared simultaneously between different actions without violating its maximal capacity constraint.

{highlights}
resource robots(?robot) {
$$?robot\in\ \{robot1,robot2\}$$;
capacity = 1;
}

resource paper_on_robot() {
capacity = 3;
}


IxTeT defines different types of state attributes that can classified as: (i) rigid attributes (or atemporal) representing attributes whose value does not change over time (they express a structural relationship between their arguments); (ii) flexible attributes (or fluents) representing attributes whose value may change over time. Flexible attributes may be further classified in: (i) controllable attributes representing attribute whose change of values can be planned for (but they can even change independently from the planning system); (ii) contingent attributes representing attributes whose changes of values cannot be controlled.

Moreover, IxTeT relies on a reified logic formalism where fluents (i.e. flexible attributes) are temporally qualified by the hold and the event (temporal) predicates.
The hold predicate

 hold(att(x1,...):v,(t1,t2))

asserts the (temporal) persistence of the value of state attribute to for each .
The event predicate

 event(att(x1,...):(v1,v2),t)

asserts the instantaneous change of the value of att(, …) from to occurred at time t.

Similarly, resource availability profile and the resource usage by the different operators are described by means of use, consume and produce predicates.
The use predicate

 use(typ(r):q,(t1,t2))

asserts the borrowing of an integer quantity of resource on the temporal interval .
The consume predicate

 consume(typ(r):q,t)

asserts that a quantity of resource is consumed at time .
The produce predicate

 produce(typ(r):q,t)

asserts that a quantity of resource is produced at time .

Temporal data representation and storage is managed by the time-map manager which relies on time-points as elementary primitives [Dechter et al., 1991]. Time is considered as a linearly ordered discrete set of instants. Time-points are seen as symbolic variables on which temporal constraints can be posted. IxTeT handles both symbolic constraints and numeric constraints expressed as a bounded interval on the temporal distance between time points. The time-map manager is responsible for propagating constraints on time-points to check the global consistency of the network and to answer queries about the relative position of time-points.

{highlights}
task feed_machine(?machine) (start, end) {
variable ?room;
place(?machine, ?room);
hold(position(robot): ?room, (start, end));
event(machine_state: (out_of_paper, ok), end);
consume(paper_on_robot(): 1, end);
produce(trunk_size(): 1, end);
(end - start) in [00:01:00, 00:02:00];
}


The initial plan is a particular task that describes a problem scenario by specifying: (i) the initial values for the set of instantiated state attributes (as a set of explained events); (ii) the expected changes on some contingent state attributes that will not be controlled by the planner (as a set of explained events); (ii) the expected availability profile of the resources (as a set of uses); (iv) the goals that must be achieved (usually, as a set of assertions).

6 Apsi-Trf

APSI-TRF [Fratini et al., 2011] is a software framework developed for Esa whose aim is to support the design and development of P&S applications by leveraging the timeline-based approach. The APSI-TRF framework provides the designer with a ready-to-use software library for modeling planning and scheduling concepts in the form of timelines. Specifically, APSI-TRF relies on the same modeling assumptions of HSTS [Muscettola, 1994] and therefore, a complex system is modeled by identifying a set of relevant features to control over time. The APSI-TRF framework makes available the modeling language and the software functionalities needed to model timeline-based domains in shape of multi-valued state variables and synchronization rules.

Multi-valued state variables model the features of the domain by describing their allowed temporal behaviors. State variables model domain features by specifying the values, the related feature may assume over time, together with the allowed durations and transitions. Thus, a state variable is defined as the tuple

 x=(V,D,T)

where (i) is the set of values the variable can assume over time, (ii) is a duration function specifying for each value the minimum and maximum duration and (iii) is a transition function specifying for each value the set of allowed successors. State variables specify causal and temporal constraints of the single features of a planning problem. They specify local rules that allow a planning system to build the timelines of the features composing the domain. Given a state variable , a timeline describes the sequence of values the variable assumes over time by specifying a sequence of valued temporal intervals called tokens. A token is defined as the tuple

 xi=(vj,[si,s′i],[ei,e′i])

where and represent respectively the flexible start and end of the temporal interval during which the variable is supposed to assume the value .

Synchronization rules model causal and temporal constraints of a planning domain by specifying global relations between tokens of different variables. In general, whenever a particular token occurs on a timeline (i.e. the trigger) a synchronization rule specifies a set of different tokens (i.e. the targets) that must occur on other timelines and a set of temporal constraints between the trigger and targets of the rule that must hold in order to build valid temporal behaviors. Indeed, synchronization rules allow the planning system to further constrain the temporal behaviors of the state variables in order to build timelines that satisfy some desired planning goals. Temporal constraints of synchronization rules are modeled by extending the qualitative relationships of the Allen’s interval algebra [Allen, 1983], with quantitative information.

It is worth observing that APSI-TRF, unlike other timeline-based frameworks (e.g. the EUROPA and IxTeT frameworks mentioned above), is not a planner but a development library for designing planning applications. In this regard, OMPS [Fratini et al., 2008] represents a domain-dependent timeline-based solver which has been developed on-top of the APSI-TRF modeling functionalities and successfully applied in space-exploration scenario [Ceballos et al., 2011].

7 Temporal Formalisms

Temporal Planners rely on expressive temporal formalisms that allow these paradigms to deal with time and temporal constraints. Many timeline-based systems (including EUROPA, IxTeT and APSI-TRF) model temporal information about plans by extending the Allen’s interval algebra [Allen, 1983] in order to represent expressive temporal relations between the temporal elements of a plan.

Temporal information represents additional knowledge the planner must properly managed during the solving process. Thus, timeline-based planners must encapsulate temporal reasoning mechanisms that process temporal information in order to verify the (temporal) consistency of plans. Temporal reasoning mechanisms are usually implemented by leveraging the formalism of Temporal Networks [Dechter et al., 1991] which represents a flexible representation of temporal data as a network of time points (i.e. the nodes of the network) and distance constraints between time points (i.e. the edges of the network).

7.1 The Simple Temporal Problem

The Simple Temporal Problem (STP) is a well-known formalism introduced in [Dechter et al., 1991] which consists of a set of events that may occur over known temporal intervals and a set of requirement constraints that specify distance constraints on the temporal occurrences of pairs of events. The problem is to find a temporal allocation of the events satisfying all the requirement constraints (i.e. the distance constraint). Namely, temporal reasoning mechanisms try to find an assignment of events to time points such that all the temporal constraints are satisfied. This concept is known as temporal consistency and is central to STPs.

Below is the description of a simple scenario taken from [Dechter et al., 1991], representing an example of the type of problems and inference the STP formalism can support. {highlights}

Example 1

John goes to work by car (30-40 minutes). Fred goes to work in a carpool (40-50 minutes). Today John left home between 7:10 and 7:20, and Fred arrived at work between 8:00 and 8:10. We also know that John arrived at work about 10-20 minutes after Fred left home. We wish to answer queries such as: "Is the information in the story consistent?", "What are the possible times at which Fred left home?", and so on.

Figure 2 shows the STP problem of Example 1 in graph form. When STPs are shown as graphs where nodes represent events and edges represent requirement constraints, they are called Simple Temporal Networks (STNs). The node "0" of Figure 2 represents the temporal origin of the plan/problem, the "absolute" time 7:00 with respect to Example 1. Node "1" in Figure 2 is associated with the event representing the time at which John leaves home. The edge between node "0" and node "1" labeled "[10, 20]" models the fact "John left home between 7:10 and 7:20" as a distance constraint (i.e. a requirement constraint) between the two related events. A strong limitation of the STP formalism concerns disjunctive constraints. STP cannot represent and therefore, cannot reason about disjunctive temporal intervals on events. Considering Example 1, STP cannot model disjunctive assertions like "John goes to work either by car (30-40 minutes), or by bus (at least 60 minutes)". Disjunctive assertions represent alternative plans the planning process may generate accordingly by branching the search space.

From a planning perspective, the temporal part of the plan can be reduced to a STP by modeling the start and end times of the activities of the plan (e.g. the start and end times of the tokens of a timeline) as events of the STP. Temporal relations and/or duration constraints concerning the activities/actions of the plan can be easily translated in the STP as one or more requirement constraints involving events related to the start/end times of the activities of the plan. Thus a planning system can leverage the STP formalism to post ordering constraints between activities during the solving process and check the temporal assignment of the activities of the plan. Namely, a planning system can check the (temporal) consistency of a plan by verifying the existence of a valid schedule of all the activities.

7.2 The Simple Temporal Problem with Uncertainty

STP makes the assumption that all the events of the plan are controllable. It means that the planning system can decide the temporal allocation (i.e. the schedule) of all the events. However, this is not always possible in real-world settings. Indeed, the activities of a plan usually model real-world tasks/actions whose durations can be affected by exogenous factors and therefore, the planning system cannot decide the temporal allocation of these activities (e.g. the planner can decide the start time of the execution of an action but not the end time). Such activities are called uncontrollable. Thus, a more expressive temporal formalism is the Simple Temporal Problem with Uncertainty (STPU).

The STNU formalism takes into account both controllable and uncontrollable events. In this formalism an event is considered uncontrollable if it is the target of contingent constraints that are typically used to model uncontrollable durations of the activities/actions of the plan. The key point of STPUs is that temporal consistency is not sufficient to solve real-world problems. Temporal uncertainty introduces the additional problem of deciding how to schedule controllable events according to the observed/possible temporal occurrences of uncontrollable events, in order to complete the execution of the plan. Such a problem is called the controllability problem which has been fairly investigated in the literature [Vidal and Fargier, 1999, Morris et al., 2001]. Broadly speaking, three different types of controllability (weak, strong and dynamic controllability) have been defined according to the different assumptions made on the uncontrollable events of a plan. Planning systems may leverage the STPU formalism to generate plans with some desired properties concerning the controllability of generated plans (i.e. properties concerning the execution of the generated plans in the real-world).

Chapter \thechapter Flexible Timeline-based Planning with Uncertainty

\lettrine

[lines=2]Despite the practical success of timeline-based planning, formal frameworks characterizing this paradigm have been proposed only recently. There is a multitude of software frameworks that have been realized and introduced in the literature, each of which applies its own interpretation of timeline-based planning. In such a context, it is not easy to evaluate the modeling and solving capabilities of different timeline-based planning systems. It is not even easy to define benchmarking domains to compare timeline-based systems, or to open the assessment to other planning techniques.

This chapter describes a complete and comprehensive formal characterization of the timeline-based approach which has been introduced in [Cialdea Mayer et al., 2016]. The proposed formalization aims at defining a clear semantics of the main planning concepts by taking into account the features of the most known timeline-based planning frameworks. In addition, the formalization takes into account temporal uncertainty which is particularly relevant in real-world domains where not everything is controllable. Indeed, the execution process is not completely under the control of the executive system. Exogenous events can affect or even prevent the complete and successful execution of generated plans. Thus, the capability of representing and dealing with temporal uncertainty and controllability properties at both planning and execution time is crucial to deploy effective timeline-based applications in real-world scenarios.

8 A Running Example: The Rover Domain

In order to support the formal definitions given below, a simple case study will be used as a running example. The domain takes inspiration from a typical scenario of AI-based control for a single autonomous agent.

The Rover domain consists of an exploration rover which can autonomously navigate a (partially) unknown environment, take samples of some targets (e.g. rocks) and communicate scientific data to a satellite. An exploration rover is a complex system endowed with several devices that must be properly controlled in order to achieve the desired objectives. A navigation facility allows the rover to move and explore the environment. A dedicated instrument facility allows the rover to take samples of targets that must be analyzed. A communication facility allows the rover to send data acquired from sampled targets to a satellite whose orbit is known.

A mission goal requires the rover to move towards a desired target, take a sample of it and communicate gathered data when possible. All the features that compose the rover must be coordinated properly in order to realize the complex behavior needed to satisfy mission goals. Thus, a set of operative constraints must be satisfied. For instance, communication of data must be performed while the rover is still and during some known communication windows that represent temporal intervals during which the target satellite is visible. Another operative constraint requires that the instrument facility must be set in a safe position/configuration while the rover is moving.

9 Domain Specification

The timeline-based approach to planning pursues the general idea that planning and scheduling for controlling complex physical systems consist of the synthesis of desired temporal behaviors (or timelines). According to this paradigm, a domain is modeled as a set of features with an associated set of temporal functions on a finite set of values. The time-varying features are usually called multi-valued state variables [Muscettola, 1994]. Like in classical control theory, the evolution of the features is described by some causal laws and limited by domain constraints. These are modeled in a domain specification.

The task of a planner is to find a sequence of decisions that brings the timelines into a final desired set, satisfying the domain specification and special conditions called goals. Causal and temporal constraints specify which value transitions are allowed, the minimal and maximal duration of each valued interval and (so-called) synchronization constraints between different state variables. Moreover, a domain specification must take into account the temporal uncertainty of planning domains in order to model more realistic problems. In particular, two sources of uncertainty are considered.

On the one hand, the evolution of some components of the domain may be completely outside the control of the system. What the planner and the executive know about them is only what is specified in the underlying planning problem. On the other hand, some events may be partially controllable. In this case, the planner and the executive can decide when to start an activity, but they cannot fix the duration of the activity. According to this characterization, two types of state variables constitute a planning domain: the planned variables model the controllable or partially controllable features of a domain; the external variables model the uncontrollable features of a domain. Thus, the planning system or the executive must respectively make planning and execution decisions, without changing the behavior of external variables or making hypothesis on the actual duration of partially controllable features.

For the sake of generality, temporal instants and durations are taken from an infinite set of non-negative numbers , including . For instance, can be the set of natural numbers (in a discrete time framework), as well as the non-negative real numbers . Sometimes, is given as an upper bound to allowed numeric values, with the meaning that for every . The notation will be used to denote , and . When dealing with temporal intervals, if , the (closed) interval denotes the set of time points .

9.1 State Variables

A state variable is characterized by four components: the set of values the variable may assume, a function mapping each value to the set of values that are allowed to follow , a function tagging each value with information about its controllability, and a function which may set upper and lower bounds on the duration of each variable value.

Definition 1

[State Variable] A state variable , where is a unique identifier, called the variable name, is a tuple , where:

1. , also denoted by , is a non-empty set, whose elements are the state variable values.

2. is a total function, called the state variable value transition function.

3. is a total function, called the controllability tagging function; is the controllability tag of the value . If , then is a controllable value, and if , then is uncontrollable.

4. is a total function such that for some and , and if , then and ; is called the state variable duration function.

If , then the planning or executive system can control the value and can decide the actual duration of related activities (e.g. the executive can decide when to start and end the execution of these activities). If , then the planning or executive system cannot control the value and cannot decide the actual duration of related activities. The behaviors of these activities are under the control of the environment.

The intuition behind the duration function is that if , then the duration of each interval in which has the value is included between and inclusive, if ; it is not shorter than and has no upper bound, if . In practice, existing systems, such as EUROPA [Barreiro et al., 2012] and APSI-TRF [Fratini et al., 2011], allow values to be represented by means of parametrized expressions In the present theoretical approach, values are taken to be completely instantiated in order to simplify the presentation. This amounts to describing sets and functions by enumeration and does not diminish the expressive power.

In what follows, it is assumed that, whenever a set of state variables is considered, for every distinct pair and in , . The set is partitioned into two disjoints sets, , containing the planned state variables, and , the set of the external ones. Every value of an external state variable is uncontrollable, i.e., . An external variable represents a component of the "external world" that is completely outside the system control: the planner cannot decide when to start or end its activities. What is known about an external variable is specified in the planning problem. On the contrary, a planned state variable represents a component of the system that is under the control of the executive. Nevertheless, controllable sub-systems may also have uncontrollable activities (i.e., activities whose starting times can be decided by the executive, but their durations and consequently their ending times, are not controllable). In other terms, the planner can decide when to start an uncontrollable activity of a planned variable (i.e. when the variable assumes an uncontrollable value), even if it cannot precisely predict how long it will last. In general, every time an activity (either controllable or not) is preceded by an uncontrollable one, the system cannot control its start time. Indeed, the start time of the activity is affected by the end of the previous activity, which is uncontrollable.

{highlights}
Example 2

In the considered running example, the timeline-based specification identifies five state variables, that will be called (for "rover"), (for "navigator"), (for "instrument"), (for "communication") and (for "window") whose values, transitions and controllability properties are illustrated in Figure 3 (values with dotted borders represent uncontrollable values).
Therefore, the set of considered state variables is where are planned state variables, while is an external one. For example, the state variable models the instrument facility the rover uses to sample targets. The state variable can be defined by the typle where:

• , , , , , , ;

• is the value transition function such that

• ,

• ,

• ,

• ,

• ,

• ,

• ;

• is the controllability tagging function such that

• ,

• ;

• is the value duration function such that

• ,

• ,

• ,

• .

The state variable , instead, is an external variable which models the availability of the communication channel during the satellite orbit. It can be defined by the tuple , where:

• ;

• ;

• is the value transition function such that

• ;

• is the duration function such that

• .

(Flexible) Timelines

A timeline represents the temporal evolution of a system component up to a given time. It consists of a sequence of valued intervals, called tokens, each of which represents a time slot in which the variable assumes a given value. A token represents a temporal interval which determines the instant the variable starts executing the related value and the time instant the variable ends executing that value.

However, planning with timelines takes into account time flexibility by allowing token durations to range within given bounds. It means that the start and end time instants of a token are replaced by temporal intervals. Thus, the notion of (flexible) timeline can be defined as follows:

Definition 2

[Timeline] If is a state variable, a token for the variable has the form:

 xi=(v,[e,e′],[d,d′],γ(v))

where , for , is the token name, , , and , for . The value is called the token controllability tag; if , then the token is controllable; if , then the token is uncontrollable. A timeline for the state variable is a finite sequence of tokens for , of the form:

where for all , and . The interval in the last token is called the horizon of the timeline and the number of tokens making up is its length. If is a token in the timeline , then:

• ;

• ;

• and ;

• ;

• with an abuse of notation, denotes the token controllability tag .

Intuitively, a token of the above form represents the set of valued intervals starting at some , ending at some and whose durations are in the range . The horizon of the timeline is the end time of its last token.

{highlights}
Example 3

Let us consider the timeline for the state variable , in the Rover domain, made of the following sequence of tokens:

 inst1=(Stowed,[20,28],[20,30],c)inst2=(Unstowing,[23,31],[3,3],c)inst3=(Unstowed,[50,55],[19,32],c)

The horizon of is .
An example of non-flexible timeline for the same state variable is made of the following sequence of tokens:

 inst1=(Stowed,[25,25],[20,30],c)inst2=(Unstowing,[28,28],[3,3],c)inst3=(Unstowed,[50,50],[19,32],c)

and its horizon is .

It is worth pointing out that often in the literature (e.g., [Fratini et al., 2008]), a flexible token contains also a start interval. However, once a token is embedded in a timeline, the time interval to which its start point belongs () can be easily computed as shown in the definition above. Thus, including it as part of the token itself is redundant.

On the contrary, duration restrictions alone would be inadequate to precisely identify when the valued intervals represented by a given token must begin and end. As a matter of fact, duration and end time bounds interact when determining which legal values a token end time may assume. Let us assume, for instance, that the duration of a given token is and that one may compute, from the durations of the previous tokens, that its start time is . One can then infer that the end points of the valued intervals it represents are necessarily in the range . However, it may be the case that a stricter end time is required, for instance . In this case, starting at and ending it at , though respecting the duration bounds, would not be a legal value to "execute" the token, since . So, differently from the case of non-flexible timelines, durations alone are not sufficient to suitably represent tokens. Analogously, end time bounds do not capture all the necessary information: the above described token does not represent a valued interval starting at and ending at , even though it respects the start and end time bounds, it violates the duration constraint.

Controllability tags are part of token structures for a different reason. Although is equal to , such information is included in the token with the aim of having a self-contained representation of flexible plans, encapsulating all the relevant execution information. This allows the executive system to handle plans with no need of considering also the description of the state variables. When considering a set FTL of timelines for the state variables in , it is always assumed that it contains exactly one timeline for each element of .

Schedules

A scheduled timeline is a particular case where each token has a singleton as its end time, i.e. the end times are all fixed. A schedule of a timeline is essentially obtained from by narrowing down to singletons (i.e. time points) the token end times.

The schedule of a token corresponds to one of the valued intervals it represents (i.e., it is obtained by choosing an exact end point in the allowed interval, without changing its duration bounds). A scheduled timeline is a sequence of scheduled tokens satisfying the duration requirements. Tokens, timelines and sets of timelines represent the set of their schedules.

In general, and STL will be used as meta-variables for scheduled timelines and sets of scheduled timelines, respectively, while and FTL as meta-variables for generic (flexible) timelines and sets of timelines. In what follows, an interval of the form , consisting of a single time point, will be identified with the time point (and, with an abuse of notation, singleton intervals are allowed as operands of additions, subtractions, comparison operators, etc.).

Definition 3

[Scheduled] A scheduled token is a token of the form

 xi=(v,[t,t],[d,d′],γ(v))

(or succinctly ). A schedule of a token is a scheduled token , where .
A scheduled timeline is a timeline consisting only of scheduled tokens and such that if is the timeline length, then: for all , if , then .
A scheduled timeline for the state variable is a schedule of if and have the same length , and for all , , the token of is a schedule of the token of .
Let FTL be a set of timelines for the state variables in . A schedule STL of FTL is a set of scheduled timelines for the state variables in , where each is a schedule of the timeline .

In simple terms, a scheduled timeline is a timeline where every end time is a singleton respecting the duration bounds. A schedule of a timeline is a way of assigning values to each token end time, so that both duration and end time bounds are respected. Tokens, timelines, and sets of timelines represent the set of their respective schedules.

{highlights}
Example 4

Let us consider the flexible timeline of example 3:

 FTLinst=inst1=(Stowed,[20,28],[20,30],c)inst2=(Unstowing,[23,31],[3,3],c)inst3=(Unstowed,[50,55],[19,32],c)

It is worth pointing out that, since the start time of the first token of a timeline is , its end time bounds are usually equal to its duration bounds, but, like this example shows, it is not necessarily so.
When the two intervals differ, the end point of the corresponding first token in any schedule of the timeline belongs to their intersection. Each schedule of the timeline represents a series of choices for the token end points, within the allowed intervals and respecting the allowed durations.
For instance, the following timeline is a schedule of :

 STLinst=inst1=(Stowed,25,[20,30],c)inst2=(Unstowing,28,[3,3],c)inst3=(Unstowed,51,[19,32],c)

In fact, it satisfies all of the endpoint and duration bounds in .
Clearly, not every sequence of scheduled tokens is a scheduled timeline. For instance, the sequence of tokens obtained from by replacing the token with is not a scheduled timeline at all, since it does not satisfy the duration constraints for : .
Let us now consider the scheduled timeline obtained from by replacing the token with . Although satisfies all the duration bounds, it is not a schedule of , since the end time of is not in the allowed interval of in .

9.2 Restricting the Behavior of State Variables

The behavior of state variables may be restricted by requiring that time intervals with given state variable values satisfy some temporal constraints. For instance, in the Rover sample domain, data can be communicated only when the communication channel is available. In other terms, for every token in the timeline for the state variable having the value , there must exist a token in the timeline for , with the value and bearing a given temporal relation with . This type of relations are expressed by means of synchronization rules that complete the definition of all the components of a domain specification.

Temporal Relations

As a first step, the set of allowed temporal relations is introduced. They are either relations between two intervals or relations between an interval and a time point. In particular, this work considers quantitative temporal constraints. For the sake of simplicity, a small set of primitive relations is chosen, all of which are parametrized by a (single) temporal interval.

Definition 4

[Temporal Relation] A temporal relation between intervals is an expression of the form , where and are time intervals, with , , and . The following table defines when a relation holds:

the relation holds if

Other relations can be defined in terms of the primitive relations in Definition 4 (and their converses). These relations, like those used by EUROPA [Barreiro et al., 2012] and APSI-TRF [Fratini et al., 2011], correspond to the quantitative extension of Allen’s temporal relations [Allen, 1983]. Thus, the relations in the left most column of Table 1, are meant as abbreviations of the corresponding expressions on their right.

Once relations on time intervals are defined, they can be transposed to relations on tokens. The expressions used to denote such relations refer to tokens by means of their names.

Definition 5

[Token Relation] Let and be names of tokens belonging to scheduled timelines for the state variables and , respectively, with , , , . Moreover, let and . Expressions of the form , for , and , for , are called relations on tokens.
The relation holds iff holds and the relation holds iff holds. When a relation on two tokens and holds, we also say that the tokens and satisfy the relation, and that any set of scheduled timelines that contain and satisfies the relation. Analogously, if a relation holds, then the token and any set of scheduled timelines containing satisfy the relation.

{highlights}
Example 5

Let be a set of timelines for the Rover domain, including , and , where contains the tokens

 cm5=(Idle,100,[1,43],c)cm6=(SendData,123,[11,32],u)

and contains the tokens

 win1=(NotAvailable,60,[60,80],u)win2=(Available,130,[50,90],u)

The expressions and are relations on tokens and they are satisfied by STL.

Synchronization Rules

A synchronization constraint can be informally considered as a statement of the form "for every token …there exist tokens such that …". Namely, it represents a kind of quantified sentence. The formal counterpart of this kind of assertions makes use of variables: for every …there exist such that …. The variables used to express synchronizations are called token variables. They are taken from a (potentially infinite) set of names, whose elements are all different from variable names, values and numbers. These variables are intended to range over tokens in the considered set of timelines.

Making a step forward, it can be observed that synchronization assertions actually use a form of bounded quantification: "for all/exist tokens with value in the timeline for the state variable …". Such token variables with restricted range will be denoted by expressions of the form , where is a token variable, is a state variable name, and . Such expression are called annotated token variables. The next definition introduces the form of the assertions that can be used to express parametrized relations on tokens.

Definition 6

[Existential Statement] An atom is either the special constant or an expression of the form or , where