Quantum Control Theory^{†}^{†}thanks: This research was supported by the Australian Research Council Centre of Excellence for Quantum Computation and Communication Technology (project number CE110001027), and US Air Force Office of Scientific Research Grant FA23861214075.
Abstract
This paper explains some fundamental ideas of feedback control of quantum systems through the study of a relatively simple twolevel system coupled to optical field channels. The model for this system includes both continuous and impulsive dynamics. Topics covered in this paper include open and closed loop control, impulsive control, optimal control, quantum filtering, quantum feedback networks, and coherent feedback control.
Contents
1 Introduction
We are currently witnessing the beginnings of a new era of technology—an era of technology that fully exploits the unique resources of quantum mechanics. This “second quantum revolution” (Dowling and Milburn, 2003) will demand a new generation of engineering methodologies. Quantum engineering will be a new branch of engineering focused on the design and manufacturing of quantum technologies. At present, quantum engineering is embryonic and includes activities that range from laboratory experimentation of new devices and systems, to the development of the theory that inspires and supports the creation of these new devices and systems. Just as today’s nonquantum engineering arose from the foundations of classical physics, mathematics, and (most importantly) entrepreneurship, quantum engineering is beginning to evolve from the fundamental principles of quantum physics and mathematics and examples of quantum entrepreneurship can now be seen.
This document is focused on one aspect of this second quantum revolution—quantum control. Control, of course, is essential to technology, and indeed played a key enabling role during the industrial revolution. Quantum control originated in the sciences, and we are now beginning to see the growth of quantum control research in engineering. In what follows we look at some recent developments in quantum control theory from the perspective of a control theorist. We hope that readers will see how various contributions to quantum control fit together. In particular, the paper explains some fundamental ideas of feedback control through the study of a relatively simple twolevel system coupled to optical field channels. This model for this system includes both continuous and impulsive dynamics.
This paper is organized as follows. In Section 2 we describe some of the main types of quantum control that have appeared in the literature. In Section 3 we discuss several aspects of open loop control: (i) time optimal control for a closed system, (ii) impulsive optimal control, and (iii) regulation of systems subject to relaxation. In preparation for our more detailed discussion of quantum feedback, Section 4 describes quantum feedback networks (QFN), while Section 5 presents some of the basic ideas of quantum filtering, which is seen as a natural extension of statistical reasoning to quantum mechanics. We then consider two types of optimal measurement feedback control problems in Section 6, and discuss the important idea of information states for feedback control. Finally, Section 7 presents some ideas concerning coherent feedback control.
Notation and Preliminaries
Quantum mechanics is usually represented mathematically using a Hilbert space. In this chapter, will denote a finite dimensional Hilbert space, say , the dimensional complex vector space. In Dirac’s notation, the inner product for is denoted
The vector is denoted (represented by a column vector of length with complex entries ), is called a ket, while dual (row) vectors are called bras and written as
A linear operator on is denoted ^{1}^{1}1We do not use to indicate operators. Later, we will use to denote an estimate of operators . (typically represented by an complex matrix). For any operator its adjoint^{2}^{2}2Note that we use to denote adjoint of an operator instead of . However, if is a matrix (with operator or complex number entries ), we write (conjugate transpose). is an operator defined by
The adjoint of a vector is a dual vector (represented by a row vector):
An operator is called normal if . Two important types of normal operators are selfadjoint (), and unitary ().
The spectral theorem says that if is a selfadjoint operator on a finite dimensional Hilbert space , the eigenvalues (not necessarily distinct) of are real and can be written as
(1) 
where is the projection
(2) 
and are the corresponding orthonormal eigenvectors. The projections resolve the identity .
Let’s denote the collection of all (bounded, linear) operators on a Hilbert space by . The set can be thought of as a vector space, where operators are “vectors”. Indeed, if and are complex numbers and , then the linear combination is the operator in defined by
(3) 
Note that the linear combination on the RHS of (3) is a linear combination of vectors in the Hilbert space . We can multiply operators,
(4) 
so that if . Also, the adjoint if . So the collection of operators is closed under addition, scalar multiplication, multiplication, and adjoints—mathematically, is called a algebra. This mathematical structure is fundamental to quantum mechanics.
Tensor products are used to describe composite systems. If and are Hilbert spaces, the tensor product is the Hilbert space consisting of linear combinations of the form , and inner product . Here, and . If and are operators on and , respectively, then is an operator on and is defined by . Often, is written as , and is written .
2 Types of Quantum Control
Due to their relative simplicity and tractability, the twolevel quantum system and the quantum harmonic oscillator are two of the most important prototype models for quantum systems. These models are widely used for describing real physical systems, as well as for tutorial purposes. In this article we discuss a range of aspects of quantum control primarily focused on the basic twolevel quantum system as the system to be controlled. Two level systems are used in quantum computing as the qubit, in NMR spectroscopy as a basic spin system, and in quantum optics as a model for an atom with two energy levels. The oscillator plays a role in the representation of an electromagnetic field to which the twolevel atomic system is coupled, Figure 1, [18, Fig. 9.1].
In this chapter we consider a twolevel atom coupled to several electromagnetic field channels. In the absence of any other influences, the behavior of the atom will evolve in time according to the laws of quantum mechanics, as determined by the selfenergy of the atom, the nature of the fields and how they are coupled to the atom. The field channels may be used to gather information about the atom, and to influence the behavior of the atom. Figure 2 shows a schematic representation of an atom coupled to a pair of optical field channels, one of which is used to describe light shone on the atom, while the other channel contains the outgoing light, [18, Sec. 9.2]. Also shown is a channel used to apply a rapid radiofrequency (RF) pulse. As will be explained, light shone on the atom may be regarded as a control signal that may be classical or quantum in nature; the later allowing for coherent feedback control. In this article we are interested in idealized impulsive models for pulses (zero width and infinite height).^{3}^{3}3In reality all pulses have nonzero width and finite height, and the impulsive model is useful when the time scales are such that the response to a rapid pulse is much faster than the other dynamics of the system. The impulsive signals are classical signals, but may be used to effect coherent transformations of the system.
We now present a model for a controlled two level atom. Since a twolevel atom is a quantum system with two energy levels, the model makes use of the Pauli matrices
(5) 
Any observable of the atom can be expressed in terms of these matrices. In particular, the atomic energy levels are the eigenvalues of the Hamiltonian describing the selfenergy of the atom.
The atom interacts with the field channels by an exchange of energy that may be described by first principles in terms of an interaction Hamiltonian, [18, Chapter 3]. In this article we use an idealized quantum noise model for the open atomfield system which is well justified theoretically and experimentally, [25], [17], [39], [18]. Each field channel has input and output components, modeled as quantum stochastic processes. The input processes, and , drive an interactionpicture equation for a unitary operator governing the atomfield system. If and are initial atomic and field states respectively, the state of the atomfield system at time is .
This continuous (in time) stochastic unitary model holds in the absence of the abovementioned impulsive actions. Now suppose that impulses are applied at times by selection of a unitary from a set of unitaries. If is selected at time , the state immediately after the impulse has been applied is . So the impulse is modeled as instantaneously effecting a unitary transformation. Combining the continuous and impulsive motions, we see that if the overall unitary at time is
(6) 
where indicates which impulse was selected at time and is the unitary for the continuous motion on the time interval (), with .
Let’s now look at the equations governing the hybrid continuousimpulsive dynamics. We suppose that the twochannel field is initially in the vacuum state, which we denote by . In this case the input processes and are independent quantum Wiener processes, for which the nonzero Ito product are (). The atomfield coupling is determined by coupling operators and , where
is the lowering operator (the raising operator is defined by ), and are parameters describing the strength of the coupling to each channel. The evolution of the unitary is given by
(7) 
Here, indicates the value immediately after , i.e., the limit from the right. Equation (7) is quantum stochastic differential equation with impulses, whose solution (of the form (6)) is determined by a sequence of timeimpulse pairs
(8) 
So far, we have not described how modulation of the field can be used to control the atom. Let us now do so. Suppose we modulate the second field channel to be in a coherent state , where is a classical function of time (a classical control signal). We may model this by replacing with (displacement, [18, sec. 9.2.4]). Provided the corresponding output channel is not used for further interconnection, this may equivalently be represented by replacing the Hamiltonian term in equation (7) by the controldependent Hamiltonian
(9) 
Thus we arrive at the following equations for the controlled atomic system:
(10) 
The solution will be determined by a classical control signal and timeimpulse sequence . The controlled twolevel atom is illustrated in Figure 2. Choice of and before being applied to the system (i.e. offline) is called open loop control; no feedback of information from the system is used.
What does feedback mean in the context of our twolevel atomic system? The answer depends on how information is extracted from the system and how this information is used to change the behavior of the system. Accordingly, it is helpful to identify the following types of quantum feedback:

Measurement feedback. The output field channel, say , is continuously monitored (measured) and the measurement signal is processed by a classical system, called a classical controller (implemented, say, in classical analog or digital electronics), to determine the closed loop control signals and . This type of feedback involves a directional exchange of information between a quantum system and a classical system, and so involves a loss of quantum information. Measurement feedback is illustrated in Figure 3.

Coherent feedback using quantum signals. An output field, say , is not measured, but rather is provided as an input field to another open quantum systems, which we may call a coherent quantum feedback controller. This quantum controller coherently “processes” to produce a field that is shone as an input into the second channel of the atom. In the coherent feedback loop, information remains at the quantum level, and flows in one direction around the loop. An example of a coherent feedback arrangement is shown in Figure 4.

Coherent feedback using direct coupling. Here, the atom is simply coupled directly to another quantum system (without the aid of the fields). This may be regarded as a form of feedback which is bidirectional, in the spirit of “control by interconnection” [43], and does not involve directional quantum signals transmitted via fields. The second quantum system also serves as a coherent quantum feedback controller, and as with coherent feedback using quantum signals, all information remains at the quantum level. Direct interaction between the atom and a coherent controller is shown schematically in Figure 5.
There is a large literature on the topic of coherent control, mainly arising from applications in chemistry and NMR spectroscopy (for example, [40], [33]). How does this fit into the terminology for control discussed above? Quite simply: direct couplings between systems may be engaged via the application of a pulse (the coupling is active while the pulse is one, and not active while it is off). The pulse may be regarded as an open loop signal applied to a composite system, resulting in a unitary transformation. In the limit that the pulse has zero width and infinite height, one obtains an impulsive representation of the open loop signal and unitary action. Note that the quantum controllers discussed above could also depend on classical control signals, although we have not included this possibility explicitly in the above discussion. So the term “coherent control” is rather broadly used, and is meant to convey that the control actions preserve quantum coherence in some way.
Of course, the twolevel model given above may be generalized in a number of ways. For instance, to cover other physical situations the Hamiltonian and coupling operators and may be redefined, and the fields may be placed in nonvacuum states.
3 Open loop control
Two of the very earliest papers on quantum control are [2] and [24]. The paper [2] discusses open quantum models, filtering, and optimal feedback control. This work was well ahead of its time, and was largely unknown for a considerable period. As we will see later on, Belavkin’s farsighted ideas are highly relevant today. The paper [24], also very much ahead of its time, looks at open loop control of quantum systems, the subject of this section.
3.1 Bilinear Systems
The simplest type of model for open loop control is that of an isolated twolevel atom (no field couplings) with a Hamiltonian depending on a classical control variable . Indeed, we have the following differential equation for the unitary :
(11) 
The open loop control system (11) defines a bilinear control system evolving on the Lie group . Bilinearity refers to the presence of products involving the control variables and the variable being solved for (the unitary ). It is completely deterministic, and beginning with the pioneering paper [24], a large literature has accumulated studying this type of system using methods from nonlinear control theory, and applying the results to a range of problems (see, for example, [9]).
Let’s now take a closer look at the dynamics of the atom, first in the Heisenberg picture, where atomic operators evolve according to . Since all atomic operators can be expressed in terms of the Pauli matrices, it suffices to determine the dynamics of and . By using the commutation relations
(12) 
we find that
(13) 
Equation (13) gives a complete description of the controlled atomic motion, expressed as a bilinear control system in the algebra (the vector space of complex matrices, with the usual matrix multiplication and involution given by the matrix adjoint).
Now we switch to the Schrodinger picture, within which state vectors evolve as , or ; more generally, density operators evolve as , or . Now any density operator may be expressed in the form
(14) 
where the (real) vector is known as the Bloch vector, with length . The Schrodinger dynamics is given in terms of the Bloch vector as follows:
(15) 
This equation is a bilinear system in the solid Block sphere . If the initial Bloch vector is on the surface of the Bloch sphere, i.e. , as is the case for initial state vectors (i.e. ), then equation (15) ensures that for all , and so describes a bilinear system on the surface of the Bloch sphere.
We see therefore that open loop control of isolated systems leads to interesting bilinear control systems defined on spaces other than Euclidean spaces (Lie groups, algebras, and manifolds). However, it is important to point out that for some types of quantum systems linear control systems arise, even though the equations for the unitary are bilinear. This happens in the case of the quantum harmonic oscillator, where the commutation relation (a constant, unlike the commutation relations (12) for the Pauli matrices) gives rise to a linear equation for the annihilation and creation operators.
3.2 Optimal Control
We turn now to the problem of optimally controlling the atomic system discussed in Section 3.1. By this we mean (for the moment) to find an open loop control signal that optimizes a performance criterion chosen to reflect a desired objective. For instance, the paper [33] used optimal control theory to design pulse sequences to achieve rapid state transfers. The performance criterion used was the time taken to go from an initial unitary to a target final unitary; a problem of timeoptimal control. The authors were able to exploit the rich structure of Lie groups and Lie algebras to develop an elegant formulation of the problem and explicit solutions in some cases.
It is worth remarking at this point that controllability is closely related to optimal control; indeed, the fundamental ideas of controllability and observability were developed by Kalman in his studies of linear quadratic optimal control problems [31]. In timeoptimal control, the minimum time function is finite precisely when it is possible to steer a system from the initial state to a given target state [13].
Let’s now look at timeoptimal control of the atomic system, not at the level of the unitary, but at the level of state vectors. Given a fixed target state , and an initial state , find a control signal that steers the atom from to in minimum time. In terms of Bloch vectors, given a fixed target state and an initial state , find a control signal that steers the atom (via the dynamics (15)) from to in minimum time.
In order to formalize this, we define the minimum time function (the value function for timeoptimal control) by
(16) 
Here, is the time taken for the atom to move from the initial Bloch vector to the final Bloch vector using the control signal . Thus, is the minimum time over all control signals.
If there is no restriction on the range of the control signal , then the timeoptimal control problem is singular and leads to impulsive solutions, as in [33], [41]. A reformulation of this problem using a hybrid model is described in Section 3.3. For the remainder of this section, let’s assume that the controls take values in closed interval . Before proceeding, let’s rewrite (15) in compact form
(17) 
where the vector field is defined by the right hand side of (15).
Dynamic programming is a basic tool in optimal control theory, [14], [15]. Suppose we have a smooth nonnegative solution to the dynamic programming equation (DPE) or HamiltonJacobiBellman (HJB) equation
(18)  
(19) 
Equation (18) is a nonlinear partial differential equation on the Bloch sphere, in which denotes the directional derivative of the function at the Bloch vector in the direction . Equation (19) is a boundary condition, corresponding to the fact that the optimal time taken to go from to itself is zero.
The main purpose of the DPE is the verification theorem [14], which allows us to test a candidate optimal control signal for optimality. Indeed, suppose we have a control signal that attains the minimum in the DPE (18), i.e.
(20) 
where^{4}^{4}4sign if and sign if . is the corresponding trajectory of (17) with initial condition . Then is optimal and equals , the minimum time function defined by (16).
To see why the verification theorem is true, let be any control signal steering to , and let . Now integrate equation (18) along the trajectory to obtain
(21)  
with equality if . Setting we see that with equality if . Hence and is optimal.
Expression (20) suggests a formula for “feedback” optimal controls. Define
(22) 
that is, for any Bloch vector , is a control value that attains the minimum of . However, this “feedback” formula requires knowledge of the Bloch vector , which is not possible in the present context  no measurement information is available, and the quantum state is not a measurable quantity. However, expression (20) may be used offline, in a computer simulation to determine an optimal open loop control signal. If one wishes, the optimizing control may be substituted into the DPE (18) to rewrite it in the form
(23) 
Where does the DPE (18) come from? Well, if the minimum time function is sufficiently smooth, then it solves the DPE (18) (by definition it satisfies (19)). To see this, for any the minimum time function satisfies
(24) 
An important technical issue is that in general is not everywhere differentiable, and nonlinear PDEs like (18) do not in general admit smooth solutions. Nevertheless, solves (18) in a weaker sense that does not require smoothness. The theory of viscosity solutions was developed to deal with nonsmooth solutions to nonlinear PDE, [15].
In order to gain some more insight into the nature of the solution to the optimal control problem, we switch to polar coordinates and use the fact that the dynamics (15) preserves states on the surface of the Bloch sphere. We write , , , and find that the dynamics becomes
(25)  
(26) 
In these polar coordinates, the target state is , and if we write we obtain the DPE
(27)  
(28) 
The control attaining the minimum in (27) is
(29) 
3.3 Impulsive Control
Suppose we remove the restriction on the range of the control signal, i.e. take . Now the vector field has the form , and so if we attempted to find the minimum in the DPE (18) we would find that it is not defined, as the controls would need to be infinitely large. This singular situation leads us to impulsive control actions, [33], [41], which form the subgroup of determined by the control Hamiltonian , that is,
(30) 
Now let’s use a hybrid continuousimpulsive model for timeoptimal control on the surface of the Bloch sphere. Let be an impulsive open loop control of the form (as in equation (8)), and consider the hybrid form of the Schrodinger equation
(31) 
where is the impulse applied at time .
The hybrid equations of motion on the surface of the Block sphere are
(32)  
(33) 
These equations describe the natural drift, a rotation in the plane, together with a choice of instantaneous rotation in the plane produced by the selected impulse.
The minimum time function is again defined by (16), but now the DPE takes the form
(34)  
(35) 
to which one seeks a nonnegative solution. If the minimum time function is sufficiently smooth, then it will be a solution of the quasivariational inequality (QVI) (34). The QVI has two parts:
(36)  
(37) 
Equation (36) simply says that drifting or impulsing will lead to a time greater than or equal to the minimum time. Equation (37) says that along the optimal trajectory the atom should drift at Block vectors for which , while if the Bloch vector is such that , then the impulse should be applied.
3.4 Relaxation
Let’s take a look at open loop control of the atom (or ensemble of atoms) in the presence of a decohering mechanism. For definiteness, consider an atom coupled to a single field channel (, in the notation of Section 2), with impulsive control only (with impulses in , the subgroup defined by (30)). The hybrid Schrodinger equation for the unitary is
(38) 
In the Heisenberg picture, atomic operators evolve according to , so that between impulses we have
(39)  
where , and
(40) 
When an impulse is applied at time , we have
(41) 
where .
Explicitly, for the operators and , we have, for ,
(42)  
(43)  
(44) 
and for ,
(45)  
(46)  
(47) 
Equations (42)(47) constitute a set of impulsive QSDEs in the algebra , where is the algebra of field operators (defined on an underlying Fock space). By taking expectations, we may conclude from (44) that the mean energy of the atom decreases exponentially, that is, the atom looses energy to the field.
The equations of motion for the atomic state may be obtained by averaging out the noise in the Schrodinger picture, ; here, we take the field to be in the vacuum state. The differential equation for , which holds between impulses, is
(48) 
where
(49) 
Equation (48) is called the master equation, and is called the Lindblad superoperator (in Schrodinger form).
The hybrid equations of motion inside the Bloch sphere are
(50)  
(51)  
(52)  
(53) 
In equations (50)(52) we can see the effect of the field coupling, which in the absence of control action causes , and as , i.e. , the pure state of lowest energy. Repeated application of impulses offers the possibility of achieving other large time behaviour. For instance, the periodic pulse sequence with period and leads to the steady state (for the case , ).
4 Quantum Feedback Networks
A glance at the various types of feedback control discussed in Section 2, or indeed any textbook on classical feedback control, tells us that feedback arrangements are networks of interconnected systems. Accordingly, the purpose of this section is to set up some easytouse tools for constructing feedback networks. What is presented in this section is a simplification of a more general quantum feedback network theory [47], [19], [20], which builds on earlier cascade theory [16], [8], network quantization [49], and quantum control [45].
The basic idea is simple, Figure 6. Take an output channel and connect it to an input channel. Such series or cascade connections are commonplace in classical electrical circuit theory. For instance, if the systems are resistors with resistances and , then the total seriesconnected system is equivalent to a single resistor with resistance . This use of simple parameters for devices, and rules for interconnecting devices in terms of these parameters, is a powerful feature of classical electrical circuit theory.
Let’s see how we can achieve an analogous simple rule for the series connection of two open quantum systems (without impulses), where the input and output channels are quantum fields, as discussed in Section 2. The physical parameters determining open quantum systems are a Hamiltonian describing the selfenergy of the system, and a vector of operators describing how the system is coupled to the field channels. These parameters appear in the Lindblad superoperator (and hence in the master equation), as well as in QSDEs, [25], [17], [39], [18]. Actually, there is a third parameter , a selfadjoint matrix of operators describing scattering between field channels, that was introduced in [25], [39]. While does not appear in the Linblad nor the master equation for a single system, it does have nontrivial use in several applications including quantum feedback networks (such as those including beamsplitters). Thus in general an open quantum system, call it , is characterized by three parameters . However, in this article we do not use the scattering parameter, and so we set ; actually, we make the abbreviation . For example, the parameters for the atom, coupled to two field channels (recall section 2), call it , are
(54) 
In network modeling, and indeed in modeling in general, it can be helpful to decompose large systems into smaller pieces, and to assemble large systems from components. In [19], the concatenation product was introduced to assist with this. The concatenation product is defined by
(55) 
For the atom, if we wish to decompose it with respect to field channels we may write
(56) 
Now suppose we have two systems and , as in Figure 6. Because the systems are separated spatially, the field segment connecting connecting the output of to the input of has nonzero length, and so this means there is a small delay in the transmission of quantum information from to . That is, . Now if the systems are sufficiently close, will be small compared with the timescales of the systems, and may be neglected. In this way, a Markovian model for the series connection may be derived. In terms of the physical parameters, the series product (defined in [19]) is given by
(57) 
Thus the series connection has parameters and , analogous to the expression for seriesconnected resistors.
The series product serves very well for Markovian approximations to cascades of independent open systems. However, importantly for us, the series product may also be used to describe an important class of quantum feedback networks. This is because the two systems and need not be independent—they can be parts of the same system.
For example, suppose we take to be the first factor in the decomposition (56) of , i.e. , and the second. Then the series connection
(58) 
describes the coherent feedback arrangement shown in Figure 7. The system is an open quantum system with Hamiltonian that is coupled to a single field channel via the coupling operator . Now that we have the parameters for the feedback system , it is easy to write down the corresponding Schrodinger equation