# LQG feedback control of a class of linear non-Markovian quantum systems

## Abstract

In this paper we present a linear quadratic Gaussian (LQG) feedback control strategy for a class of linear non-Markovian quantum systems. The feedback control law is designed based on the estimated states of a whitening quantum filter for an augmented Markovian model of the non-Markovian open quantum systems. In this augmented Markovian model, an ancillary system plays the role of internal modes of the environment converting white noise into Lorentzian noise and a principal system obeys non-Markovian dynamics due to the direct interaction with the ancillary system. The simulation results show the LQG controller with the whitening filter obtains a better control performance than that with a Markovian filter in the problem of minimizing the photon numbers of the principal system when the ancillary system is disturbed by thermal noise.

## I Introduction

Feedback is a fundamental technique in classical control engineering and has been applying to quantum systems as well, for example, stabilizing a quantum system in an eigenstate by using Lyapunov feedback control [1] and rejecting noise in a linear quantum system by using feedback control [2], etc.

The feedback controlled quantum systems in most existing works are Markovian quantum systems. They refer to quantum systems disturbed by memoryless environments [3, 4, 5, 6, 7, 8], where the correlation time of the environment is much larger than that of the quantum system. Thus the noise from the environment can be assumed to be white noise satisfying a delta-commutative relation [8]. Here, the filter used for the feedback control is designed based on the Markovian system model. However, when a complicated environment, e.g., a non-Markovian environment involving colored noise, is considered, the existing feedback strategies will induce degraded performance. The main reason for this is that the quantum filters or observers used for feedback control of Markovian quantum systems have not taken the internal dynamics of the non-Markovian environment into account.

In classical control engineering, we can always append the state of the colored noise to the state of the system so as to obtain an augmented model of the total system which is only driven by white noise [9, 10]. Based on this augmented model, a whitening filter can be constructed via observing the output of the system. For example, considering an augmented classical dynamical system

(1) |

the evolution of the system state is apparently Markovian driven by white noise , i.e., the variation of the state is determined by the current state. While the dynamics of the subsystem is non-Markovian due to the injected colored noise from the subsystem . Similar ideas also arise in quantum systems, e.g., the pseudo-mode method [11, 12] and the hierarchy equation approach [13, 14] for modelling non-Markovian quantum systems, and the estimation of the quantum system driven by non-classical fields [15, 16, 17]. For effectively estimating the states of linear non-Markovian quantum systems, a whitening quantum filter is presented in [18] based on an augmented system model. An ancillary system defined on a Hilbert space is introduced, which takes the effects of the internal modes of the non-Markovian environment converting white noise into colored noise, and it is directly coupled with a principal system defined on a Hilbert space . Hence, the total system evolves on the Hilbert space , where is a Fock space for white noise. A standard Belavkin quantum filter can be obtained for this augmented system by applying a non-demolition probing field to the principal system. Thus the estimated states of the principal system can be used in the feedback control of the principal system.

In this paper, we utilize the estimated states of the whitening quantum filter in the feedback control of a linear and Lorentzian-noise-disturbed non-Markovian quantum system. Since both the Lorentzian noise generator; i.e., the ancillary system, and the principal system are linear quantum systems, the feedback control law can be designed for the augmented system by using a linear quadratic Gaussian (LQG) approach. The optimal controller includes the whitening quantum filter for reconstructing the states of the non-Markovian quantum system and a linear function of the estimated states [10]. The separation principal guarantees that the quantum filter and the feedback control law can be designed separately [10].

The paper is organized as follows. Markovian quantum systems are briefly reviewed in Section II. In Section III, an augmented system model for a Lorentzian-noise-disturbed non-Markovian quantum system is introduced. Based on the augmented system model, a whitening quantum filter for the non-Markovian quantum system is given in Section IV. The LQG feedback control for this linear non-Markovian quantum system is presented in Section V. A possible experimental realization using optical systems and corresponding simulation results are shown in Section VI. Conclusions are drawn in Section VII.

## Ii Review of Markovian quantum systems

### Ii-a description

A Markovian quantum system refers to a quantum system interacting with white noise fields, which can be systematically described by an description as

(2) |

where the component is a scattering matrix describing the input-output relation of fields passing through beam splitters, the operator vector is a collection of system operators interacting with the white noise fields, and is the system Hamiltonian [19].

### Ii-B White noise field

The white noise field on the Boson Fock space can be defined as

(3) |

satisfying the delta commutation relations

(4) |

From the definition (3), an integrated operator can be defined as , whose commutation relation can be calculated as and thus the operator is the quantum analog of the Wiener process and is quantum white noise. Also a scattering process can be defined as . Note that we assume the initial state of the field on the Fock space is a vacuum state such that this process is analogous to Gaussian white noise with zero mean.

### Ii-C Quantum stochastic differential equation

The evolution of the system satisfies a quantum stochastic differential equation (QSDE)

(5) | |||||

In the Heisenberg picture, the evolution of a system operator can be defined as satisfying a QSDE , namely a quantum Langevin equation, as

(6) | |||||

with a generator

(7) |

The notation defines a Lindblad superoperator which can be calculated as for two arbitrary operators and with suitable dimensions. Such an equation describes the dynamics of the system driven by an external white noise field, which has been widely used in the analysis and control of Markovian quantum systems [8].

### Ii-D Input-output relations

To observe the dynamics of the system, one may consider an output field. The output field is the field after interaction with the system, which satisfies a QSDE as

(8) |

which shows that the output field not only carries information of the system but also is affected by the noise. As a result, the output field can be utilized by a quantum filter or a feedback controller [3, 20, 21].

## Iii Markovian representation for a class of non-Markovian quantum systems disturbed by Lorentzian noise

In this section, we consider a class of non-Markovian quantum systems disturbed by Lorentzian noise. Instead of describing the system in a traditional way [22, 23], we will introduce an augmented system model. In this model, colored noise in the non-Markovian environment is generated by an ancillary system driven by white noise and a principal system is directly coupled with the ancillary system resulting in its non-Markovian dynamics. The structure of the augmented system model is given in Fig. 1.

### Iii-a Ancillary system driven by white noise

To convert white noise to colored noise with a Lorentzian spectrum, we consider an ancillary system is described by

(9) |

i.e., an optical mode in a leakage cavity, where is the angular frequency and () is the annihilation (creation) operator of the ancillary system defined on a Hilbert space . Here the coupling operator is chosen as , where is a damping rate with respect to the white noise field.

With respect to , a QSDE for the annihilation operator can be obtained as

(10) |

We define

(11) |

as a fictitious output.

It is easy to check that the power spectral density for is Lorentzian calculated to be

(12) |

with a center frequency and a linewidth determined by the angular frequency of the ancillary system and the damping rate with respect to the white noise field, respectively. Note that in the broadband limit [24]; i.e., , the fictitious output reduces to white noise with a delta correlation function [18].

### Iii-B Principal system interacting with the ancillary system

A principal system on a Hilbert space with a free Hamiltonian is of interest, which is disturbed by the colored noise created by the noise model, i.e., the ancillary system, via a direct interaction. Thus the principal system and the ancillary system constitute an augmented system.

The interaction Hamiltonian for the coupling between the principal system and the ancillary system is

(13) |

where is a direct coupling operator of the principal system expressed as a product between a principal system operator and a coupling strength . The principal and ancillary systems influence each other due to their direct interaction as shown in Fig. 1.

This augmented principal-ancillary system can be described by using an description as

(14) |

By cancelling the operators of the ancillary system, a non-Markovian Langevin equation for an operator of the principal system can be obtained as

(15) | |||||

where the convolution terms are expressed as

(16) |

Note that we label corresponding operators with a bar in this augmented system picture.

## Iv Whitening quantum filtering for non-Markovian quantum systems

To estimate the dynamics of the non-Markovian system, we can construct a quantum filter using a probing field defined on a Fock space . The total system can be described as

(17) |

where is the coupling operator of the principal system for the probing field.

Note that supposing an operator of the augmented system can be denoted as , the generator can be written as

(18) | |||||

where

(19) | |||||

(20) |

are the generators for the principal system and the ancillary system, respectively. The notation defines a Lindblad superoperator [18].

Here, the probing field satisfies a non-demolition condition and the detection efficiency is assumed to be [3, 27]. Hence, we can obtain a Belavkin quantum filter for the augmented system as

(21) | |||||

where the conditional expectation is denoted as . The measurement results generate a commutative subspace and also drive an innovation process which is equivalent to a classical Wiener process. Note that the increment is independent of . Note that for linear principal systems, the Belavkin filter is actually a quantum Kalman filter [28, 29].

## V LQG feedback control of the Lorentzian-noise-disturbed linear non-Markovian quantum systems

### V-a The augmented system in a quadrature representation

In this section, we may consider a more complicated case that the ancillary system is driven by thermal noise as well as quantum noise, since the noise spectrum generated by the ancillary system is independent on the input fields of the ancillary system. The thermal noise disturbs the ancillary system through the coupling operator . Here, we denote the white quantum noise and thermal noise for the ancillary system as and , respectively. In a stochastic description, the corresponding infinitesimal increment for white quantum noise satisfies and . However, the infinitesimal increment for thermal noise satisfies different relations as , , and , where is the numbers of photons carried by the thermal noise [8, 30].

On the other hand, the controlled principal system we considered is an optical mode in a cavity, which is also a linear quantum system. The Hamiltonian of the controlled cavity can be written as

(22) |

with an angular frequency and an annihilation (creation) operator (), where is a semiclassical control field. Then, the coupling operator and can be specified as and .

Thus the Langevin equations for the augmented principal and ancillary cavities can be written as

where is the white noise probing field. The output of the probing field can be calculated as

(24) |

which carries the information of the principal system.

It is convenient to move the Eqs. (V-A) and (24) to a quadrature representation with real-valued coefficient matrices as

(25) | |||||

(26) |

by using a transformation matrix . Here, the coefficient matrices

(27) | |||||

(28) | |||||

(29) |

are real matrices and the components of are calculated as , . Since we observe the position component of the output field, the output is defined as .

In addition, the noise and are calculated as and , where the matrices and are expressed as

(30) |

The components of are calculated as , , and .

The white noise and thermal noise fields for the ancillary system and the probing field for the principal system are initially uncorrelated, leading to a covariance matrix of the noise as

(31) |

where is the covariance matrix of the thermal noise and is a identity matrix.

However, the noise and are correlated such that the intensity of the noise vector can be calculated as

(32) | |||||

Note that there are non-zero elements in the off-diagonal block and is invertible.

### V-B The whitening quantum Kalman filter

### V-C LQG control problem

Based on the estimation of the states of both the principal and ancillary systems, a semiclassical feedback control law can be designed by solving a LQG control problem. The LQG problem for this non-Markovian quantum system can be formulated as follows:

Consider the Markovian representation of a non-Markovian quantum system (25) whose output (26) can be used to construct the quantum Kalman filter (V-B). The joint white noise process has the intensity (32). The LQG feedback control problem is to find the feedback control based on the estimate , such that the objective

(34) |

is minimized. Here, , , and are symmetric weighting matrices such that , , and and the final time is denoted as . Note that the objective is a function of the states of the principal system except the states of the ancillary system . The ancillary system plays the role of the internal modes of the environment such that the states are uncontrollable.

### V-D LQG feedback control

The optimal feedback control law to minimize the objective over the times can be obtained by using the dynamical programming method [10, 30]. It turns out that the optimal feedback control can be expressed as

(35) |

with . The control is generated based on the estimated state of the augmented system. The symmetric matrix is a solution of the Riccati equation

(36) |

Note that the separation principle implies that the controller and the filter can be designed separately [10].

### V-E Closed loop dynamics

By applying the feedback control to the principal system, we can obtain the dynamics of the closed loop system including the controlled plant and the controller. Denoting the closed loop state as , the closed loop system dynamical equation can be written as

(37) |

where

(38) | |||||

(39) |

And the covariance matrix of the closed loop system in a steady state satisfies a Lyapunov equation

(40) |

## Vi Example

A possible experimental realization of the controlled non-Markovian quantum system is shown in Fig. 3. The principal system is an optical mode in a cavity, which is directly coupled with an optical mode in an ancillary cavity via an optical crystal. The ancillary system is driven by both white quantum noise and thermal noise. A probing field is injected to the principle system, whose output is observed via Homodyne detection to determine a whitening filter. The estimated state is utilized by the LQG control law.

Since the ancillary system is driven by thermal noise, the principal system will be also heated due to the direct interaction; i.e., the photon numbers in the principal system will increase. Hence, we apply LQG feedback control to cool the principal system, for which the objective can be written as

(41) |

where and with a identity matrix . Alternatively, the objective (41) in the steady state can be calculated as

(42) |

with where is the solution of the Lyapunov equation (40) for the closed loop system.

Fig. 4 shows the steady state photon numbers of the principal system; i.e., the objective varies with the photon number of the thermal noise . In this simulation, the angular frequencies of the principal and ancillary system are assumed to be . The damping rates of the ancillary system with respect to the noise and that of the principal system with respect to the probing field are . The coupling strength between the principal and ancillary system is . In addition, the weighting matrix is assumed to be far smaller than the element of , e.g., . The simulation results show that the LQG controller with a whitening filter has a better performance than that with a Markovian filter. As the strength of the thermal noise increases, the objective for the controller with a Markovian filter increases linearly. The derivation of the LQG controller with a Markovian filter is given in the Appendix. On the contrary, the controller with the whitening filter can achieve better performance. Since the whitening filter can estimate the state of both the principal and ancillary systems, the controller with the whitening filter can give the optimal control field. However, the controller with the Markovian filter lacks information on the ancillary system. As a result, its control performance degrades when the effect of the ancillary system becomes large.

## Vii Conclusion

In this paper, we have presented a LQG feedback strategy to control a Lorentzian-noise-disturbed linear non-Markovian quantum system based on its Markovian representation model. In this model, the ancillary systems play the role of the internal modes of the non-Markovian environment converting white noise to Lorentzian noise and thus the principal system obeys a non-Markovian dynamics due to its direct interaction with the ancillary system. With this Markovian representation model, the states of the augmented system can be continuously estimated by a whitening filter. The LQG feedback control is designed for the non-Markovian system based on the results of the whitening filter. Simulation results show the control performance is enhanced when we substitute the Markovian filter by the whitening filter. For future work, it is possible to apply this feedback control strategy to more general non-Markovian quantum systems with arbitrary noise spectra.

### -a System model for designing a Markovian quantum filter

If the whitening filter is substituted by a Markovian quantum filter, the control performance may degrade. Supposing we design the quantum filter based on a Markovian system model only; i.e., we only consider the probed principal system, the dynamical equations in the quadrature representation can be written as

(43) | |||||

(44) |

with

(45) |

where is the state of the principal system. The angular frequency is denoted as . In the simulation, we let . The noise terms can be further expressed as and with

where is the noise of the probing field. Note that the noise and are correlated and their covariance matrix can be calculated as

(46) | |||||

Here, is invertible and .

### -B Markovian quantum filter

Based on this Markovian model, we can obtain a Markovian filter as

(47) | |||||

where the estimated state is denoted as and its covariance matrix as . The gain of the filter can be calculated as .

### -C LQG control based on the Markovian filter

The LQG feedback control law can be designed as

(48) |

by using the estimated state of the Markovian filter , where . is a solution of the Riccati equation

(49) |

### -D Closed loop system dynamics

By applying the control (48) to the non-Markovian system model (25) and (26), the closed loop system dynamical equation can be obtained as

(50) |

where and the coefficient matrices are written as

(51) | |||||

(52) |

The covariance matrix for the steady state of the closed loop system satisfies a Lyapunov equation as

(53) |

And thus the objective in a steady state can be calculated as

(54) |

with

### References

- M. Mirrahimi and R. V. Handel, “Stabilizing feedback controls for quantum systems” SIAM J. Control Optim., vol. 46, no. 2, pp. 445-467, 2007.
- M. R. James, H. I. Nurdin, I. R. Petersen, “ control of linear quantum stochastic systems” IEEE Trans. Autom. Control, vol. 53, no. 8, pp. 1787-1803, 2008.
- L. Bouten, R. V. Handel, M. R. James, “An introduction to quantum filtering” SIAM J. Control Optim., vol. 46, no. 6, pp. 2199-2241, 2007.
- H. M. Wiseman, “Quantum-theory of continuous feedback,” Phys. Rev. A, vol. 49, no. 3, pp. 2133-2150, 1994.
- C. Sayrin, I. Dotsenko, X. Zhou, B. Peaudecerf, T. Rybarczyk, S. Gleyzes, P. Rouchon, M. Mirrahimi, H. Amini, M. Brune, J. M. Raimond, and S. Haroche, “Real-time quantum feedback prepares and stabilizes photon number states,” Nature, vol. 477, no. 7362, pp. 73-77, 2011.
- C. Altafini and F. Ticozzi, “Modeling and control of quantum systems: an introduction,” IEEE Trans. Autom. Control, vol. 57, no. 8, pp. 1898-1917, 2012.
- H. P. Breuer and F. Petruccione, The theory of open quantum systems, Oxford: Oxford University Press, 2002.
- C. W. Gardiner and P. Zoller, Quantum noise, Berlin: Springer, 2000.
- P. Hnggi and P. Jung, “Colored noise in dynamical systems,” Adv. Chem. Phys., vol. 89, pp. 239-326, 1995.
- H. Kwakernaak and R. Sivan, “Linear Optimal Control Systems”, New York: Wiley-Interscience, 1972.
- A. Imamolu, “Stochastic wave-function approach to non-Markovian systems,” Phys. Rev. A, vol. 50, pp. 3650-3653, 1994.
- L. Mazzola, S. Maniscalco, J. Piilo, K.-A. Suominen, and B. M. Garraway, “Pseudomodes as an effective description of memory: Non-Markovian dynamics of two-state systems in structured reservoirs,” Phys. Rev. A, vol. 80, p. 012104, 2009.
- J. Ma, Z. Sun, X. G. Wang, F. Nori, “Entanglement dynamics of two qubits in a common bath,” Phys. Rev. A, vol. 85, p. 062323, 2012.
- A. Shabani, J. Roden, and K. B. Whaley, “Continuous measurement of a non-Markovian open quantum system,” Phys. Rev. Lett., vol. 112, p. 113601, 2014.
- J. E. Gough, M. R. James, and H. I. Nurdin, “Quantum trajectories for a class of continuous matrix product input states,” New J. Phys., vol. 16, p. 075008, 2014.
- J. E. Gough, M. R. James, and H. I. Nurdin, “Quantum filtering for systems driven by fields in single photon states and superposition of coherent states using non-Markovian embeddings,” Quantum Inf. Process., vol. 12, no. 3, pp. 1469-1499, 2013.
- J. E. Gough, M. R. James, H. I. Nurdin, and J. Combes, “Quantum filtering for systems driven by fields in single-photon states or superposition of coherent states,” Phys. Rev. A, vol. 86, p. 043819, 2012.
- S. Xue, M. R. James, A. Shabani, V. Ugrinovskii, and I. R. Petersen, “Quantum filter for a class of non-Markovian quantum systems,” arXiv:1503.07999v2.
- J. Gough and M. R. James, “The series product and its application to quantum feedforward and feedback networks,” IEEE Trans. Autom. Control, vol. 54, no. 11, pp. 2530-2544, 2009.
- M. Yanagisawa and H. Kimura, “Transfer function approach to quantum control part I: dynamics of quantum feedback systems,” IEEE Trans. Autom. Control, vol. 48, no. 12, pp. 2107-2120, 2003.
- G. F. Zhang and M. R. James, “Direct and indirect couplings in coherent feedback control of linear quantum systems,” IEEE Trans. Autom. Control, vol. 56, no. 7, pp. 1535-1550, 2011.
- M. W. Y. Tu and W. M. Zhang, “Non-Markovian decoherence theory for a double-dot charge qubit,” Phys. Rev. B, vol. 78, no. 23, p. 235311, 2008.
- S. Xue, R. B. Wu, and T. J. Tarn, “Modeling and analysis of non-Markovian open quantum systems for coherent feedback,” 3rd IFAC Conference on Intelligent Control and Automation Science, IFAC Secretariat, vol. 3, pp. 365-370, 2013.
- J. Gough, “Quantum Stratonovich calculus and the quantum Wong-Zakai theorm,” J. Math. Phys., vol. 47, no. 11, p. 113509, 2006.
- S. Xue, R. B. Wu, W. M. Zhang, J. Zhang, C. W. Li, and T. J. Tarn, “Decoherence suppression via non-Markovian coherent feedback control,” Phys. Rev. A, vol. 86, p. 052304, 2012.
- H. T. Tan and W. M. Zhang, “Non-Markovian dynamics of an open quantum system with initial system-reservoir correlations: a nanocavity coupled to a coupled-resonator optical waveguide,” Phys. Rev. A, vol. 83, p. 032102, 2011.
- V. P. Belavkin, “Quantum diffusion, measurement, and filtering,” Theory Probab. Appl., vol. 38, no. 4, pp. 573-585, 1994.
- A. C. Doherty and K. Jacobs, “Feedback control of quantum systems using continuous state estimation,” Phys. Rev. A, vol. 60, no. 4, pp. 2700-2711, 1999.
- H. M. Wiseman and A. C. Doherty, “Optimal unravellings for feedback control in linear quantum systems,” Phys. Rev. Lett., vol. 94, p. 070405, 2005.
- R. Harmerly and H. Mabuchi, “Coherent controllers for optical-feedback cooling of quantum oscillators,” Phys. Rev. A. vol. 87, p. 013815, 2013.