Efficient Desynchronization of Thermostatically Controlled Loads
Abstract
This paper considers demand side management in smart power grid systems containing significant numbers of thermostatically controlled loads such as air conditioning systems, heat pumps, etc. Recent studies have shown that the overall power consumption of such systems can be regulated up and down centrally by broadcasting small setpoint change commands without significantly impacting consumer comfort. However, sudden simultaneous setpoint changes induce undesirable power consumption oscillations due to sudden synchronization of the on/off cycles of the individual units. In this paper, we present a novel algorithm for counteracting these unwanted oscillations, which requires neither central management of the individual units nor communication between units. We present a formal proof of convergence of homogeneous populations to desynchronized status, as well as simulations that indicate that the algorithm is able to effectively dampen power consumption oscillations for both homogeneous and heterogeneous populations of thermostatically controlled loads.
footnoteinfo]The first author was supported in part by the Otto Mønsted and Dir. Ib Henriksen foundations, Denmark; The second author was partially supported by AFOSR.
First]Jan Bendtsen\thanksreffootnoteinfo Second]Srinivas Sridharan
1 Introduction
With growing penetration of renewable energy sources in modern power grids, demand side management has been gaining attention as a means of achieving better balancing between supply and demand (MohsenianRad et al. (2010), Strbac (2008), Short et al. (2007)). Indeed, it appears that the higher intermittency and lack of dispatchability associated with increased dependence on renewable energy sources can be taken care of more effectively by electrical loads than by conventional generators, which are typically not designed for fast up and down regulation (Strbac (2008), Klobasa (2010)).
Various technologies are currently being considered in the context of demand side management; coordinated charging of batteries, e.g., in electric vehicles (Mets et al. (2010)), deliberate scheduling of loads with flexible deadlines (Petersen et al. (2012)) as well as allowing local consumers with slow dynamics (large time constants) to store more or less energy at convenient times and thereby adjusting the momentary consumption (e.g., Moslehi and Kumar (2010)), among others. In particular, socalled thermostatically controlled loads (TCLs), such as deep freezers, refrigerators, local heat pumps etc., are showing great potential in this context, since they account for a large volume of consumption in most countries with significant renewable penetration—for instance, as of 2009, about 87 percent of all US homes were equipped with air conditioning.^{1}^{1}1 http://www.eia.gov/consumption/residential/reports/2009/airconditioning.cfm
Thus, at least in theory, manipulating the operating conditions of large populations of units slightly while avoiding discomfort to end users appears to be attractive, given that it can be achieved simply by broadcasting setpoint changes. Control strategies based on this principle were considered in Callaway (2009), Kundu et al. (2011), Perfumo et al. (2012) and Bashash and Fathy (2012), among others. In these strategies, subtle changes in the thermostat setpoint temperature (less than a degree Celcius) are transmitted to all the participating consumers simultaneously. Such small variations in the thermostat setpoints are expected to remain almost unnoticed by the customers, but if a sufficiently large number of units are shifted in the same direction, the overall power consumption can be shifted quickly by significant amounts, allowing to compensate for power production fluctuations in the grid.
There is an intrinsic problem with this approach, however. When the setpoint is changed concurrently in a large number of TCLs, their on/off cycles tend to become synchronized, which leads to large unwanted fluctuations (damped oscillations) in the overall power consumption Callaway (2009), Kundu et al. (2011). If the entire population of TCLs consists of identical units (homogeneous), it is possible to counteract the fluctuations by means of a centralized control law. In the much more realistic case of nonidentical (heterogeneous) units, on the other hand, the fluctuations are far harder to remove using centralized control strategies. To the authors’ knowledge, the only solution strategy presented in the literature so far is Kundu and Sinitsyn (2012), where a strategy in which the individual TCLs deliberately slow down their transition in order to avoid synchronization was proposed.
In this paper, we propose a novel, decentralized algorithm for avoiding synchronization without having to actively communicate between units. The algorithm proposed here is fundamentally different from the one presented in Kundu and Sinitsyn (2012)) in that it first adjusts the temperature bands in which the TCLs operate to achieve early (randomized) desynchronization, and then makes use of a technique inspired by contentionbased media access protocols to adaptively adjust the on/off cycles to achieve as little synchronization among units as possible. We present a formal proof of convergence of homogeneous populations to this desynchronized status, as well as simulations that indicate that the algorithm is able to effectively dampen power consumption oscillations for both homogeneous and heterogeneous populations of thermostatically controlled loads.
The outline of the rest of the paper is as follows. Section 2 first presents the simplified TCL model we shall employ, along with the synchronization issue. Section 3 then presents the proposed algorithm along with the aforementioned convergence result. Section 4 illustrates the feasibility of the approach through two simulation examples. Finally, Section 5 offers some concluding remarks.
2 Problem setup
We consider a population of individual temperature controlled loads. The TCLs are modeled in a generic manner, which is deliberately kept as simple as possible. Nonetheless, as will be illustrated with a few simulation examples, the behavior of a large population of simple units can be fairly complex.
2.1 Model
The individual TCLs are modeled as follows. Let the internal and ambient temperatures of the volumes (living spaces, refrigerators, cold storages etc.) affected by the action of the heating/cooling hardware of the ’th consumer be denoted and , respectively, and assume that the hardware is purely on/offregulated. A simple model for the internal temperature can then be formulated as Malhame and Chong (1985), Callaway (2009), Bashash and Fathy (2012):
(1)  
(2) 
for , where is the thermal capacitance of the consumer, is the corresponding thermal resistance and is the (constant) heating/cooling power supplied by the hardware when switched on. is a binary switching variable that determines whether or not the hardware is turned on; basically, it switches status whenever the internal temperature encounters the limits of a preset temperature span (, where is an infinitesimal positive value). Note that both heating and cooling systems can be modeled by (3)–(2), by choosing the sign of the term appropriately.
The temperature limits and are related to the ’th consumer’s setpoint through the relations
where is the width of the temperature interval. Furthermore, the cumulative power consumption of the population of TCLs at any given time can be computed as
(3) 
where is the coefficient of performance for the ’th heating/cooling unit.
Figure 1 illustrates the characteristic behavior of a single TCL. The parameters are chosen as C/kW, kWhC, kW, , C, C and C, respectively. The temperature setpoint is changed to C at time h, resulting in a slightly longer period in which the cooling power is turned off, until the new operating interval is reached.
2.2 Population behavior
As illustrated in Callaway (2009), Perfumo et al. (2012), Bashash and Fathy (2012) and others, the power consumption of populations of units with the individual dynamics given above can be manipulated via broadcasts of small setpoint changes. Figure 2 shows the behavior of a population of 10,000 homogeneous (identical) TCLs with the same set of parameters as in figure 1 (only the temperature profiles of 25 TCLs are shown in the figure). The devices are started at random temperatures unifomly distributed within , and their on/off cycles are thus desynchronized, yielding a fairly smooth power consumption trajectory. At time h, all the TCLs are subjected to a common step in the setpoint of 0.5C. As can be seen, the overall power consumption drops by an amount proportional to the size of the step, but significant oscillations appear in the power consumption due to synchronization of the individual units’ on/off cycles.
Figure 3 illustrates the behavior of a population of TCLs under the same simulation circumstances, but with chosen from a normal distribution with mean 5 and spread 0.5 kWhC (again, only the temperature profiles of 25 TCLs are shown in the figure). A step of 0.5C in the setpoint is broadcast to all the TCLs at time h. Notice how the power consumption is quite smooth before the step because the on/off cycles of the TCLs are desynchronized, whereas immediately after the step the TCLs become synchronized and the power consumption again exhibits large fluctuations, which in this case die out slowly. The oscillations die out faster if the parameters vary more (e.g., if is chosen from a distribution with larger spread); furthermore, the amplitude is relatively larger for larger populations of units. However, qualitatively the behavior remains the same. The oscillations are known as parasitic oscillations and are hard to remove via centralized control signals (Kundu and Sinitsyn (2012)).
2.3 Problem formulation
In order to reduce the aforementioned oscillations in power consumption after the setpoint change broadcasts, it is clearly necessary to deliberately desynchronize the TCLs without interrupting their operation. However, a centralized algorithm for doing so is undesirable; it is considered infeasible to keep close track of the internal states (temperatures, set points etc.) of all the units in a centralized manner, since doing so would require regular measurement feedback from all the devices, which would give rise to a very heavy communication and computational load on the system.
Thus, we look for a lowcomplexity, decentralized algorithm that can be implemented locally in each TCL, and which satisfies the following requirements:

The algorithm may only use local information; this means that the TCLs may not communicate with each other, and labels identifying each individual TCL may not be preassigned

Communication with the power supply utility must be limited to broadcast from the utility; no communication originating from the TCLs is allowed

The general operation of each individual TCL may not be altered in a manner that is detrimental to the user comfort; for example, the unit may not be deliberately kept turned off or on long enough for the temperature to leave the interval
3 Desynchronization algorithm
In the following, we present a completely decentralized algorithm that is able to reduce the power oscillations discussed above without violating any of the requirements R1–R3. The only significant assumption we require to be satisfied is that the individual TCLs have access to time stamps of changes in the total power consumption. While this may be considered a form of global information, its usage is limited to identifying the times at which the immediate predecessor and the immediate successor of a local TCL changes state. It is not used to actually identify said units; i.e., we only require local temporal information. The information about changes in global power consumption may either be measured locally or broadcast from the utility, thus avoiding violating requirements R1 and R2.
The algorithm comprises two main components. Firstly, the algorithm is reset whenever each TCL in question receives a setpoint change . At this point, each TCL immediately narrows its operation interval by a random value , causing the dynamics of unit momentarily to be governed by the dynamics
This operation rapidly induces a large degree of desynchronization among the population of TCLs without violating requirement R3, as illustrated in Figure 4. However, randomly choosing the temperature bounds may yield very tight bounds, which in turn may give rise to constant rapid on/offswitching. This is generally not desirable, so we drive the bounds back to their original settings by introducing the dynamics
(4) 
for some appropriately chosen , as also illustrated in the figure.
Now, even though most of the synchronization among the population of TCLs is likely to have disappeared with the random shrinking of the operating interval, it may still happen that some TCLs remain synchronized, or close to synchronized. The second component of the proposed algorithm is designed to deal with any ‘leftover’ synchronization among TCLs that are close to being identical. Here, we make use of the fact that the behavior of the TCLs is periodic in steady state operation. Let the period be denoted . Essentially, this part of the desynchronization algorithm repeatedly forces the devices to switch status at some point within the interval , and adaptively adjusts this enforced switch timing to achieve maximal spread across the period; see Figure 5.
In short, each TCL records the switch timings of the immediately preceding and the immediately succeeding TCLs; then it sets the new enforced switch time to the mean of the switch times of the succeeding and preceding TCLs, i.e.,
(5) 
where denotes the time of the enforced switch within the following period, modulo .
The enforced switch timings are initially distributed randomly; the TCL which randomly obtains the smallest value will then not be able detect a switch prior to its own. When it recognizes this fact, it sets its own switch timing to and maintains it at that value (modulo ); it is thus not necessary to assign a global label to the first and/or last TCL a priori.
Let denote the time of the ’th forced transition, modulo , at iteration ; that is, . We gather these switch timings in a vector .We can then prove the following asymptotic result:
If all switch timings for are updated according to (5), the globally attracting switch timing state (modulo ) is described by
(6) 
Proof.
The algorithm described above for the evolution of the state (time) dynamics requires that each state be the mean value of the prior and next state with the caveat that the ’th element should update its value based on the next pulse i.e. the value of the first state element in the next time instant. Hence, we first extend the state vector by placing the new value of the first state into the fictitious th dimension of the state vector to generate a new state vector, denoted . The update dynamics of the enforced switch times can be written as
(7)  
which implies that
Using eigenvalue decomposition, we may write the aforementioned limit as
(8) 
where is a diagonal matrix of eigenvalues of and contains the corresponding eigenvectors. has two repeated unit eigenvalues with linearly independent eigenvectors and , where is a vector of oneelements and
(9) 
The remaining eigenvalues are of absolute magnitude less than , implying that all except two of the diagonal elements in the eigenvalue matrix tend to zero as . Hence, in order to obtain the limiting value of the state dynamics, it remains only to utilize the eigenvectors corresponding to the unit eigenvalues to obtain the limit in (8).
Partition the eigenvector matrices as follows
and let and denote the eigenvectors corresponding to the unit eigenvalues, i.e., , . It is now possible to verify from the structure of and the partitioning of and that
(10) 
Finally, substituting , into (8) we get
(11) 
Thus it is seen that the asymptotic limit of the state is dependent only on the first and last elements of the state vector . As these are fixed at and respectively, the statement of the theorem follows as desired. ∎
It is of interest to note that if had been a positive (or irreducible) matrix, the PerronFrobenuis theorem would have been an elegant approach to obtain the steady state matrix directly. However, in this case, owing to the zero entires and the lack of strong connectivity of the graph associated with (a necessary and sufficient condition for irreducibility), the above more involved proof is required.
Finally, we state the full desynchronization algorithm as Algorithm 1, where we use the notation to denote the operation of picking a random value from a uniform distribution over the interval .
4 Simulation examples
Armed with Algorithm 1, we revisit the simulation examples in Section 2. Figure 6 shows the temperature curves of a subset of a population of 10,000 identical TCLs, simulated under the same conditions as above, i.e., C/kW, kWhC, kW, , C, C and C. The top subplot shows the behavior without desynchronization, while the bottom plot shows the behavior with Algorithm 1 applied. The temperature curves clearly look more ‘jumbled’ after the step, which indicates a greater degree of desynchronization, and the operation intervals can be seen to return to their original size after in the space of a few hours.
Figure 7 shows the corresponding power curves. The effect of the desynchronization is very obvious almost immediately after the step, as the amplitude of already the first peak is less than the case without desynchronization, and most of the oscillations are suppressed after about one period. We also notice that there are some oscillations remaining, which will gradually die out due to the longterm behavior of the adaptive algorithm.
Figure 8 shows a simulation with a heterogenous population of TCLs. Here, as opposed to Figure 7, the parasitic oscillations die out in the case without desynchronization as well. However, Algorithm 1 clearly speeds up the process, however. Note that the oscillations that can be seen before the setpoint change is due to the initialization of the TCL on/off states, which does not match exactly with the steadystate distribution (the desynchronization is not active before h). Note also that, by changing the setpoint of all the TCLs in the simulation by half a degree, it was possible to reduce the mean total power consumption by approximately 2 MW after the step.
5 Discussion
This paper presented a novel algorithm for counteracting unwanted oscillations caused by synchronization of populations of temperature controlled loads, which requires neither central management of the individual units nor communication between units. The algorithm comprises two main part, a ‘fast’ randomization of the temperature bands within which the TCLs operate, and a ‘slow’ adaptive adjustment of enforced switch timings, which maximizes desynchronization over time. We presented a formal proof of convergence of homogeneous populations to the desynchronized status, as well as simulations that indicate that the algorithm is able to effectively dampen power consumption oscillations for both homogeneous and heterogeneous populations.
However, the simulations also indicate that even with active desynchronization, it is hard to avoid large peaks right after a common broadcast setpoint step. Thus, highlevel modelbased control along the lines of the work presented in Kundu et al. (2011), will probably be preferable to simple steps.
References
 Bashash and Fathy [2012] S. Bashash and H.K. Fathy. Modeling and control of aggregate air conditioning loads for robust renewable power management. IEEE Transactions on Control Systems Technology, page to appear, 2012.
 Callaway [2009] D.S. Callaway. Tapping the energy storage potential in electric loads to deliver load following and regulation, with application to wind energy. Energy Conversion and Management, 50(9):1389–1400, 2009.
 Klobasa [2010] M. Klobasa. Analysis of demand response and wind integration in Germany’s electricity market. IET Renewable Power Generation, 4(1):55–63, 2010.
 Kundu and Sinitsyn [2012] S. Kundu and N. Sinitsyn. Safe protocol for controlling power consumption by a heterogeneous population of loads. In Proc. of 2012 American Control Conference, June 2012.
 Kundu et al. [2011] S. Kundu, N. Sinitsyn, S. Backhaus, and I. A. Hiskens. Modeling and control of thermostaticallycontrolledloads. In Proc. of the 17th Power Systems Computations Conference, 2011.
 Malhame and Chong [1985] R.P. Malhame and C.Y. Chong. Electric load model synthesis by diffusion approximation of a higherorder hybridstate stochastic system. IEEE Transactions on Automatic Control, 30(9):854–860, 1985.
 Mets et al. [2010] K. Mets, T. Verschueren, W. Haerick, C. Develder, and F. De Turck. Optimizing smart energy control strategies for plugin hybrid electric vehicle charging. In Network Operations and Management Symposium Workshops, pages 293–299. IEEE, 2010.
 MohsenianRad et al. [2010] A. MohsenianRad, v.W.S. Wong, J. Jatskevich, R. Schober, and A. LeonGarcia. Autonomous demandside management based on gametheoretic energy consumption scheduling for the future smart grid. IEEE Transactions on Smart Grid, 1(3):320–331, 2010.
 Moslehi and Kumar [2010] K. Moslehi and R. Kumar. A reliability perspective of the smart grid. IEEE Transactions on Smart Grid, 1(1):57–64, 2010.
 Perfumo et al. [2012] C. Perfumo, E. Kofman, J.H. Braslavsky, and J.K. Ward. Load management: Modelbased control of aggregate power for populations of thermostatically controlled loads. Energy Conversion and Management, 55(1):36–48, 2012.
 Petersen et al. [2012] M. Petersen, J. Bendtsen, and J. Stoustrup. Optimal dispatch strategy for the agile virtual power plant imbalance compensation problem. In Proc. of American Control Conference, Montreal, Canada, June 2012.
 Short et al. [2007] J. Short, D. G. Infield, and L. L. Freris. Stabilization of grid frequency through dynamic demand control. IEEE Transactions on Power Systems, 22(3):1284–1293, 2007.
 Strbac [2008] G. Strbac. Demand side management: Benefits and challenges. Energy Policy, 36(12):4419–4426, 2008.