Optimal Thermostat Programming and Optimal Electricity Rates for Customers with Demand Charges
Abstract
We consider the coupled problems of optimal thermostat programming and optimal pricing of electricity. Our framework consists of a single user and a single provider (a regulated utility). The provider sets prices for the user, who pays for both total energy consumed ($/kWh, including peak and offpeak rates) and the peak rate of consumption in a month (a demand charge) ($/kW). The cost of electricity for the provider is based on a combination of capacity costs ($/kW) and fuel costs ($/kWh). In the optimal thermostat programming problem, the user minimizes the amount paid for electricity while staying within a predefined temperature range. The user has access to energy storage in the form of thermal capacitance of the interior structure of the building. The provider sets prices designed to minimize the total cost of producing electricity while meeting the needs of the user. To solve the userproblem, we use a variant of dynamic programming. To solve the providerproblem, we use a descent algorithm coupled with our dynamic programming code  yielding optimal onpeak, offpeak and demand prices. We show that thermal storage and optimal thermostat programming can reduce electricity bills using current utility prices from utilities Arizona Public Service (APS) and Salt River Project (SRP). Moreover, we obtain optimal utility prices which lead to significant reductions in the cost of generating electricity and electricity bills.
I Introduction
To ensure the reliability of power networks, utility companies must maintain an uninterrupted balance between power generation and demand. In some ways this problem is becoming easier. Partially due to the development of energyefficient appliances and new materials for insulation, US electricity demand has plateaued [1] and is expected to remain flat (less than 1% growth) for the indefinite future (see Fig. 1(a)). The result is reduced reliance on carbonproducing fossil fuels. However, a new problem has arisen  partially due to increasing use of intermittent renewable energy sources such as distributed solar and wind  in that demand peaks continue to grow. Specifically, as per the US Energy Information Administration (EIA) [2], the ratio of peak demand to average demand has increased dramatically over the last 20 years, setting records of 1.89 in New England in 2012 and 1.96 in California in 2010 (see Fig. 1(b)). Because most utilities are required to maintain generating capacity as determined by peak demand, yet typically only charge customers for total consumption, there is real concern about the viability of existing business models. For example, due to net metering, a typical residential solar customer might have negative consumption during the day and positive consumption during the evening and morning. Such as customer might pay nothing for electricity while contributing substantially to the costs incurred by the utility. In response to this problem, many utilities have sought to halt or reverse growth of the netmetering framework  a process which has met with some limited success.
In this paper, we look at pricing strategies for reducing peak load while retaining the incentives necessary to create a robust distributed renewable sector. Naturally, utilities have been studying this problem for some time and with the widespread adoption of smartmetering (95% in Arizona), have begun to implement such strategies at scale. Examples of this include onpeak, offpeak and superpeak pricing  rate plans wherein the energy price ($/kWh) depends on the time of day [3]. By charging more during peak hours, utilities encourage conservation or deferred consumption during hours of peak demand. More aggressive strategies which have emerged recently include voluntary onpeak demandlimiting programs wherein customers are rewarded for reducing consumption when requested to do so by the utility [4]. A yet more aggressive strategy is direct load control [5, 6] wherein Heating, Ventilating, and Air Conditioning (HVAC) or other appliances are under the direct control of the utilities and can be deferred or deactivated at will. Quite recently, some utilities have introduced demand charges for residential customers. These charges are not based on energy consumption, but rather the maximum rate of consumption ($/kW) over a billing period. While such charges more accurately reflect the cost of generation for the utilities, in practice the effects of such charges on consumption are not wellunderstood  meaning that the magnitude of the demand charge must be set in an adhoc manner (typically proportional to marginal cost of generation).
An alternative approach to reducing peaks in demand is to use energy storage. In this scenario, batteries, pumping and retained heat are used during periods of low demand to create reservoirs of energy which can then be tapped during periods of high demand  thus reducing the need to increase maximum generating capacity. Indeed, the optimal usage of energy storage in a smartgrid environment with dynamic pricing has been recently studied in, for example, [7]. See [8] for optimal distributed load scheduling in the presence of network capacity constraints. However, to date the high marginal costs of storage infrastructure relative to incentives/marginal cost of additional generating capacity have limited the widespread use of energy storage by consumers/utilities [9]. As a costfree alternative to direct energy storage, it has been demonstrated experimentally [10, 11] and insilico [12, 13] that the interior structure of buildings and appliances can be exploited as a passive thermal energy storage system to reduce the peakload of the HVAC. A typical strategy  known as precooling  is to artificially cool the interior thermal mass (e.g., walls and floor) during periods of low demand. Then, during periods of high demand, heat absorption by these cool interior structures supplements or replaces electricity which would otherwise be consumed by the HVAC. Quantitative assessment of the effect of precooling on demand peak and electricity bills has been evaluated in, e.g., [14] and sun2013peak . It is important to note, however, that adhoc strategies such as precooling are only economical when using differential onpeak and offpeak pricing or demand charges.
The goal of this paper is twofold. First, we consider optimal HVAC usage for a consumer with fixed onpeak, offpeak and demand charges and model passive thermal energy storage using the heat equation. For a given range of acceptable temperatures and using typical data for exterior temperature, we pose the optimal thermostat programming problem as a constrained optimization problem and present a Dynamic Programming (DP) algorithm which is guaranteed to converge to the solution. This yields the temperature setpoints which minimize the monthly electricity bill for the consumer. After solving the thermostat programming problem, we use this solution as a model of user behaviour in order to quantify the consumer response to changes in onpeak rates, offpeak rates, and demand charges. We then apply descent methods to this model in order to determine the prices which minimize the costofgeneration for the utility. In a case study, we show that the optimal prices are NOT necessarily proportional to the marginal costs of generation  meaning that current pricing strategies may be inefficient.
Before presenting our results, we note that models for thermal energy storage do appear in the optimal thermostat programming literature [4, 14, 15, 16]. Furthermore, there is an extensive literature on thermostat programming for HVAC systems for onpeak/offpeak pricing [17, 18, 19] as well as realtime pricing (prices which are constantly changing) [20, 21, 16, 22] using Model Predictive Control. [23] and [24] consider optimal thermostat programming with passive thermal energy storage and onpeak/offpeak rates. Perhaps closest to our work, in [14], the authors use the concept of deep and shallow mass to create a simplified analogue circuit model of the thermal dynamics of the structure. By using this model and certain assumptions on the gains of the circuit elements, [14] derives an analytical optimal temperature setpoint for the demand limiting period which minimizes the demand peak. This scenario would be equivalent to minimizing the demand charge while ignoring onpeak or offpeak rates. Again, referring to [7] and subsequent publications, there has been some excellent work on optimal pricing (albeit without demand charges) for energy storage using batteries in an unregulated electricity market using a social welfare model. This paper differs from existing literature in that it: 1) Considers demand charges (demand charges are far more effective at reducing demand peaks than dynamic pricing) 2) Uses a PDE model for thermal storage (yields a more accurate model of thermal storage) 3) Uses a regulated model for the utility. Although unregulated utility models are popular, the fact is that most US utilities remain regulated.
Ii Problem Statement
In this section, we first define a model of the thermodynamics which govern heating and cooling of the interior structures of a building. We then use this model to pose the userlevel (optimal thermostat programming) problem in Sections IIB as minimization of a monthly electricity bill (with on/peak, offpeak and demand charges) subject to constraints on the interior temperature of the building. Finally, we use this map of onpeak, offpeak and demand prices to consumption to define the utilitylevel problem in Section IIC as minimizing the cost of producing electricity.
Iia A Model for the Building Thermodynamics
To model heat storage in interior walls and floors of a building, we use the onedimensional unsteady heat conduction equation
(1) 
where represents the temperature distribution in the interior walls/floor with nominal width and where is the coefficient of thermal diffusivity. Here is the coefficient of thermal conductivity, is the density and is the specific heat capacity. The wall is coupled to the interior air temperature using Dirichlet boundary conditions, i.e., , where represents the interior temperature which we assume can be controlled instantaneously by the thermostat. We model the heat loss through the exterior walls by the linear heat sink as
(2) 
where is the outside temperature and is the thermal resistance of the ext. walls, is the nominal width of ext. walls, is the coefficient of thermal conductivity and is the nominal area of the ext. walls. The heat/energy flux through the surface of the interior wall is modelled as
(3) 
where is the thermal capacitance of interior walls and is the nominal area of interior walls. By conservation of energy, the power required from the HVAC to maintain the interior air temperature is
(4) 
See Fig. 2 for a depiction of the model.
IiB UserLevel Problem: Optimal Thermostat Programming
In this section, we define the problem of optimal thermostat programming. We first divide each day into three periods: offpeak hours from 12 AM to with electricity price ; onpeak hours beginning at and ending at with electricity price ; and offpeak hours from to 12 AM with electricity price . In addition to the onpeak and offpeak charges, we consider a monthly charge which is proportional to the maximum rate of consumption during the peak hours. The proportionality constant is called the demand price . Given prices and , the total cost of consumption (daily electricity bill) is divided as
(7) 
where is the energy cost, is the demand cost and
The energy cost is
(8) 
where if and otherwise. That is, and correspond to the set of onpeak and offpeak sampling times, respectively. The function is a discretized version of (Eqn. (4)):
(9) 
This is the power consumed by the HVAC, where denotes the external temperature at timestep . If demand charges are calculated monthly, the demand cost for a single day is
(10) 
We now define the optimal thermostat programming problem at the userlevel as minimization of the total cost of consumption as defined in (7), subject to the building thermodynamics (Eqn. (6)) and interior temperature constraints ().
(11) 
where and are the acceptable bounds on the interior temperature. Note that this optimization problem depends implicitly on exterior temperature through the timevarying function .
IiC UtilityLevel Optimization Problem
We define the utilitylevel optimization problem as minimization of the cost of generating electricity such that generation is equal to consumption, and revenue is equal to cost of generation. Let be the amount of electricity produced as a function of time and . First, we consider a linear model of the production cost (adopted from Arizona Public Utility SRP) as
where is the marginal cost of producing the next of energy and is the marginal cost of installing the next of production capacity. Values of the coefficients and for SRP can be found in e.g., [25]. Now define and to be minimizing arguments to the userlevel problem defined in (11). Then the constraint that production equals consumption implies . We now define the utilitylevel optimization problem as minimization of the cost of electricity production subject to equality of production and consumption.
(12)  
where the last two lines constrain that costs equal revenue (recall is revenue from the users as defined in (7)).
Iii Solving User and Utilitylevel Problems
First, we solve the optimal thermostat programming problem using a variant of dynamic programming. This yields consumption as a function of prices . Next, we embed this implicit function in a descent algorithm in order to find prices which minimize the Utilitylevel optimization problem as formulated in (12). We start by defining a costtogo function, . Given , at the final time , we have
(13) 
Here for simplicity, we use to represent the discretized temperature distribution in the wall. Define prices if and otherwise. Then, we construct the costtogo function inductively as
(14) 
where is the set of allowable inputs at time and state :
Now we present the main result.
To prove Theorem 1, we require the following definitions.
Definition 1
Given , , and such that , define the costtogo functions
(16) 
where is defined as in (9), and and are the timesteps corresponding to start and end of the onpeak hours.
Note that from (8), it is clear that .
Definition 2
Definition 3
We now present a proof for Theorem 1.
Since the costtogo function , if we show that
(18) 
for and for any , where
then it will follow that . For brevity, we denote by , by and we drop the last two arguments of . To show (18), we use induction as follows.
Induction hypothesis: Suppose
for some and for any . Then, we need to prove that
(19) 
for any . Here, we only prove (19) for the case which . The proofs for the cases and follow the same exact logic.
Assume that . Then, from Definition 1
(20) 
where . From the principle of optimality [26] it follows that
(21) 
By combining (20) and (21) we have
(22) 
From Definition 1, we can write
(23) 
Then, by combining (22) and (23) and using the induction hypothesis it follows that
for any . By substituting for from (6) and using the definition of in (14) we have
for any . By using the same logic it can be shown that for any and for any . Therefore, by induction, (18) is true. Thus, .
Iv Numerical Examples and Analysis
In this section, we demonstrate convergence of our algorithm for optimal thermostat programming using electricity prices from APS and temperature data from Phoenix, AZ. In addition, we study the problem of optimal electricity pricing using the marginal cost data from SRP. We ran all the numerical simulations for three consecutive days with timestep , spacestep and with building’s parameters listed in Table I. We used an external temperature profile measured for three typical summer days in Phoenix, Arizona (see Fig. 3). For each day, the onpeak period starts at PM and ends at PM. In all scenarios, we used and .
0.4  0.0015  45 
onpeak ( per )  offpeak ( per )  demand ( per )  

APS  0.089  0.044  13.50 
Iva Scenario 1: The Effect of Electricity Prices on Peak Demand and Production Costs
In this scenario, we first consider the optimal thermostat programming problem (See (11)) using the electricity prices and as determined by APS [27] (See Table II). The results of the dynamic programming algorithm are given in Table III as total price paid by the user (we also include the peak demand). For comparison, we have run the same optimal control problem using the generalpurpose optimization solver GPOPS [28]. Moreover, we have compared our result with a typical precooling strategy and a naive strategy of setting the temperature to (constant). As can be seen, our algorithm outperforms the heuristic approaches. The power consumption and the temperature setting as a function of time for each strategy can be found in Fig. 4. For convenience, the onpeak and offpeak intervals are indicated on the figure. As can be seen, for APS prices and our building’s parameters, the optimal strategy does not reduce the peak demand with respect to the precooling strategy.
To examine the impact of changes in electricity prices on peak demand, we next chose several different prices corresponding to high, medium and low penalties for peak electricity demand. Again, in each case, our algorithm is compared to GPOPS and a precooling strategy. The results are summarized in Table IV. For each price, the smallest computed production cost and demand peak are typed in bold. The power consumption and the temperature settings as a function of time for the optimal strategy can be found in Fig. 5. For the optimal strategy, notice that by increasing the demand penalty, relative to the lowpenalty case, the peak consumption is reduced by 14% and 23% in the medium and high penalty cases respectively. Furthermore, notice that by using the optimal strategy and the high demandlimiting prices, we have reduced the demand peak by 29% with respect to the constant strategy in Table III. Of course, a moderate reduction in peak demand at the expense of large additional energy costs may not be desirable. The question of optimal distribution of electricity prices is discussed in Scenario II.
temperature setting  Electricity bill ()  demand peak () 

Optimal (Theorem 1)  36.58  9.222 
GPOPS [28]  37.03  9.155 
Precooling  39.81  8.803 
Constant  39.42  10.462 
Prices  Demandlimiting  Production cost  Demand peak  

high  88.712  7.4132  

medium  85.793  8.2898  
low  86.565  9.6749 
Prices  Demandlimiting  Production cost  Demand peak  

high  84.396  7.9440  

medium  86.182  9.1486  
low  87.382  9.6221 
Prices  Demandlimiting  Production cost  Demand peak  

high  91.064  8.8031  

medium  91.064  8.8031  
low  91.064  8.8031 
IvB Scenario 2: Optimal Thermostat Programming with Optimal Electricity Prices
In this case, we applied Algorithm 1 to find optimal onpeak, offpeak and demand prices under the assumption that the building’s parameters in Table I represent an averaged user. The marginal production costs and are taken as as estimated by SRP. The optimal prices, associated production cost, and associated peak demand are listed in Table V. A typical pricing strategy for SRP and other utilities is to set prices proportional to marginal production costs. The production cost associated with this strategy is also listed in Table V. Notice that the optimal prices are in fact not proportional to the marginal costs of generation.
Strategy  Production cost  Demand peak  

Optimal  83.333  8.3008  
SRP  89.005  7.4661 
V Conclusion and Future Work
In this work, we proposed a dynamicprogrammingbased algorithm for solving the optimal control problem associated with thermostat programming in the presence of distributed thermal energy storage in interior structures. We used a pricing model which is a combination of onpeak, offpeak and demand charges. Using the solution to this optimal control problem as a model of behavior, we determined the optimal prices which minimize production costs for the utility. We concluded that optimal thermostat programming can significantly reduce electricity bills and demand peak by taking advantage of energy storage using thermal mass. Furthermore, we showed that the typical approach to electricity pricing is suboptimal at reducing production costs. The results of this paper assume a rational consumer and accurate models of both the daily temperature and utility production costs.
Vi Acknowledgements
This material is based upon work supported by the National Science Foundation under Grant No. CMMI1301851. We would like to thank Salt River Project (SRP) for providing us with their suggestions and data.
References
 [1] J. J. Conti, “Annual energy outlook 2014 with projections to 2040,” US Energy Information Administration (EIA), Independent statistics & Analysis, 2014.
 [2] T. Shear, “Today in energy: February archive,” US Energy Information Administration (EIA), Independent statistics & Analysis, 2014.
 [3] M. H. Albadi and E. F. ElSaadany, “Demand response in electricity markets: An overview,” IEEE Power Engineering Society General Meeting, pp. 1–5, 2007.
 [4] S. Katipamula and N. Lu, “Evaluation of residential HVAC control strategies for demand response programs,” Transactions on American Society of Heating, Refrigerating and AirConditioning Engineers, vol. 112, no. 1, pp. 535–546, 2006.
 [5] Y. Y. Hsu and C. C. Su, “Dispatch of direct load control using dynamic programming,” IEEE Transactions on Power Systems, vol. 6, no. 3, pp. 1056–1061, 1991.
 [6] D. Wei and N. Chen, “Air conditioner direct load control by multipass dynamic programming,” IEEE Transactions on Power Systems, vol. 10, no. 1, pp. 307–313, 1995.
 [7] N. Li, L. Chen, and S. Low, “Optimal demand response based on utility maximization in power networks,” Proceedings of IEEE Power and Energy Society General Meeting, pp. 1–8, 2011.
 [8] W.J. Ma, V. Gupta, and U. Topcu, “On distributed charging control of electric vehicle with power network capacity constraints,” in American Control Conference, Portland, 2014.
 [9] EPRIDOE, “Handbook of energy storage for transmission and distribution applications,” 1001834, EPRI, Palo Alto, CA, and the U.S. Department of Energy, Washington, DC, 2003.
 [10] J. E. Braun, T. Lawrence, C. Klaassen, and J. House, “Demonstration of load shifting and peak load reduction with control of building thermal mass,” Teaming for Efficiency: Commercial buildings: technologies, design, performance analysis, and building industry trends, vol. 3, p. 55, 2002.
 [11] J. E. Braun, “Load control using building thermal mass,” Journal of solar energy engineering, vol. 125, no. 3, pp. 292–301, 2003.
 [12] J. E. Braun, K. W. Montgomery, and N. Chaturvedi, “Evaluating the performance of building thermal mass control strategies,” HVAC&R Research, vol. 7, no. 4, pp. 403–428, 2001.
 [13] K. Keeney and J. E. Braun, “Application of building precooling to reduce peak cooling requirements,” ASHRAE transactions, vol. 103, no. 1, pp. 463–469, 1997.
 [14] J. E. Braun and K. H. Lee, “Assessment of demand limiting using building thermal mass in small commercial buildings,” Transactions on American Society of Heating, Refrigerating and AirConditioning Engineers, vol. 112, no. 1, pp. 547–558, 2006.
 [15] R. T. Guttromson, D. P. Chassin, and S. E. Widergren, “Residential energy resource models for distribution feeder simulation,” in IEEE Power Engineering Society General Meeting, vol. 1, 2003.
 [16] G. Henze, C. Felsmann, and G. Knabe, “Evaluation of optimal control for active and passive building thermal storage,” International Journal of Thermal Sciences, vol. 43, no. 2, pp. 173–183, 2004.
 [17] A. Kelman and F. Borrelli, “Bilinear model predictive control of a HVAC system using sequential quadratic programming,” in IFAC World Congress, 2011.
 [18] L. Lu, W. Cai, Y. S. Chai, and L. Xie, “Global optimization for overall HVAC systems—part II problem solution and simulations,” Energy Conversion and Management, vol. 46, no. 7, pp. 1015–1028, 2005.
 [19] B. ArguelloSerrano and M. VelezReyes, “Nonlinear control of a heating, ventilating, and air conditioning system with thermal load estimation,” IEEE Transactions on Control Systems Technology, vol. 7, no. 1, pp. 56–63, 1999.
 [20] P. Constantopoulos, F. C. Schweppe, and R. C. Larson, “ESTIA: A realtime consumer control scheme for space conditioning usage under spot electricity pricing,” Computers & operations research, vol. 18, no. 8, pp. 751–765, 1991.
 [21] F. Oldewurtel, A. Ulbig, A. Parisio, G. Andersson, and M. Morari, “Reducing peak electricity demand in building climate control using realtime pricing and model predictive control,” in IEEE Conference on Decision and Control 49th, pp. 1927–1932, 2010.
 [22] T. Y. Chen, “Realtime predictive supervisory operation of building thermal systems with thermal mass,” Journal of Energy and Buildings, vol. 33, no. 2, pp. 141–150, 2001.
 [23] J. E. Braun, “Reducing energy costs and peak electrical demand through optimal control of building thermal storage,” ASHRAE transactions, vol. 96, no. 2, pp. 876–888, 1990.
 [24] M. KintnerMeyer and A. F. Emery, “Optimal control of an HVAC system using cold storage and building thermal capacitance,” Energy and Buildings, vol. 23, no. 1, pp. 19–31, 1995.
 [25] “Proposed adjustments to SRP’s standard electric price plans,” Salt River Project: Agricultural Improvement and Power District, July, 2012.
 [26] R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming. Princeton University Press, 1962.
 [27] D. J. Rumolo, “Manager, regulation and pricing,” Arizona Public Service Company (APS), JULY, 2012.
 [28] M. A. Patterson and A. V. Rao, “GPOPS II: A MATLAB software for solving multiplephase optimal control problems,” ACM Transactions on Mathematical Software, vol. 39, no. 3, pp. 1–41, 2013.