Optimal Thermostat Programming and Optimal Electricity Rates for Customers with Demand Charges
We consider the coupled problems of optimal thermostat programming and optimal pricing of electricity. Our framework consists of a single user and a single provider (a regulated utility). The provider sets prices for the user, who pays for both total energy consumed ($/kWh, including peak and off-peak rates) and the peak rate of consumption in a month (a demand charge) ($/kW). The cost of electricity for the provider is based on a combination of capacity costs ($/kW) and fuel costs ($/kWh). In the optimal thermostat programming problem, the user minimizes the amount paid for electricity while staying within a pre-defined temperature range. The user has access to energy storage in the form of thermal capacitance of the interior structure of the building. The provider sets prices designed to minimize the total cost of producing electricity while meeting the needs of the user. To solve the user-problem, we use a variant of dynamic programming. To solve the provider-problem, we use a descent algorithm coupled with our dynamic programming code - yielding optimal on-peak, off-peak and demand prices. We show that thermal storage and optimal thermostat programming can reduce electricity bills using current utility prices from utilities Arizona Public Service (APS) and Salt River Project (SRP). Moreover, we obtain optimal utility prices which lead to significant reductions in the cost of generating electricity and electricity bills.
To ensure the reliability of power networks, utility companies must maintain an uninterrupted balance between power generation and demand. In some ways this problem is becoming easier. Partially due to the development of energy-efficient appliances and new materials for insulation, US electricity demand has plateaued  and is expected to remain flat (less than 1% growth) for the indefinite future (see Fig. 1(a)). The result is reduced reliance on carbon-producing fossil fuels. However, a new problem has arisen - partially due to increasing use of intermittent renewable energy sources such as distributed solar and wind - in that demand peaks continue to grow. Specifically, as per the US Energy Information Administration (EIA) , the ratio of peak demand to average demand has increased dramatically over the last 20 years, setting records of 1.89 in New England in 2012 and 1.96 in California in 2010 (see Fig. 1(b)). Because most utilities are required to maintain generating capacity as determined by peak demand, yet typically only charge customers for total consumption, there is real concern about the viability of existing business models. For example, due to net metering, a typical residential solar customer might have negative consumption during the day and positive consumption during the evening and morning. Such as customer might pay nothing for electricity while contributing substantially to the costs incurred by the utility. In response to this problem, many utilities have sought to halt or reverse growth of the net-metering framework - a process which has met with some limited success.
In this paper, we look at pricing strategies for reducing peak load while retaining the incentives necessary to create a robust distributed renewable sector. Naturally, utilities have been studying this problem for some time and with the widespread adoption of smart-metering (95% in Arizona), have begun to implement such strategies at scale. Examples of this include on-peak, off-peak and super-peak pricing - rate plans wherein the energy price ($/kWh) depends on the time of day . By charging more during peak hours, utilities encourage conservation or deferred consumption during hours of peak demand. More aggressive strategies which have emerged recently include voluntary on-peak demand-limiting programs wherein customers are rewarded for reducing consumption when requested to do so by the utility . A yet more aggressive strategy is direct load control [5, 6] wherein Heating, Ventilating, and Air Conditioning (HVAC) or other appliances are under the direct control of the utilities and can be deferred or deactivated at will. Quite recently, some utilities have introduced demand charges for residential customers. These charges are not based on energy consumption, but rather the maximum rate of consumption ($/kW) over a billing period. While such charges more accurately reflect the cost of generation for the utilities, in practice the effects of such charges on consumption are not well-understood - meaning that the magnitude of the demand charge must be set in an ad-hoc manner (typically proportional to marginal cost of generation).
An alternative approach to reducing peaks in demand is to use energy storage. In this scenario, batteries, pumping and retained heat are used during periods of low demand to create reservoirs of energy which can then be tapped during periods of high demand - thus reducing the need to increase maximum generating capacity. Indeed, the optimal usage of energy storage in a smart-grid environment with dynamic pricing has been recently studied in, for example, . See  for optimal distributed load scheduling in the presence of network capacity constraints. However, to date the high marginal costs of storage infrastructure relative to incentives/marginal cost of additional generating capacity have limited the widespread use of energy storage by consumers/utilities . As a cost-free alternative to direct energy storage, it has been demonstrated experimentally [10, 11] and in-silico [12, 13] that the interior structure of buildings and appliances can be exploited as a passive thermal energy storage system to reduce the peak-load of the HVAC. A typical strategy - known as pre-cooling - is to artificially cool the interior thermal mass (e.g., walls and floor) during periods of low demand. Then, during periods of high demand, heat absorption by these cool interior structures supplements or replaces electricity which would otherwise be consumed by the HVAC. Quantitative assessment of the effect of pre-cooling on demand peak and electricity bills has been evaluated in, e.g.,  and sun2013peak . It is important to note, however, that ad-hoc strategies such as pre-cooling are only economical when using differential on-peak and off-peak pricing or demand charges.
The goal of this paper is two-fold. First, we consider optimal HVAC usage for a consumer with fixed on-peak, off-peak and demand charges and model passive thermal energy storage using the heat equation. For a given range of acceptable temperatures and using typical data for exterior temperature, we pose the optimal thermostat programming problem as a constrained optimization problem and present a Dynamic Programming (DP) algorithm which is guaranteed to converge to the solution. This yields the temperature set-points which minimize the monthly electricity bill for the consumer. After solving the thermostat programming problem, we use this solution as a model of user behaviour in order to quantify the consumer response to changes in on-peak rates, off-peak rates, and demand charges. We then apply descent methods to this model in order to determine the prices which minimize the cost-of-generation for the utility. In a case study, we show that the optimal prices are NOT necessarily proportional to the marginal costs of generation - meaning that current pricing strategies may be inefficient.
Before presenting our results, we note that models for thermal energy storage do appear in the optimal thermostat programming literature [4, 14, 15, 16]. Furthermore, there is an extensive literature on thermostat programming for HVAC systems for on-peak/off-peak pricing [17, 18, 19] as well as real-time pricing (prices which are constantly changing) [20, 21, 16, 22] using Model Predictive Control.  and  consider optimal thermostat programming with passive thermal energy storage and on-peak/off-peak rates. Perhaps closest to our work, in , the authors use the concept of deep and shallow mass to create a simplified analogue circuit model of the thermal dynamics of the structure. By using this model and certain assumptions on the gains of the circuit elements,  derives an analytical optimal temperature set-point for the demand limiting period which minimizes the demand peak. This scenario would be equivalent to minimizing the demand charge while ignoring on-peak or off-peak rates. Again, referring to  and subsequent publications, there has been some excellent work on optimal pricing (albeit without demand charges) for energy storage using batteries in an unregulated electricity market using a social welfare model. This paper differs from existing literature in that it: 1) Considers demand charges (demand charges are far more effective at reducing demand peaks than dynamic pricing) 2) Uses a PDE model for thermal storage (yields a more accurate model of thermal storage) 3) Uses a regulated model for the utility. Although unregulated utility models are popular, the fact is that most US utilities remain regulated.
Ii Problem Statement
In this section, we first define a model of the thermodynamics which govern heating and cooling of the interior structures of a building. We then use this model to pose the user-level (optimal thermostat programming) problem in Sections II-B as minimization of a monthly electricity bill (with on/peak, off-peak and demand charges) subject to constraints on the interior temperature of the building. Finally, we use this map of on-peak, off-peak and demand prices to consumption to define the utility-level problem in Section II-C as minimizing the cost of producing electricity.
Ii-a A Model for the Building Thermodynamics
To model heat storage in interior walls and floors of a building, we use the one-dimensional unsteady heat conduction equation
where represents the temperature distribution in the interior walls/floor with nominal width and where is the coefficient of thermal diffusivity. Here is the coefficient of thermal conductivity, is the density and is the specific heat capacity. The wall is coupled to the interior air temperature using Dirichlet boundary conditions, i.e., , where represents the interior temperature which we assume can be controlled instantaneously by the thermostat. We model the heat loss through the exterior walls by the linear heat sink as
where is the outside temperature and is the thermal resistance of the ext. walls, is the nominal width of ext. walls, is the coefficient of thermal conductivity and is the nominal area of the ext. walls. The heat/energy flux through the surface of the interior wall is modelled as
where is the thermal capacitance of interior walls and is the nominal area of interior walls. By conservation of energy, the power required from the HVAC to maintain the interior air temperature is
See Fig. 2 for a depiction of the model.
Ii-B User-Level Problem: Optimal Thermostat Programming
In this section, we define the problem of optimal thermostat programming. We first divide each day into three periods: off-peak hours from 12 AM to with electricity price ; on-peak hours beginning at and ending at with electricity price ; and off-peak hours from to 12 AM with electricity price . In addition to the on-peak and off-peak charges, we consider a monthly charge which is proportional to the maximum rate of consumption during the peak hours. The proportionality constant is called the demand price . Given prices and , the total cost of consumption (daily electricity bill) is divided as
where is the energy cost, is the demand cost and
The energy cost is
where if and otherwise. That is, and correspond to the set of on-peak and off-peak sampling times, respectively. The function is a discretized version of (Eqn. (4)):
This is the power consumed by the HVAC, where denotes the external temperature at time-step . If demand charges are calculated monthly, the demand cost for a single day is
We now define the optimal thermostat programming problem at the user-level as minimization of the total cost of consumption as defined in (7), subject to the building thermodynamics (Eqn. (6)) and interior temperature constraints ().
where and are the acceptable bounds on the interior temperature. Note that this optimization problem depends implicitly on exterior temperature through the time-varying function .
Ii-C Utility-Level Optimization Problem
We define the utility-level optimization problem as minimization of the cost of generating electricity such that generation is equal to consumption, and revenue is equal to cost of generation. Let be the amount of electricity produced as a function of time and . First, we consider a linear model of the production cost (adopted from Arizona Public Utility SRP) as
where is the marginal cost of producing the next of energy and is the marginal cost of installing the next of production capacity. Values of the coefficients and for SRP can be found in e.g., . Now define and to be minimizing arguments to the user-level problem defined in (11). Then the constraint that production equals consumption implies . We now define the utility-level optimization problem as minimization of the cost of electricity production subject to equality of production and consumption.
where the last two lines constrain that costs equal revenue (recall is revenue from the users as defined in (7)).
Iii Solving User- and Utility-level Problems
First, we solve the optimal thermostat programming problem using a variant of dynamic programming. This yields consumption as a function of prices . Next, we embed this implicit function in a descent algorithm in order to find prices which minimize the Utility-level optimization problem as formulated in (12). We start by defining a cost-to-go function, . Given , at the final time , we have
Here for simplicity, we use to represent the discretized temperature distribution in the wall. Define prices if and otherwise. Then, we construct the cost-to-go function inductively as
where is the set of allowable inputs at time and state :
Now we present the main result.
To prove Theorem 1, we require the following definitions.
Given , , and such that , define the cost-to-go functions
where is defined as in (9), and and are the time-steps corresponding to start and end of the on-peak hours.
Note that from (8), it is clear that .
We now present a proof for Theorem 1.
Since the cost-to-go function , if we show that
for and for any , where
then it will follow that . For brevity, we denote by , by and we drop the last two arguments of . To show (18), we use induction as follows.
Induction hypothesis: Suppose
for some and for any . Then, we need to prove that
for any . Here, we only prove (19) for the case which . The proofs for the cases and follow the same exact logic.
Assume that . Then, from Definition 1
where . From the principle of optimality  it follows that
From Definition 1, we can write
for any . By using the same logic it can be shown that for any and for any . Therefore, by induction, (18) is true. Thus, .
Iv Numerical Examples and Analysis
In this section, we demonstrate convergence of our algorithm for optimal thermostat programming using electricity prices from APS and temperature data from Phoenix, AZ. In addition, we study the problem of optimal electricity pricing using the marginal cost data from SRP. We ran all the numerical simulations for three consecutive days with time-step , space-step and with building’s parameters listed in Table I. We used an external temperature profile measured for three typical summer days in Phoenix, Arizona (see Fig. 3). For each day, the on-peak period starts at PM and ends at PM. In all scenarios, we used and .
|on-peak ( per )||off-peak ( per )||demand ( per )|
Iv-a Scenario 1: The Effect of Electricity Prices on Peak Demand and Production Costs
In this scenario, we first consider the optimal thermostat programming problem (See (11)) using the electricity prices and as determined by APS  (See Table II). The results of the dynamic programming algorithm are given in Table III as total price paid by the user (we also include the peak demand). For comparison, we have run the same optimal control problem using the general-purpose optimization solver GPOPS . Moreover, we have compared our result with a typical precooling strategy and a naive strategy of setting the temperature to (constant). As can be seen, our algorithm outperforms the heuristic approaches. The power consumption and the temperature setting as a function of time for each strategy can be found in Fig. 4. For convenience, the on-peak and off-peak intervals are indicated on the figure. As can be seen, for APS prices and our building’s parameters, the optimal strategy does not reduce the peak demand with respect to the precooling strategy.
To examine the impact of changes in electricity prices on peak demand, we next chose several different prices corresponding to high, medium and low penalties for peak electricity demand. Again, in each case, our algorithm is compared to GPOPS and a precooling strategy. The results are summarized in Table IV. For each price, the smallest computed production cost and demand peak are typed in bold. The power consumption and the temperature settings as a function of time for the optimal strategy can be found in Fig. 5. For the optimal strategy, notice that by increasing the demand penalty, relative to the low-penalty case, the peak consumption is reduced by 14% and 23% in the medium and high penalty cases respectively. Furthermore, notice that by using the optimal strategy and the high demand-limiting prices, we have reduced the demand peak by 29% with respect to the constant strategy in Table III. Of course, a moderate reduction in peak demand at the expense of large additional energy costs may not be desirable. The question of optimal distribution of electricity prices is discussed in Scenario II.
|temperature setting||Electricity bill ()||demand peak ()|
|Optimal (Theorem 1)||36.58||9.222|
|Prices||Demand-limiting||Production cost||Demand peak|
|Prices||Demand-limiting||Production cost||Demand peak|
|Prices||Demand-limiting||Production cost||Demand peak|
Iv-B Scenario 2: Optimal Thermostat Programming with Optimal Electricity Prices
In this case, we applied Algorithm 1 to find optimal on-peak, off-peak and demand prices under the assumption that the building’s parameters in Table I represent an averaged user. The marginal production costs and are taken as as estimated by SRP. The optimal prices, associated production cost, and associated peak demand are listed in Table V. A typical pricing strategy for SRP and other utilities is to set prices proportional to marginal production costs. The production cost associated with this strategy is also listed in Table V. Notice that the optimal prices are in fact not proportional to the marginal costs of generation.
|Strategy||Production cost||Demand peak|
V Conclusion and Future Work
In this work, we proposed a dynamic-programming-based algorithm for solving the optimal control problem associated with thermostat programming in the presence of distributed thermal energy storage in interior structures. We used a pricing model which is a combination of on-peak, off-peak and demand charges. Using the solution to this optimal control problem as a model of behavior, we determined the optimal prices which minimize production costs for the utility. We concluded that optimal thermostat programming can significantly reduce electricity bills and demand peak by taking advantage of energy storage using thermal mass. Furthermore, we showed that the typical approach to electricity pricing is suboptimal at reducing production costs. The results of this paper assume a rational consumer and accurate models of both the daily temperature and utility production costs.
This material is based upon work supported by the National Science Foundation under Grant No. CMMI-1301851. We would like to thank Salt River Project (SRP) for providing us with their suggestions and data.
-  J. J. Conti, “Annual energy outlook 2014 with projections to 2040,” US Energy Information Administration (EIA), Independent statistics & Analysis, 2014.
-  T. Shear, “Today in energy: February archive,” US Energy Information Administration (EIA), Independent statistics & Analysis, 2014.
-  M. H. Albadi and E. F. El-Saadany, “Demand response in electricity markets: An overview,” IEEE Power Engineering Society General Meeting, pp. 1–5, 2007.
-  S. Katipamula and N. Lu, “Evaluation of residential HVAC control strategies for demand response programs,” Transactions on American Society of Heating, Refrigerating and Air-Conditioning Engineers, vol. 112, no. 1, pp. 535–546, 2006.
-  Y. Y. Hsu and C. C. Su, “Dispatch of direct load control using dynamic programming,” IEEE Transactions on Power Systems, vol. 6, no. 3, pp. 1056–1061, 1991.
-  D. Wei and N. Chen, “Air conditioner direct load control by multi-pass dynamic programming,” IEEE Transactions on Power Systems, vol. 10, no. 1, pp. 307–313, 1995.
-  N. Li, L. Chen, and S. Low, “Optimal demand response based on utility maximization in power networks,” Proceedings of IEEE Power and Energy Society General Meeting, pp. 1–8, 2011.
-  W.-J. Ma, V. Gupta, and U. Topcu, “On distributed charging control of electric vehicle with power network capacity constraints,” in American Control Conference, Portland, 2014.
-  EPRI-DOE, “Handbook of energy storage for transmission and distribution applications,” 1001834, EPRI, Palo Alto, CA, and the U.S. Department of Energy, Washington, DC, 2003.
-  J. E. Braun, T. Lawrence, C. Klaassen, and J. House, “Demonstration of load shifting and peak load reduction with control of building thermal mass,” Teaming for Efficiency: Commercial buildings: technologies, design, performance analysis, and building industry trends, vol. 3, p. 55, 2002.
-  J. E. Braun, “Load control using building thermal mass,” Journal of solar energy engineering, vol. 125, no. 3, pp. 292–301, 2003.
-  J. E. Braun, K. W. Montgomery, and N. Chaturvedi, “Evaluating the performance of building thermal mass control strategies,” HVAC&R Research, vol. 7, no. 4, pp. 403–428, 2001.
-  K. Keeney and J. E. Braun, “Application of building precooling to reduce peak cooling requirements,” ASHRAE transactions, vol. 103, no. 1, pp. 463–469, 1997.
-  J. E. Braun and K. H. Lee, “Assessment of demand limiting using building thermal mass in small commercial buildings,” Transactions on American Society of Heating, Refrigerating and Air-Conditioning Engineers, vol. 112, no. 1, pp. 547–558, 2006.
-  R. T. Guttromson, D. P. Chassin, and S. E. Widergren, “Residential energy resource models for distribution feeder simulation,” in IEEE Power Engineering Society General Meeting, vol. 1, 2003.
-  G. Henze, C. Felsmann, and G. Knabe, “Evaluation of optimal control for active and passive building thermal storage,” International Journal of Thermal Sciences, vol. 43, no. 2, pp. 173–183, 2004.
-  A. Kelman and F. Borrelli, “Bilinear model predictive control of a HVAC system using sequential quadratic programming,” in IFAC World Congress, 2011.
-  L. Lu, W. Cai, Y. S. Chai, and L. Xie, “Global optimization for overall HVAC systems—-part II problem solution and simulations,” Energy Conversion and Management, vol. 46, no. 7, pp. 1015–1028, 2005.
-  B. Arguello-Serrano and M. Velez-Reyes, “Nonlinear control of a heating, ventilating, and air conditioning system with thermal load estimation,” IEEE Transactions on Control Systems Technology, vol. 7, no. 1, pp. 56–63, 1999.
-  P. Constantopoulos, F. C. Schweppe, and R. C. Larson, “ESTIA: A real-time consumer control scheme for space conditioning usage under spot electricity pricing,” Computers & operations research, vol. 18, no. 8, pp. 751–765, 1991.
-  F. Oldewurtel, A. Ulbig, A. Parisio, G. Andersson, and M. Morari, “Reducing peak electricity demand in building climate control using real-time pricing and model predictive control,” in IEEE Conference on Decision and Control 49th, pp. 1927–1932, 2010.
-  T. Y. Chen, “Real-time predictive supervisory operation of building thermal systems with thermal mass,” Journal of Energy and Buildings, vol. 33, no. 2, pp. 141–150, 2001.
-  J. E. Braun, “Reducing energy costs and peak electrical demand through optimal control of building thermal storage,” ASHRAE transactions, vol. 96, no. 2, pp. 876–888, 1990.
-  M. Kintner-Meyer and A. F. Emery, “Optimal control of an HVAC system using cold storage and building thermal capacitance,” Energy and Buildings, vol. 23, no. 1, pp. 19–31, 1995.
-  “Proposed adjustments to SRP’s standard electric price plans,” Salt River Project: Agricultural Improvement and Power District, July, 2012.
-  R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming. Princeton University Press, 1962.
-  D. J. Rumolo, “Manager, regulation and pricing,” Arizona Public Service Company (APS), JULY, 2012.
-  M. A. Patterson and A. V. Rao, “GPOPS- II: A MATLAB software for solving multiple-phase optimal control problems,” ACM Transactions on Mathematical Software, vol. 39, no. 3, pp. 1–41, 2013.