1 Introduction

This paper proposes a concept of generalized energy storage (GES) to facilitate the integration of large-scale heterogeneous flexible resources with electric/thermal energy storage capacity to participate in multiple markets. First, a generalized state variable referred to as degree of satisfaction (DoS) is defined, and dynamic models with a unified form are derived for different types of GESs. Second, a real-time market-based coordination framework is proposed to facilitate control, and ensure user privacy and device security. Demand curves of different GESs are then developed based on DoS to express their demand urgencies as well as flexibilities. Furthermore, a low-dimensional aggregate dynamic model of a GES cluster is derived thanks to the DoS-equality control feature provided by the design of demand curve. At last, an optimization model for a large-scale GESs to participate in both the energy market and regulation market is established based on the aggregate model. Simulations results demonstrate that the optimization algorithm could effectively reduce the total cost of an aggregator. Additionally, the proposed coordination method has high tracking accuracy and could well satisfy users’ diversified power demand.

Generalized energy storage; Coordination control; Aggregate dynamic model; Market-based control; Multiple markets

1 \pubvolumexx \issuenum1 \articlenumber5 2018 \copyrightyear2018 \historyReceived: date; Accepted: date; Published: date \TitleAggregating Large-Scale Generalized Energy Storages to Participate in Energy Market and Regulation Market \AuthorYao Yao , Peichao Zhang *, Sijie Chen \AuthorNamesYao Yao and Peichao Zhang and Sijie Chen \corresCorrespondence: pczhang@sjtu.edu.cn

1 Introduction

The increasing integration of distributed energy resources (DERs) poses threat on stability and reliability of power system operation. With the development of smart grid technologies, the control of flexible loads, e.g., electric energy storage (EES), thermostatically control load (TCL) and electric vehicle (EV), has become a promising research field to address the problem brought by DERs Hao et al. (2018); Mingshen et al. (2018). Due to the great variety, large scale, wide distribution and small individual capacity of flexible loads, load aggregators (LAs) are required to aggregate and dispatch such loads and provide flexible services to power grid. The coordination strategies have been widely studied. For example, a three-step EV charging algorithm is presented in Vandael et al. (2013) to minimize charging cost. In Hao et al. (2014), a priority-stack-based method is put forward for fixed-frequency air-conditioner (FFA) fleet to track automatic generation control signal. Literature Song et al. (2018) establishes physical model of inverter air-conditioners (IVAs) and proposes a hierarchical control framework. A unified state model for EVs and TCLs is developed in Mingshen et al. (2018). In Hao et al. (2018), a generalized battery model is presented to describe the operational characteristics of commercial/residential building HVAC and energy storage system, based on which optimal control methods are proposed to provide various services.

However, most of these methods have the following disadvantages. First, most approaches can only be applied to a certain type of load Vandael et al. (2013); Hao et al. (2014); Song et al. (2018). An LA should provide different interfaces to integrate different types of resources, which makes it difficult for the LA to fully utilize flexibility of various flexible loads and increase the control cost. Second, many literatures use centralized control method based on direct load control Hao et al. (2014); Mingshen et al. (2018); Hao et al. (2018). An LA should collect detailed parameters of all controlled loads and specify the response power of each load, which has heavy computational burden and communication traffic, and may lead to privacy issues and device security problems. Such methods may not be suitable for coordination of large-scale flexible loads. Some papers develop unified models to coordinate different types of resources Mingshen et al. (2018); Hao et al. (2018), while they depend on centralized control as they mainly focus on individual behaviour rather than aggregate performance.

The objective of LAs is to provide flexible services to power grid. The method in Hao et al. (2014) could provide reliable regulation services. Literature Mingshen et al. (2018) demonstrates its effectiveness in power fluctuation smoothing. In Vandael et al. (2013); Song et al. (2018), LAs participate in optimal dispatch and achieve benefits from energy markets. However, the above researches only consider a single market. It is pointed in Namor et al. (2018) that since different services have different requirements, participating in multiple markets helps to better utilize the control flexibility of resources and obtain more benefits. To stack multiple services, energy storage systems are controlled in Dubey et al. (2017) to simultaneously participate in N-1 contingency requirement, voltage management and frequency regulation. An optimal control method is presented in Wu et al. (2015) for energy storages to provide grid services including energy arbitrage, balancing service, capacity value and distribution system deferral, and outage mitigation. Cheng and Powell (2018) and Anderson and El Gamal (2017) introduce a Markov decision process model and a stochastic control method respectively to co-optimize battery storage for energy arbitrage and frequency regulation. However, these papers only consider energy storages. Although literature Hao et al. (2018) considers coordination of various flexible loads to provide multiple grid services, it has to solve some centralized optimization problems, which may lead to a high computational cost.

To address the above problems, a unified modelling method and coordination strategy for generalized energy storages (GESs) is proposed. It has unified information interface and control method, and requires a relatively low communication and computation cost, which is helpful for an LA to conduct coordination control over large-scale GESs. The contributions of our work are threefold:

  1. Dynamic models with a unified form are developed for heterogenous GESs based on a generalized state variable referred to as degree of satisfaction (DoS).

  2. A real-time coordination framework based on the market equilibrium mechanism is presented to allocate aggregate power to individual GESs. General demand curves are constructed under the framework to achieve equal degree of satisfaction and meet users’ diversified requirements.

  3. A low-dimensional aggregate dynamic model for a GES cluster is derived and used in an optimization model for an LA to participate in both the energy market and regulation market.

The rest of this paper is organized as follows. Section 2 introduces the dynamic model of different GESs. In section 3, the market-based real-time coordination framework is proposed and demand curve construction methods for GESs are presented. Section 4 derives the aggregate dynamic model for a GES cluster. In section 5, the optimization problem that considers both energy and regulation markets is given. Section 6 shows the simulation results which demonstrate the effectiveness of our method. Finally, section 7 summarizes our contribution and future work.

2 Dynamic Models for GESs

This paper focuses on four typical types of GESs, i.e., EES, EV, inverter air-conditioner (IVA) and fixed-frequency air-conditioner (FFA) since they account for a large share of flexible resources in the demand side. These resources are called GES for that they all can store energy, i.e., electric energy or cold/thermal energy, thus their power consumption could be adjusted without affecting the user satisfaction. Meanwhile, they have similar dynamic characteristics, which makes it possible to establish a unified physical model for them. For example, there have been a few literatures that develop battery modelling method for the thermal energy storage loads. However, most researches only focus on FFAsHao et al. (2013); Mathieu et al. (2015); Sanandaji et al. (2014). A battery-type reduced-order model for building HVAC system is proposed in Hughes et al. (2016), but it only considers the dynamics of individual load. Literature Song et al. (2018) studies the battery modelling for individual IVA and aggregated IVA cluster, while the aggregating strategy would lead to the result that the IVA cluster’s regulation ability could not be fully utilized. This paper will put forward a unified dynamic model for different types of GESs, and study both individual model (introduced in this section) and aggregate model (detailed in section.4).

2.1 Degree of Satisfaction, DoS

Before establishing the dynamic models, a dimensionless state variable referred to as degree of satisfaction (DoS) is defined for GESs, with the following purposes:

  1. It could be used to measure the user satisfaction. The range of DoS is set to [-1,1], and the closer DoS is to 0, the higher user satisfaction is.

  2. DoS could reflect a GES’s state of energy: DoS equalling 0 indicates that the stored energy is at the expected level, while DoS close to 1 means the stored energy is near the allowed range.

  3. Since a GES can deviate from its ideal state (DoS=0) to provide services, the DoS can be used to quantify its current flexility, i.e., a DoS value close to 0 implies a high flexility reservation.

  4. Finally, as DoS is a generalized index, it can be used to establish a unified dynamic model for various GESs.

2.2 Derivation of Dynamic Models

2.2.1 Electric Energy Storage (EES)

Ignoring the charge/discharge efficiency, the dynamic model of an EES is given by


where and denote the electric energy and power of EES at time , respectively ( when EES is charging); is the control cycle.

Variable is hereafter used to denote DoS. Definition of EES’s DoS is given by


Substituting Eq.(2) and into Eq.(1), the following dynamic model can be derived:


where denotes the nominal capacity of EES .

2.2.2 Electric Vehicle (EV)

The physical model of an EV is


where and denote the electric energy and power of EV at time , respectively; is the charge efficiency.

According to the current charger technology, we focus on the prevailing EV type, which operates at two discrete states: idle and charging with a fixed rate Binetti et al. (2015); Floch et al. (2016). Let denote the target energy of EV at user-specified departure time , and denote the energy at the time EV is connected into power grid . The average power required to charge EV to can be calculated by


If EV charges at , the expected energy profile is


An EV can provide its flexibility by deviating from . Referring to the hysteretic model in Callaway and Hiskens (2010), DoS of EV is defined as


where denotes the nominal capcity of EV ; denotes the energy deadband, which limits the error between and the actual energy at within .

Combined with Eq.(5)-(7), Eq.(4) can be transformed into


2.2.3 Inverter Air-conditioner (IVA)

Without loss of generality, cooling air-conditioners are studied in this paper. The thermal dynamic process is modelled by a first-order differential equation Hao et al. (2014); Song et al. (2018) as


where and denote the indoor air temperature and outdoor temperature at time , respectively; is the heat rate of IVA ; the thermal parameter .

The analytical solution of Eq.(9) in recursive form is


The electrical model of an IVA adopts the simplified linear model in Song et al. (2018), which is given by


where denotes the electric power of IVA ; is the operation frequency of the compressor; , , and are coefficients.

DoS of an IVA is defined as


where denotes the setpoint; denotes the allowed temperature deviation.

Combined with Eq.(11) and Eq.(12), Eq.(10) can be transformed into


where , , .

2.2.4 Fixed-Frequency Air-conditioner (FFA)

The thermal model of an FFA can also be established by Eq.(9). Its electrical model is given by


where is the coefficient of performance. FFA is ON/OFF controlled load, so its electric power equals to nominal power when it is ON and when it is OFF.

FFA’s definition of DoS is identical to that of IVA, which is given by Eq.(12). Combined with Eq.(12) and Eq.(14), Eq.(10) can be transformed to


where .

2.3 Unified Dynamic Model

The dynamic models of the above GESs can now be represented in a unified form:


where , , are coefficients of the unified dynamic model.

Mappings between DoS and the original variables of different GESs are summarized in Fig.1. This paper would study a real-time coordination method of GESs, which could make DoS of individual GESs (or average DoS of a cluster of GESs) approximately equal. Such control characteristic is referred to as DoS-equality control in this paper and will be detailed in the next section.

Figure 1: DoS of different GESs.

3 Real-time Coordination Method of Large-Scale GESs

3.1 DoS-equality Control Based on Market Equilibrium Mechanism

Market equilibrium mechanism Yao and Zhang (2017) is introduced in this paper to coordinate large-scale GESs. Main stages of the coordination method are:

(1) Bidding: Each GES expresses its urgency and flexibility by constructing a demand curve. The demand curve is denoted by in this paper, which is a non-increasing function.

(2) Aggregating and clearing: LA collects demand curves from all GESs and forms the aggregate demand curve , where is the number of controlled GESs; Assume the aggregate target power is , then LA can calculate the clearing price by .

(3) Disaggregating: LA broadcasts to all GESs. Each GES responds to locally according to its demand curve. The response power of GES can be obtained by .

Following the above steps, the aggregate target power can be allocated among the GESs, thus realizing accurate power tracking. It should be noted that: (1) The target power depends on applications. In this paper, it will be determined by an optimization problem to be discussed in the next section; (2) The clearing price is only a control signal rather than a price signal. It is dimensionless and its range is set to [-1,1]. Therefore, it is called "virtual price" in this paper.

The proposed market-based coordination method highlights the following advantages:

  1. It improves the autonomy of the GES. Each GES can convert its private information, e.g., user preferences, current adjustable range and security constraints, into demand curve. Since demand curves of all GESs have a unified form, it can shield the differences among various GESs and effectively protect user privacy. Besides, the LA does not have permission to directly control GES, which improves device security.

  2. It simplifies the control of the LA. An LA does not need to specify each GES’s type and is able to coordinates various GESs via an identical signal, i.e., the virtual price signal , which significantly reduces control complexity and requirement of communication bandwidth.

In addition, the proposed method can realize the DoS-equality control to obtain the following advantages:

  1. GESs could have same degree of user satisfaction regardless of the resource type or capacity, which ensures control fairness. In addition, since DoS reflects a GES’s state of energy, the DoS-equality control could avoid some GESs going beyond their adjustable range prematurely, thus better utilizing the regulation ability of a GES cluster.

  2. The unique DoS of a GES cluster can be a state variable to derive an aggregate dynamic model, making it possible to treat the whole GES cluster as a virtual storage, which will be detailed in section 4.

According to the operation characteristic, the GES can be further classified into two types: GES operating at continuous power (CP-GES) and GES operating at discrete power with discrete states (DP-GES). A CP-GES, e.g., an EES or an IVA, is able to keep its DoS at the desired value by adjusting its operating power, while for a DP-GES, e.g., an EV, an FFA or an electric heater, its DoS generally fluctuates within the allowed range. Besides, the state switching frequency of a DP-GES should generally be controlled to prolong the device’s lifetime.

The key of the proposed coordination method is the construction of demand curves for different GESs, which will be introduced in the following subsections.

3.2 Demand Curve of CP-GES

3.2.1 Demand Curve

Let denote the current DoS of GES . The following construction principle of demand curve is established in this paper:


where denotes the electric power required to maintain the current over a control cycle.

Eq.(17) is used to realize the DoS-equality control feature. For explanation, the clearing price is assumed to be constant. When , the response power is higher than , leading to the decrease of ; when , is lower than , leading to the increase of ; when , equals , keeping unchanged. Therefore, values of all GESs following the principle in Eq.(17) will approach the control signal .

Based on Eq.(17), the demand curve of CP-GES is constructed as shown in Fig.2. Note again that the price herein is virtual and is limited between -1 and 1. The demand curve consists of 5 key points. The anchor point A could satisfy the condition in Eq.(17). Points B and C are used to keep the DoS within limits when responding to any clearing price . Thus, and are minimum and maximum power that would not make DoS go beyond limit over a certain period of time (which is set 5min in this paper), without considering operation constraints. Points D and E which lies on line AB and line AC respectively are further introduced to guarantee the response power would not exceed operational constraints. Thus, and are minimum and maximum power that a GES could operate at in current control cycle. The calculation method of the above characteristic power will be detailed in the next section.

Figure 2: Demand curve of a CP-GES.

3.2.2 Characteristic Power

The EES and IVA are typical CP-GESs. They can both adopt the demand curve shown in Fig.2.

An EES’s characteristic power can be calculated by


where denotes the nominal charging/discharging power; and are charge/discharge efficiency, respectively; and are the minimum and maximum allowed energy, respectively; is the stored energy at time .

IVA’s characteristic power can be calculated by


where and denotes the minimum and maximum power; the function obtains the electric power that makes the indoor air temperature change from current value to the target temperature over a period of time . The derivation of is detailed in Appendix A.

3.3 Demand Curve of DP-GES

3.3.1 Demand Curve

A DP-GES typically has two states, i.e., ON and OFF states. The proposed demand curve for a DP-GES is illustrated in Fig.3(a), where denotes the operating power when the GES is ON, and is transformed from its DoS value by offsetting and then normalizing:


To explain the principle, the aggregate demand curve of a cluster of DP-GESs is shown in Fig.3(b), which is obtained by sorting DP-GESs in descending order of . As can be seen, the proposed bidding strategy can divide DP-GESs into two groups according to their operation states, i.e., an ON group in the upper half-plane and an OFF group in the lower half-plane, and achieve the following purposes:

(1) For DP-GESs in the same group, a DP-GES’s reflects its power consumption priority. The higher is, the higher probability to maintain or switch to ON state is, and vice versa.

(2) A DP-GES in the ON group always has a higher than that in the OFF group, which gives high priority for DP-GESs to maintain their current states, thus avoiding frequent switching.

Figure 3: Demand curve of DP-GES.

To explain the change rule of a single DP-GES’s DoS, the clearing price is assumed to be constant over a period of time. Note that the state change of a DP-GES may be triggered either by its value, or by to ensure comfort. The trajectory of and at two different , i.e., and , are illustrated in Fig.4(a) and Fig.4(b), and the following laws can be found: when , ranges in [-1+2,1]; when , ranges in [-1,1+2], which means that the DoS value fluctuates within a symmetrical range around . Therefore, if DoS values of DP-GESs are assumed to be uniformly distributed in such range, the average DoS of the cluster equals .

Figure 4: and of a DP-GES at different .

3.3.2 Characteristic Power

EV and FFA are typical DP-GESs. They can both adopt the demand curve in Fig.3(a).

An EV’s characteristic power is


where denotes the EV’s nominal charging power.

An FFA’s characteristic power is


3.4 Locked State

In certain control cycles, a GES may get into the locked state, which means it should maintain its current operating state and power. There are two typical situations: (1) To reduce mechanical wear and protect the device Liu and Shi (2016); Tang et al. (2017), DP-GESs, such as FFA and EV, should satisfy lockout time constraints before they switch state; (2) Restricted by the device capability, response cycle of some GESs may be longer than the real-time control cycle (which is 10s in this paper).

The lockout mechanism can be easily realized in this paper, as a GES can simply submit the following demand curve during the lockout time:


which means the response power of GES maintains its current operating power for any .

4 Aggregate Dynamic Model of a GES Cluster

One significant advantage of our method is that the aggregate dynamic model of a large-scale GES cluster can be easily derived thanks to the DoS-equality control feature, as well as the unified dynamic model for different GESs defined in Eq.(16).

For CP-GESs, add their dynamic models together:


where represents the set of CP-GESs.

Under the DoS-equality control, DoS of all CP-GESs becomes equal. Let denote DoS of the CP-GES cluster, and denote the aggregate power. The aggregate dynamic model of CP-GESs can then be derived as


For DP-GESs, to facilitate analysis, we assume coefficients of their dynamic models to be equal first. Add their dynamic models together, and we obtain


where represents the set of DP-GESs.

The instantaneous DoS value of each DP-GES is a random variable. Denote the average DoS of DP-GESs as , and denote the aggregate power as . Then Eq.(26) can be written as:


where denotes the number of DP-GESs.

Under the DoS-equality control, of CP-GESs and of DP-GESs tend to be equal (both equal ), thus can both be denoted by . Therefore, Eq.(25) and Eq.(27) can be added together:


where denotes the total number of GESs; is the aggregate power.

Eq.(28) can be represented in a compact form as


The above derivation is based on the assumption that all DP-GESs have equal model coefficients. For those with different coefficients, we can first divide them into different groups according to their coefficients, then aggregate each group using Eq.(27), and finally form the aggregate dynamic model of all the GESs (including CP-GESs and DP-GESs) by Eq.(28).

The aggregate dynamic model of heterogeneous GESs proposed above has the advantage that the LA could obtain the aggregate model easily by adding model coefficients of all GESs with no need to identify GES’s type, thus having a low computational cost. Furthermore, from the control aspect, the low-dimensional aggregate model greatly reduces the complexity of the optimization problem, which will be discussed in the next section.

5 Application

5.1 Optimal Multi-Market Flexibility Allocation

An LA can aggregate flexibility of large-scale GESs to provide multiple services to power grid. For example, it can schedule an optimal power consumption profile according to the electricity price of energy market Song et al. (2018); Hao et al. (2018); Zhang et al. (2017), as the profile in Fig.5. LA can provide other ancillary services at the same time to obtain higher benefits Hao et al. (2018); Zhang et al. (2017), e.g., responding to the regulation signal in Fig.5.

The regulation signal can be decomposed into the low frequency part denoted by regA, and the high frequency part denoted by regD Xu et al. (2016). Literature Zechun et al. (2014) analyses the regulation signal of a certain power grid, and finds that the high frequency part could account for up to 30%. In this paper, the LA responds to the regD signal considering the following facts: First, the regD signal has zero-mean over a period of time Xu et al. (2016), which could significantly reduce requirements for the capacity of GESs; Second, the extra energy introduced by the regD signal is close to 0, thus having little impact on electricity bills; Third, if regulation payments are determined by the performance-based policy used in PJM regulation market Aho et al. (2015), the LA could obtain high benefits.

Considering both the energy and regulation markets, the target power of an LA is given by


where is the hourly scheduled power, which determines the bill paid to the energy market; regD is the regulation signal normalized to [0,1]; denotes the contracted regulation capacity.

Figure 5: Participation in energy and regulation market.

Let the optimization cycle be 1 hour, then the LA solves the following convex optimization problem in the th cycle to allocate its flexibility to the two markets:


where , , denote respectively the electricity price, regulation capacity price and regulation mileage price in the th cycle; denotes the statistical value of the regulation performance score defined by PJM Aho et al. (2015), and denotes the statistical value of the regulation mileage Aho et al. (2015); and are the maximum and minimum power of the GES cluster, which are calculated in every optimization cycle as and , where


Note that EESs are able to discharge, thus the LA may sell electricity to power grid at this time. The purchase price and sale price are assumed to be equal in this paper.

The first constraint in Eq.(31) is the aggregate dynamic model of the GES cluster, the second constraint ensures the user satisfaction, and the last three ones constrain the regulation capacity. Thanks to the established aggregate dynamic model, the scale of the GES cluster does not affect the computational complexity of the optimization problem.

To improve user satisfaction, Eq.(31) includes a penalty term , which is defined as


where is a proportionality coefficient, which is assigned 0.1 here; is the daily average electricity price.

5.2 Three-Layer Control Structure

To sum up, this paper develops four models, i.e., a unified dynamic model and a unified demand model for individual GES, as well as an aggregate dynamic model and an optimization model for an LA to approximate dynamics of a GES cluster and participate in multi-markets. These four models are organized in a a three-layer control structure, as illustrated in Fig.6.

Figure 6: Three-layer control structure.

(1) Rolling Optimization Layer

At the beginning of current optimization cycle , the LA gathers model coefficients from GESs and update the aggregate dynamic model according to Eq.(28). Meanwhile, current DoS values of all GESs are collected to update the initial recursive value of by


The optimization problem in Eq.(31) can then be solved, and the optimal scheduled power sequence as well as the regulation capacity sequence are obtained. Implement only the first elements of these optimal sequences, i.e., and in the current optimization cycle, and repeat the above steps each hour. It’s well known that this idea of rolling optimization comes from the model predictive control (MPC) Liu and Shi (2016), which is adopted here to update the aggregate dynamic model iteratively, and to consider constraints in future time slots explicitly.

(2) Real-time Coordination Layer

In each control cycle (10s in this paper), the LA receives the regulation signal regD from the control center, and then calculates the real-time target power according to Eq.(30). The virtual market is cleared according to section 3, and the clearing price is broadcast to each GES.

(3) GES Autonomy Layer

In each optimization cycle (1h), each GES updates its DoS, model coefficients in Eq.(16) and power constraints in Eq.(32), and then reports these information to the LA. In each control cycle (10s), each GES reports its flexibility and responds to the clearing price both through the demand curve.

It is worth mentioning that, since the LA interacts with different GESs through a unified set of information, i.e., DoS, model coefficients, power constraints, demand curve and virtual price, the method in this paper supports a flexible tree-like structure. For example, a local concentrator can be deployed in an community to pre-aggregate the information. Therefore, the method has high scalability and is suitable for wide-area coordination of large-scale GESs.

6 Simulation Studies

6.1 Simulation Settings

The simulation cases are based on a residential community system. The simulation lasts for 24h. Electricity price , regulation capacity price and mileage price adopt the data in literature Song et al. (2018) and Yao et al. (2018), and their profile are illustrated in Fig.7. The regD signal uses the PJM data on Jul 13th, 2016. According to the statistical analysis on the regD signal in 2016 PJM (2017), we set . According to the simulation results based on historical data, we conservatively assign .

Figure 7: Price.

Four types of GESs are considered in this paper, i.e., EES, EV, FFA and IVA, whose parameters are shown in Table.1, where U(a,b) indicates a uniform distribution between [a,b] and denotes the response cycle of GES. The profile of the outdoor temperature in simulation cases is shown in Fig.16.

Type Parameter Value Type Parameter Value
EES Number 10 TCL Thermal Parameter (/kW) U(1,1.5)
(kWh) U(40,50) (kWh/) U(0.8,1.2)
(kW) U(40,50) Preference () U(23,28)
/ 0.9/0.9 () U(2,3)
(s) 10 FFA Number 100
EV Number 20 (kW) U(4.5,5.5)
(kWh) U(20,30) COP U(3,4)
(kW) U(6,8) (min) 5
0.9 IVA Number 100
(h) U(18,22) (kW) U(5,6)
(h) U(6,9) (kW) U(0.4,0.5)
% 2.50% /(kW/Hz) 0.03/0.06
U(0.75,0.85) /(kW) -0.4 / -0.3
(min) 5 (s) 60
Table 1: Parameters of GESs.

To evaluate the control effect, the LA has to estimate the baseline power of the GES cluster when all GESs are uncontrolled, which will be denoted by hereafter. Many papers have studied the estimation method of the baseline load, e.g., a statistical based method proposed in Chen et al. (2017). In this paper, since the aggregate model of the GES cluster is available to the LA, it can estimate in each hour by solving the following optimization problem:


Solution of Eq.(35) is referred to as the baseline case in this paper.

6.2 Case 1: only participate in the energy market

To evaluate the effectiveness of the aggregate dynamic model and the DoS-equality control, an optimization problem simplified from Eq. (31) is solved which only considers the energy market:


LA coordinates GESs to make the aggregate power track the scheduled power . As illustrated in Fig.8, the proposed method has high tracking accuracy. In addition, the aggregate power can vary up and down around the baseline power as the electricity price changes to reduce energy cost. Therefore, the GES cluster can be scheduled as virtual energy storage, since it can be charged by making higher than and discharged by making lower than .

Figure 8: Aggregate power and electricity price in case 1.

Fig.9 demonstrates the performance of the DoS-equality control. As shown in Fig.9(a), DoS of all CP-GESs, i.e., IVAs and EESs, can track quite well as they are able to adjust their power continuously. For DP-GESs (including FFAs and EVs), note again that the average DoS of a cluster rather than an individual’s DoS can follow . It can be seen in Fig.9(b) that of FFAs tracks well. However, for the EVs, slightly fluctuates around . This is because the number of EVs is small, making the statistical characteristics inconspicuous and the distribution of DoS not well aligned with the analysis in section 3.3. Therefore, a larger-scale DP-GES cluster yields better DoS-equality control effect.

Figure 9: Performance of the DoS-equality control in case 1.

Fig.10 shows how well the aggregate dynamic model defined in Eq.(29) fits the cluster with heterogeneous GESs. As can be seen, of all GESs except EVs are very close to the aggregate state variable at the end of every optimization cycle. Since the number of EVs is small, especially during 6:00-9:00 and 18:00-20:00 when some EVs are off-grid, fluctuates around with relatively large errors.

In addition to the small number of DP-GESs, some other factors may also lead to errors in the aggregate dynamic model, e.g., the charge/discharge efficiency is not considered in EES’s dynamic model. In order to prevent the error from being accumulated, this paper adopts the rolling optimization method to mitigate impacts of these factors continually. As can be observed from Fig.10, when combined with the rolling optimization, the simple aggregate model in (29) could be a useful tool for an LA to capture the aggregate dynamic feature of large-scale GESs.

Figure 10: Average DoS and aggregate DoS in case 1.

6.3 Case 2: participate in both energy and regulation markets

In case 2, the GES cluster participates in both the energy and regulation markets. By solving the optimization problem in Eq.(31), the scheduled power and regulation capacity can be obtained, as illustrated in Fig.11. For comparison purposes, the scheduled power in case 1 is also plotted in the figure, which is denoted by . The scheduled power profile in case 2 has significant difference from that in case 1 because the LA should allocate the cluster’s flexibility to two markets according to both electricity price () and regulation price ( and ). Since a symmetric regulation signal is used in this paper, can reach its maximum value when . It can be seen that when is relatively high, e.g., during 14:00-15:00, 19:00-21:00, the LA tends to maximize to gain higher payments from regulation market.

Figure 11: Scheduled power and regulation capacity in case 2.

The target power in this case is calculated by Eq.(30), and the tracking performance is shown in Fig.12. The hourly value of and are shown in Fig.17. According to the result, can basically reach 0.95 under the proposed control framework. In comparison, when responding to regD signal, of a hydroelectric generator can be 0.70.8, while that of an electric energy storage can be higher than 0.9 PJM (2015). Therefore, the simulation results demonstrate that the GESs discussed in this paper are very promising alternatives to provide fast and accurate frequency regulation services.

Figure 12: Tracking performance in case 2.

Trajectory of DoS and clearing price are illustrated in Fig.13. Compared with Fig.10, since the GES cluster also needs to respond to the rapidly changing regD signal, some fluctuations and sudden changes can be observed in , which makes DoS of GESs unable to exactly follow . However, DoS always tends to approach , and thus the DoS-equality control is basically realized. In addition, at the end of some optimization cycles, e.g., at 10:00, 16:00, 24:00, the difference between and is a little larger than that in Fig.10. It is because regD signal is not exactly zero-mean, which affects the actual energy consumption in each optimization cycle and enlarges the difference.

Figure 13: DoS of different GESs and clearing price in case 2.

To analyse the economic benefits of the proposed method, the energy bill, regulation payments and total cost between the above cases (case 1 and case 2) and the baseline case are compared. Hourly calculated cost is shown in Fig.18, and Table.2 lists the daily results. As can be observed, case 1 which only considers energy market can significantly reduce the energy bill compared with the baseline case. By optimally allocating flexility in two markets, case 2 further reduce the total cost compared to case 1, even though it receives a higher energy bill in the energy market. By providing fast regulation service, an LA obtains high payments from the regulation market, leading to a significant reduction in the total cost. Therefore, our method could achieve great economic benefits.

Baseline Case Case 1 Case 2
Energy Bill/ 1062.7 923.8 982.5
Change Rate /% / -13.1 -7.5
Regulation Payments/ 0 0 595.2
Total Cost/ 1062.7 923.8 387.3
Change Rate/% / -13.1 -63.6
Table 2: Comparison of costs (one day) in different cases.

6.4 Response Performance of Individual GESs

The proposed control strategy ensures DoS-equality among GESs. While for different types of GESs, their response behaviour may be different due to their distinguishing features. To demonstrate this, pick one GES randomly from each type of GESs, and plot their response power in Fig.14.

Figure 14: Response power of a single GES in case 2.

For CP-GESs, this paper allows different response cycles. For example, an EES adjusts its response power each 10s (), while an IVA adjusts its power every 1min () considering its response ability, thus an IVA’s response power is stair-shaped as shown in Fig.14(a).

DP-GESs adjust their response power by regulating the duty cycle. Among them, an EV’s duty cycle at different time is basically similar, because its operation is irrelevant to external conditions and mainly depends on the user’s charging pattern. In contrast, the required power of an FFA varies over time as it is significantly affected by environment conditions, e.g., outdoor temperature. For example, it can be observed from Fig.14(b) that the duty cycle of an FFA during 13:00-16:00 is higher than that in the rest of the day as more cooling energy is required during these time slots.

In addition, an EV should ensure that the electric energy reaches its target value at departure time. The change of electric energy is shown in Fig.15, where , . As we can see, the hysteretic model and control strategy used in this paper can guarantee that the difference between the energy at departure time and the target energy would not exceed .

Figure 15: Electric energy of a single EV in case 2.

7 Conclusions

In this paper, a unified coordination method is developed for large-scale heterogeneous GESs to participate in both energy and regulation market.

A generalized state variable referred to as DoS is first defined for GESs. The dynamic models with a unified form are then developed for different GESs. In real-time control, a market-based coordination framework is adopted, and a DoS-equality control method is then developed by construction of generalized demand curves for both GESs operating at continuous power and GESs with discrete states. Based on the unified dynamic models and the DoS-equality control feature, a low-dimensional aggregate dynamic model for a GES cluster is derived. At last, an optimization model aiming to allocate the flexibility of a GES cluster into both the energy market and the regulation market is developed, which uses the aggregate model to significantly reduce the mathematical complexity of the optimization problem.

The control framework has unified uplink/downlink information interfaces and supports a tree-like structure in both real-time coordination stage and optimization control stage, which makes it flexible, scalable and suitable for the control of large-scale GESs. Simulation results demonstrate that the aggregate model well describes the dynamic behaviour of a GES cluster. Additionally, the real-time control method can track the target power accurately while satisfying diversified requirements of different GESs and ensuring control fairness. It is also shown that an LA could gain considerable energy bill savings and high payments by participating in both energy and regulation markets.

However, simulations in this paper are based on ideal communication system and perfect model parameter identification. Future work would study the robustness of our method under communication problems and model errors.


Conceptualization and review: Peichao Zhang; simulation and writing: Yao Yao; review and editing: Sijie Chen.


This research was funded by National Key R&D Program of China (2018YFB0905000).


The authors declare no conflict of interest


The following abbreviations are used in this manuscript:

GES Generalized Energy Storage
EES Electric Energy Storage
EV Electric Vehicle
FFA Fixed-Frequency Air-conditioner
IVA Inverter Air-conditioner
TCL Thermostatically Controlled Load
DoS Degree of Satisfaction
LA Load Aggregator

no \appendixsectionsmultiple

Appendix A Derivation of the function

The analytical solution of Eq.9 can be described as:


where and denote the current indoor air temperature and outdoor temperature of IVA ; denotes the indoor air temperature at time .

To derive the electric power required to make change from to over a certain period of time , we let , , and the required heat rate can be calculated by


According to Eq.(11), the required electric power can be derived by


Appendix B

Figure 16: Outdoor temperature.
Figure 17: and .
Figure 18: Hourly energy bill and regulation payments.





  • Hao et al. (2018) Hao, H.; Wu, D.; Lian, J.; Yang, T. Optimal Coordination of Building Loads and Energy Storage for Power Grid and End User Services. IEEE Transactions on Smart Grid 2018, 9, 4335 – 4345.
  • Mingshen et al. (2018) Mingshen, W.; Yunfei, M.; Jiang, T.; Hongjie, J.; Xue, L.; Kai, H.; Tong, W. Load curve smoothing strategy based on unified state model of different demand side resources. Journal of Modern Power Systems and Clean Energy 2018, 6, 540–554.
  • Vandael et al. (2013) Vandael, S.; Claessens, B.; Hommelberg, M.; Holvoet, T.; Deconinck, G. A Scalable Three-Step Approach for Demand Side Management of Plug-in Hybrid Vehicles. IEEE Transactions on Smart Grid 2013, 4, 720–728.
  • Hao et al. (2014) Hao, H.; Sanandaji, B.M.; Poolla, K.; Vincent, T.L. Aggregate Flexibility of Thermostatically Controlled Loads. IEEE Transactions on Power Systems 2014, 30, 189–198.
  • Song et al. (2018) Song, M.; Gao, C.; Yan, H.; Yang, J. Thermal Battery Modeling of Inverter Air Conditioning for demand response. IEEE Transactions on Smart Grid 2018, 9, 5522 – 5534.
  • Namor et al. (2018) Namor, E.; Sossan, F.; Cherkaoui, R.; Paolone, M. Control of Battery Storage Systems for the Simultaneous Provision of Multiple Services. IEEE Transactions on Smart Grid 2018, PP, 1–1.
  • Dubey et al. (2017) Dubey, A.; Chirapongsananurak, P.; Santoso, S. A Framework for Stacked-Benefit Analysis of Distribution-Level Energy Storage Deployment. Inventions 2017, 2, 6.
  • Wu et al. (2015) Wu, D.; Jin, C.; Balducci, P.; Kintner-Meyer, M. An energy storage assessment: Using optimal control strategies to capture multiple services. Power & Energy Society General Meeting, 2015 IEEE. IEEE, 2015, pp. 1–5.
  • Cheng and Powell (2018) Cheng, B.; Powell, W.B. Co-optimizing battery storage for the frequency regulation and energy arbitrage using multi-scale dynamic programming. IEEE Transactions on Smart Grid 2018, 9, 1997–2005.
  • Anderson and El Gamal (2017) Anderson, K.; El Gamal, A. Co-optimizing the value of storage in energy and regulation service markets. Energy Systems 2017, 8, 369–387.
  • Hao et al. (2013) Hao, H.; Sanandaji, B.; Poolla, K.; Vincent, T. A Generalized Battery Model of a Collection of Thermostatically Controlled Loads for Providing Ancillary Service. Communication, Control, &Computing, 2013.
  • Mathieu et al. (2015) Mathieu.; Kamgarpour, M.; Lygeros, J.; Andersson, G.; Callaway, D.S. Arbitraging Intraday Wholesale Energy Market Prices With Aggregations of Thermostatic Loads. IEEE Transactions on Power Systems 2015, 30, 763–772.
  • Sanandaji et al. (2014) Sanandaji, B.M.; Hao, H.; Poolla, K.; Vincent, T.L. Improved battery models of an aggregation of Thermostatically Controlled Loads for frequency regulation. 2014 American Control Conference - ACC 2014; IEEE, IEEE: Portland, OR, USA, 2014; pp. 38–45. doi:\changeurlcolorblack10.1109/ACC.2014.6858956.
  • Hughes et al. (2016) Hughes, J.T.; Domínguez-García, A.D.; Poolla, K. Identification of Virtual Battery Models for Flexible Loads. IEEE Transactions on Power Systems 2016, 31, 4660–4669.
  • Binetti et al. (2015) Binetti, G.; Davoudi, A.; Naso, D.; Turchiano, B.; Lewis, F.L. Scalable Real-Time Electric Vehicles Charging With Discrete Charging Rates. IEEE Transactions on Smart Grid 2015, 6, 2211–2220.
  • Floch et al. (2016) Floch, C.L.; Kara, E.; Moura, S. PDE Modeling and Control of Electric Vehicle Fleets for Ancillary Services: A Discrete Charging Case. IEEE Transactions on Smart Grid 2016, PP, 1–1.
  • Callaway and Hiskens (2010) Callaway, D.S.; Hiskens, I.A. Achieving Controllability of Electric Loads. Proceedings of the IEEE 2010, 99, 184–199.
  • Yao and Zhang (2017) Yao, Y.; Zhang, P. Transactive control of air conditioning loads for mitigating microgrid tie-line power fluctuations. 2017 IEEE Power Energy Society General Meeting, 2017, pp. 1–5. doi:\changeurlcolorblack10.1109/PESGM.2017.8273853.
  • Liu and Shi (2016) Liu, M.; Shi, Y. Model Predictive Control for Thermostatically Controlled Appliances Providing Balancing Service. IEEE Transactions on Control Systems Technology 2016, 24, 2082 – 2093.
  • Tang et al. (2017) Tang, J.; Wang, D.; Jia, H.; Gao, S.; Song, Y.; Li, J.; Sun, C.; Fan, M. A Study of V2G Control Strategies of Aggregated Electric Vehicles for Real-Time Demand Response Based on Hysteresis Model. Power System Technology 2017, pp. 2155–2164.
  • Zhang et al. (2017) Zhang, T.; Chen, S.X.; Gooi, H.B.; Maciejowski, J.M. A Hierarchical EMS for Aggregated BESSs in Energy and Performance-based Regulation Markets. IEEE Transactions on Power Systems 2017, 32, 1751 – 1760.
  • Xu et al. (2016) Xu, B.; Dvorkin, Y.; Kirschen, D.S.; Silva-Monroy, C.A.; Watson, J.P. A comparison of policies on the participation of storage in U.S. frequency regulation markets. Power and Energy Society General Meeting, 2016.
  • Zechun et al. (2014) Zechun, H.U.; Xu, X.; Fang, Z.; Jing, Z.; Song, Y. Research on Automatic Generation Control Strategy Incorporating Energy Storage Resources. Proceedings of the CSEE 2014, 34.
  • Aho et al. (2015) Aho, J.; Pao, L.Y.; Fleming, P.; Ela, E. Controlling Wind Turbines for Secondary Frequency Regulation: An Analysis of AGC Capabilities Under New Performance Based Compensation Policy: Preprint 2015.
  • Yao et al. (2018) Yao, Y.; Zhang, P.; Wang, Y. A Two-layer Control Method for Thermostatically Controlled Loads to Provide Fast Frequency Regulation. Proceedings of the CSEE 2018, 38.
  • PJM (2017) PJM-RTO regulation Data. https://www.pjm.com/markets-and-operations/ancillary-services.aspx, 2017.
  • Chen et al. (2017) Chen, Y.; Xu, P.; Chu, Y.; Li, W.; Wu, Y.; Ni, L.; Bao, Y.; Wang, K. Short-term electrical load forecasting using the Support Vector Regression (SVR) model to calculate the demand response baseline for office buildings. Applied Energy 2017, 195, 659–670.
  • PJM (2015) Performance, Mileage and the Mileage Ratio. https://www.pjm.com/~/media/committees-groups/task-forces/rmistf/20151111/20151111-item-05-performance-based-regulation-concepts.ashx, 2015.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description