Abstract
This paper proposes a concept of generalized energy storage (GES) to facilitate the integration of largescale heterogeneous flexible resources with electric/thermal energy storage capacity to participate in multiple markets. First, a generalized state variable referred to as degree of satisfaction (DoS) is defined, and dynamic models with a unified form are derived for different types of GESs. Second, a realtime marketbased coordination framework is proposed to facilitate control, and ensure user privacy and device security. Demand curves of different GESs are then developed based on DoS to express their demand urgencies as well as flexibilities. Furthermore, a lowdimensional aggregate dynamic model of a GES cluster is derived thanks to the DoSequality control feature provided by the design of demand curve. At last, an optimization model for a largescale GESs to participate in both the energy market and regulation market is established based on the aggregate model. Simulations results demonstrate that the optimization algorithm could effectively reduce the total cost of an aggregator. Additionally, the proposed coordination method has high tracking accuracy and could well satisfy users’ diversified power demand.
1 \pubvolumexx \issuenum1 \articlenumber5 2018 \copyrightyear2018 \historyReceived: date; Accepted: date; Published: date \TitleAggregating LargeScale Generalized Energy Storages to Participate in Energy Market and Regulation Market \AuthorYao Yao , Peichao Zhang *, Sijie Chen \AuthorNamesYao Yao and Peichao Zhang and Sijie Chen \corresCorrespondence: pczhang@sjtu.edu.cn
1 Introduction
The increasing integration of distributed energy resources (DERs) poses threat on stability and reliability of power system operation. With the development of smart grid technologies, the control of flexible loads, e.g., electric energy storage (EES), thermostatically control load (TCL) and electric vehicle (EV), has become a promising research field to address the problem brought by DERs Hao et al. (2018); Mingshen et al. (2018). Due to the great variety, large scale, wide distribution and small individual capacity of flexible loads, load aggregators (LAs) are required to aggregate and dispatch such loads and provide flexible services to power grid. The coordination strategies have been widely studied. For example, a threestep EV charging algorithm is presented in Vandael et al. (2013) to minimize charging cost. In Hao et al. (2014), a prioritystackbased method is put forward for fixedfrequency airconditioner (FFA) fleet to track automatic generation control signal. Literature Song et al. (2018) establishes physical model of inverter airconditioners (IVAs) and proposes a hierarchical control framework. A unified state model for EVs and TCLs is developed in Mingshen et al. (2018). In Hao et al. (2018), a generalized battery model is presented to describe the operational characteristics of commercial/residential building HVAC and energy storage system, based on which optimal control methods are proposed to provide various services.
However, most of these methods have the following disadvantages. First, most approaches can only be applied to a certain type of load Vandael et al. (2013); Hao et al. (2014); Song et al. (2018). An LA should provide different interfaces to integrate different types of resources, which makes it difficult for the LA to fully utilize flexibility of various flexible loads and increase the control cost. Second, many literatures use centralized control method based on direct load control Hao et al. (2014); Mingshen et al. (2018); Hao et al. (2018). An LA should collect detailed parameters of all controlled loads and specify the response power of each load, which has heavy computational burden and communication traffic, and may lead to privacy issues and device security problems. Such methods may not be suitable for coordination of largescale flexible loads. Some papers develop unified models to coordinate different types of resources Mingshen et al. (2018); Hao et al. (2018), while they depend on centralized control as they mainly focus on individual behaviour rather than aggregate performance.
The objective of LAs is to provide flexible services to power grid. The method in Hao et al. (2014) could provide reliable regulation services. Literature Mingshen et al. (2018) demonstrates its effectiveness in power fluctuation smoothing. In Vandael et al. (2013); Song et al. (2018), LAs participate in optimal dispatch and achieve benefits from energy markets. However, the above researches only consider a single market. It is pointed in Namor et al. (2018) that since different services have different requirements, participating in multiple markets helps to better utilize the control flexibility of resources and obtain more benefits. To stack multiple services, energy storage systems are controlled in Dubey et al. (2017) to simultaneously participate in N1 contingency requirement, voltage management and frequency regulation. An optimal control method is presented in Wu et al. (2015) for energy storages to provide grid services including energy arbitrage, balancing service, capacity value and distribution system deferral, and outage mitigation. Cheng and Powell (2018) and Anderson and El Gamal (2017) introduce a Markov decision process model and a stochastic control method respectively to cooptimize battery storage for energy arbitrage and frequency regulation. However, these papers only consider energy storages. Although literature Hao et al. (2018) considers coordination of various flexible loads to provide multiple grid services, it has to solve some centralized optimization problems, which may lead to a high computational cost.
To address the above problems, a unified modelling method and coordination strategy for generalized energy storages (GESs) is proposed. It has unified information interface and control method, and requires a relatively low communication and computation cost, which is helpful for an LA to conduct coordination control over largescale GESs. The contributions of our work are threefold:

Dynamic models with a unified form are developed for heterogenous GESs based on a generalized state variable referred to as degree of satisfaction (DoS).

A realtime coordination framework based on the market equilibrium mechanism is presented to allocate aggregate power to individual GESs. General demand curves are constructed under the framework to achieve equal degree of satisfaction and meet users’ diversified requirements.

A lowdimensional aggregate dynamic model for a GES cluster is derived and used in an optimization model for an LA to participate in both the energy market and regulation market.
The rest of this paper is organized as follows. Section 2 introduces the dynamic model of different GESs. In section 3, the marketbased realtime coordination framework is proposed and demand curve construction methods for GESs are presented. Section 4 derives the aggregate dynamic model for a GES cluster. In section 5, the optimization problem that considers both energy and regulation markets is given. Section 6 shows the simulation results which demonstrate the effectiveness of our method. Finally, section 7 summarizes our contribution and future work.
2 Dynamic Models for GESs
This paper focuses on four typical types of GESs, i.e., EES, EV, inverter airconditioner (IVA) and fixedfrequency airconditioner (FFA) since they account for a large share of flexible resources in the demand side. These resources are called GES for that they all can store energy, i.e., electric energy or cold/thermal energy, thus their power consumption could be adjusted without affecting the user satisfaction. Meanwhile, they have similar dynamic characteristics, which makes it possible to establish a unified physical model for them. For example, there have been a few literatures that develop battery modelling method for the thermal energy storage loads. However, most researches only focus on FFAsHao et al. (2013); Mathieu et al. (2015); Sanandaji et al. (2014). A batterytype reducedorder model for building HVAC system is proposed in Hughes et al. (2016), but it only considers the dynamics of individual load. Literature Song et al. (2018) studies the battery modelling for individual IVA and aggregated IVA cluster, while the aggregating strategy would lead to the result that the IVA cluster’s regulation ability could not be fully utilized. This paper will put forward a unified dynamic model for different types of GESs, and study both individual model (introduced in this section) and aggregate model (detailed in section.4).
2.1 Degree of Satisfaction, DoS
Before establishing the dynamic models, a dimensionless state variable referred to as degree of satisfaction (DoS) is defined for GESs, with the following purposes:

It could be used to measure the user satisfaction. The range of DoS is set to [1,1], and the closer DoS is to 0, the higher user satisfaction is.

DoS could reflect a GES’s state of energy: DoS equalling 0 indicates that the stored energy is at the expected level, while DoS close to 1 means the stored energy is near the allowed range.

Since a GES can deviate from its ideal state (DoS=0) to provide services, the DoS can be used to quantify its current flexility, i.e., a DoS value close to 0 implies a high flexility reservation.

Finally, as DoS is a generalized index, it can be used to establish a unified dynamic model for various GESs.
2.2 Derivation of Dynamic Models
2.2.1 Electric Energy Storage (EES)
Ignoring the charge/discharge efficiency, the dynamic model of an EES is given by
(1) 
where and denote the electric energy and power of EES at time , respectively ( when EES is charging); is the control cycle.
Variable is hereafter used to denote DoS. Definition of EES’s DoS is given by
(2) 
2.2.2 Electric Vehicle (EV)
The physical model of an EV is
(4) 
where and denote the electric energy and power of EV at time , respectively; is the charge efficiency.
According to the current charger technology, we focus on the prevailing EV type, which operates at two discrete states: idle and charging with a fixed rate Binetti et al. (2015); Floch et al. (2016). Let denote the target energy of EV at userspecified departure time , and denote the energy at the time EV is connected into power grid . The average power required to charge EV to can be calculated by
(5) 
If EV charges at , the expected energy profile is
(6) 
An EV can provide its flexibility by deviating from . Referring to the hysteretic model in Callaway and Hiskens (2010), DoS of EV is defined as
(7) 
where denotes the nominal capcity of EV ; denotes the energy deadband, which limits the error between and the actual energy at within .
2.2.3 Inverter Airconditioner (IVA)
Without loss of generality, cooling airconditioners are studied in this paper. The thermal dynamic process is modelled by a firstorder differential equation Hao et al. (2014); Song et al. (2018) as
(9) 
where and denote the indoor air temperature and outdoor temperature at time , respectively; is the heat rate of IVA ; the thermal parameter .
The analytical solution of Eq.(9) in recursive form is
(10) 
The electrical model of an IVA adopts the simplified linear model in Song et al. (2018), which is given by
(11) 
where denotes the electric power of IVA ; is the operation frequency of the compressor; , , and are coefficients.
DoS of an IVA is defined as
(12) 
where denotes the setpoint; denotes the allowed temperature deviation.
2.2.4 FixedFrequency Airconditioner (FFA)
The thermal model of an FFA can also be established by Eq.(9). Its electrical model is given by
(14) 
where is the coefficient of performance. FFA is ON/OFF controlled load, so its electric power equals to nominal power when it is ON and when it is OFF.
2.3 Unified Dynamic Model
The dynamic models of the above GESs can now be represented in a unified form:
(16) 
where , , are coefficients of the unified dynamic model.
Mappings between DoS and the original variables of different GESs are summarized in Fig.1. This paper would study a realtime coordination method of GESs, which could make DoS of individual GESs (or average DoS of a cluster of GESs) approximately equal. Such control characteristic is referred to as DoSequality control in this paper and will be detailed in the next section.
3 Realtime Coordination Method of LargeScale GESs
3.1 DoSequality Control Based on Market Equilibrium Mechanism
Market equilibrium mechanism Yao and Zhang (2017) is introduced in this paper to coordinate largescale GESs. Main stages of the coordination method are:
(1) Bidding: Each GES expresses its urgency and flexibility by constructing a demand curve. The demand curve is denoted by in this paper, which is a nonincreasing function.
(2) Aggregating and clearing: LA collects demand curves from all GESs and forms the aggregate demand curve , where is the number of controlled GESs; Assume the aggregate target power is , then LA can calculate the clearing price by .
(3) Disaggregating: LA broadcasts to all GESs. Each GES responds to locally according to its demand curve. The response power of GES can be obtained by .
Following the above steps, the aggregate target power can be allocated among the GESs, thus realizing accurate power tracking. It should be noted that: (1) The target power depends on applications. In this paper, it will be determined by an optimization problem to be discussed in the next section; (2) The clearing price is only a control signal rather than a price signal. It is dimensionless and its range is set to [1,1]. Therefore, it is called "virtual price" in this paper.
The proposed marketbased coordination method highlights the following advantages:

It improves the autonomy of the GES. Each GES can convert its private information, e.g., user preferences, current adjustable range and security constraints, into demand curve. Since demand curves of all GESs have a unified form, it can shield the differences among various GESs and effectively protect user privacy. Besides, the LA does not have permission to directly control GES, which improves device security.

It simplifies the control of the LA. An LA does not need to specify each GES’s type and is able to coordinates various GESs via an identical signal, i.e., the virtual price signal , which significantly reduces control complexity and requirement of communication bandwidth.
In addition, the proposed method can realize the DoSequality control to obtain the following advantages:

GESs could have same degree of user satisfaction regardless of the resource type or capacity, which ensures control fairness. In addition, since DoS reflects a GES’s state of energy, the DoSequality control could avoid some GESs going beyond their adjustable range prematurely, thus better utilizing the regulation ability of a GES cluster.

The unique DoS of a GES cluster can be a state variable to derive an aggregate dynamic model, making it possible to treat the whole GES cluster as a virtual storage, which will be detailed in section 4.
According to the operation characteristic, the GES can be further classified into two types: GES operating at continuous power (CPGES) and GES operating at discrete power with discrete states (DPGES). A CPGES, e.g., an EES or an IVA, is able to keep its DoS at the desired value by adjusting its operating power, while for a DPGES, e.g., an EV, an FFA or an electric heater, its DoS generally fluctuates within the allowed range. Besides, the state switching frequency of a DPGES should generally be controlled to prolong the device’s lifetime.
The key of the proposed coordination method is the construction of demand curves for different GESs, which will be introduced in the following subsections.
3.2 Demand Curve of CPGES
3.2.1 Demand Curve
Let denote the current DoS of GES . The following construction principle of demand curve is established in this paper:
(17) 
where denotes the electric power required to maintain the current over a control cycle.
Eq.(17) is used to realize the DoSequality control feature. For explanation, the clearing price is assumed to be constant. When , the response power is higher than , leading to the decrease of ; when , is lower than , leading to the increase of ; when , equals , keeping unchanged. Therefore, values of all GESs following the principle in Eq.(17) will approach the control signal .
Based on Eq.(17), the demand curve of CPGES is constructed as shown in Fig.2. Note again that the price herein is virtual and is limited between 1 and 1. The demand curve consists of 5 key points. The anchor point A could satisfy the condition in Eq.(17). Points B and C are used to keep the DoS within limits when responding to any clearing price . Thus, and are minimum and maximum power that would not make DoS go beyond limit over a certain period of time (which is set 5min in this paper), without considering operation constraints. Points D and E which lies on line AB and line AC respectively are further introduced to guarantee the response power would not exceed operational constraints. Thus, and are minimum and maximum power that a GES could operate at in current control cycle. The calculation method of the above characteristic power will be detailed in the next section.
3.2.2 Characteristic Power
The EES and IVA are typical CPGESs. They can both adopt the demand curve shown in Fig.2.
An EES’s characteristic power can be calculated by
(18) 
where denotes the nominal charging/discharging power; and are charge/discharge efficiency, respectively; and are the minimum and maximum allowed energy, respectively; is the stored energy at time .
IVA’s characteristic power can be calculated by
(19) 
where and denotes the minimum and maximum power; the function obtains the electric power that makes the indoor air temperature change from current value to the target temperature over a period of time . The derivation of is detailed in Appendix A.
3.3 Demand Curve of DPGES
3.3.1 Demand Curve
A DPGES typically has two states, i.e., ON and OFF states. The proposed demand curve for a DPGES is illustrated in Fig.3(a), where denotes the operating power when the GES is ON, and is transformed from its DoS value by offsetting and then normalizing:
(20) 
To explain the principle, the aggregate demand curve of a cluster of DPGESs is shown in Fig.3(b), which is obtained by sorting DPGESs in descending order of . As can be seen, the proposed bidding strategy can divide DPGESs into two groups according to their operation states, i.e., an ON group in the upper halfplane and an OFF group in the lower halfplane, and achieve the following purposes:
(1) For DPGESs in the same group, a DPGES’s reflects its power consumption priority. The higher is, the higher probability to maintain or switch to ON state is, and vice versa.
(2) A DPGES in the ON group always has a higher than that in the OFF group, which gives high priority for DPGESs to maintain their current states, thus avoiding frequent switching.
To explain the change rule of a single DPGES’s DoS, the clearing price is assumed to be constant over a period of time. Note that the state change of a DPGES may be triggered either by its value, or by to ensure comfort. The trajectory of and at two different , i.e., and , are illustrated in Fig.4(a) and Fig.4(b), and the following laws can be found: when , ranges in [1+2,1]; when , ranges in [1,1+2], which means that the DoS value fluctuates within a symmetrical range around . Therefore, if DoS values of DPGESs are assumed to be uniformly distributed in such range, the average DoS of the cluster equals .
3.3.2 Characteristic Power
EV and FFA are typical DPGESs. They can both adopt the demand curve in Fig.3(a).
An EV’s characteristic power is
(21) 
where denotes the EV’s nominal charging power.
An FFA’s characteristic power is
(22) 
3.4 Locked State
In certain control cycles, a GES may get into the locked state, which means it should maintain its current operating state and power. There are two typical situations: (1) To reduce mechanical wear and protect the device Liu and Shi (2016); Tang et al. (2017), DPGESs, such as FFA and EV, should satisfy lockout time constraints before they switch state; (2) Restricted by the device capability, response cycle of some GESs may be longer than the realtime control cycle (which is 10s in this paper).
The lockout mechanism can be easily realized in this paper, as a GES can simply submit the following demand curve during the lockout time:
(23) 
which means the response power of GES maintains its current operating power for any .
4 Aggregate Dynamic Model of a GES Cluster
One significant advantage of our method is that the aggregate dynamic model of a largescale GES cluster can be easily derived thanks to the DoSequality control feature, as well as the unified dynamic model for different GESs defined in Eq.(16).
For CPGESs, add their dynamic models together:
(24) 
where represents the set of CPGESs.
Under the DoSequality control, DoS of all CPGESs becomes equal. Let denote DoS of the CPGES cluster, and denote the aggregate power. The aggregate dynamic model of CPGESs can then be derived as
(25) 
For DPGESs, to facilitate analysis, we assume coefficients of their dynamic models to be equal first. Add their dynamic models together, and we obtain
(26) 
where represents the set of DPGESs.
The instantaneous DoS value of each DPGES is a random variable. Denote the average DoS of DPGESs as , and denote the aggregate power as . Then Eq.(26) can be written as:
(27) 
where denotes the number of DPGESs.
Under the DoSequality control, of CPGESs and of DPGESs tend to be equal (both equal ), thus can both be denoted by . Therefore, Eq.(25) and Eq.(27) can be added together:
(28) 
where denotes the total number of GESs; is the aggregate power.
Eq.(28) can be represented in a compact form as
(29) 
The above derivation is based on the assumption that all DPGESs have equal model coefficients. For those with different coefficients, we can first divide them into different groups according to their coefficients, then aggregate each group using Eq.(27), and finally form the aggregate dynamic model of all the GESs (including CPGESs and DPGESs) by Eq.(28).
The aggregate dynamic model of heterogeneous GESs proposed above has the advantage that the LA could obtain the aggregate model easily by adding model coefficients of all GESs with no need to identify GES’s type, thus having a low computational cost. Furthermore, from the control aspect, the lowdimensional aggregate model greatly reduces the complexity of the optimization problem, which will be discussed in the next section.
5 Application
5.1 Optimal MultiMarket Flexibility Allocation
An LA can aggregate flexibility of largescale GESs to provide multiple services to power grid. For example, it can schedule an optimal power consumption profile according to the electricity price of energy market Song et al. (2018); Hao et al. (2018); Zhang et al. (2017), as the profile in Fig.5. LA can provide other ancillary services at the same time to obtain higher benefits Hao et al. (2018); Zhang et al. (2017), e.g., responding to the regulation signal in Fig.5.
The regulation signal can be decomposed into the low frequency part denoted by regA, and the high frequency part denoted by regD Xu et al. (2016). Literature Zechun et al. (2014) analyses the regulation signal of a certain power grid, and finds that the high frequency part could account for up to 30%. In this paper, the LA responds to the regD signal considering the following facts: First, the regD signal has zeromean over a period of time Xu et al. (2016), which could significantly reduce requirements for the capacity of GESs; Second, the extra energy introduced by the regD signal is close to 0, thus having little impact on electricity bills; Third, if regulation payments are determined by the performancebased policy used in PJM regulation market Aho et al. (2015), the LA could obtain high benefits.
Considering both the energy and regulation markets, the target power of an LA is given by
(30) 
where is the hourly scheduled power, which determines the bill paid to the energy market; regD is the regulation signal normalized to [0,1]; denotes the contracted regulation capacity.
Let the optimization cycle be 1 hour, then the LA solves the following convex optimization problem in the th cycle to allocate its flexibility to the two markets:
(31)  
where , , denote respectively the electricity price, regulation capacity price and regulation mileage price in the th cycle; denotes the statistical value of the regulation performance score defined by PJM Aho et al. (2015), and denotes the statistical value of the regulation mileage Aho et al. (2015); and are the maximum and minimum power of the GES cluster, which are calculated in every optimization cycle as and , where
(32)  
Note that EESs are able to discharge, thus the LA may sell electricity to power grid at this time. The purchase price and sale price are assumed to be equal in this paper.
The first constraint in Eq.(31) is the aggregate dynamic model of the GES cluster, the second constraint ensures the user satisfaction, and the last three ones constrain the regulation capacity. Thanks to the established aggregate dynamic model, the scale of the GES cluster does not affect the computational complexity of the optimization problem.
To improve user satisfaction, Eq.(31) includes a penalty term , which is defined as
(33) 
where is a proportionality coefficient, which is assigned 0.1 here; is the daily average electricity price.
5.2 ThreeLayer Control Structure
To sum up, this paper develops four models, i.e., a unified dynamic model and a unified demand model for individual GES, as well as an aggregate dynamic model and an optimization model for an LA to approximate dynamics of a GES cluster and participate in multimarkets. These four models are organized in a a threelayer control structure, as illustrated in Fig.6.
(1) Rolling Optimization Layer
At the beginning of current optimization cycle , the LA gathers model coefficients from GESs and update the aggregate dynamic model according to Eq.(28). Meanwhile, current DoS values of all GESs are collected to update the initial recursive value of by
(34) 
The optimization problem in Eq.(31) can then be solved, and the optimal scheduled power sequence as well as the regulation capacity sequence are obtained. Implement only the first elements of these optimal sequences, i.e., and in the current optimization cycle, and repeat the above steps each hour. It’s well known that this idea of rolling optimization comes from the model predictive control (MPC) Liu and Shi (2016), which is adopted here to update the aggregate dynamic model iteratively, and to consider constraints in future time slots explicitly.
(2) Realtime Coordination Layer
In each control cycle (10s in this paper), the LA receives the regulation signal regD from the control center, and then calculates the realtime target power according to Eq.(30). The virtual market is cleared according to section 3, and the clearing price is broadcast to each GES.
(3) GES Autonomy Layer
In each optimization cycle (1h), each GES updates its DoS, model coefficients in Eq.(16) and power constraints in Eq.(32), and then reports these information to the LA. In each control cycle (10s), each GES reports its flexibility and responds to the clearing price both through the demand curve.
It is worth mentioning that, since the LA interacts with different GESs through a unified set of information, i.e., DoS, model coefficients, power constraints, demand curve and virtual price, the method in this paper supports a flexible treelike structure. For example, a local concentrator can be deployed in an community to preaggregate the information. Therefore, the method has high scalability and is suitable for widearea coordination of largescale GESs.
6 Simulation Studies
6.1 Simulation Settings
The simulation cases are based on a residential community system. The simulation lasts for 24h. Electricity price , regulation capacity price and mileage price adopt the data in literature Song et al. (2018) and Yao et al. (2018), and their profile are illustrated in Fig.7. The regD signal uses the PJM data on Jul 13th, 2016. According to the statistical analysis on the regD signal in 2016 PJM (2017), we set . According to the simulation results based on historical data, we conservatively assign .
Four types of GESs are considered in this paper, i.e., EES, EV, FFA and IVA, whose parameters are shown in Table.1, where U(a,b) indicates a uniform distribution between [a,b] and denotes the response cycle of GES. The profile of the outdoor temperature in simulation cases is shown in Fig.16.
Type  Parameter  Value  Type  Parameter  Value  

EES  Number  10  TCL  Thermal Parameter  (/kW)  U(1,1.5) 
(kWh)  U(40,50)  (kWh/)  U(0.8,1.2)  
(kW)  U(40,50)  Preference  ()  U(23,28)  
/  0.9/0.9  ()  U(2,3)  
(s)  10  FFA  Number  100  
EV  Number  20  (kW)  U(4.5,5.5)  
(kWh)  U(20,30)  COP  U(3,4)  
(kW)  U(6,8)  (min)  5  
0.9  IVA  Number  100  
(h)  U(18,22)  (kW)  U(5,6)  
(h)  U(6,9)  (kW)  U(0.4,0.5)  
%  2.50%  /(kW/Hz)  0.03/0.06  
U(0.75,0.85)  /(kW)  0.4 / 0.3  
(min)  5  (s)  60 
To evaluate the control effect, the LA has to estimate the baseline power of the GES cluster when all GESs are uncontrolled, which will be denoted by hereafter. Many papers have studied the estimation method of the baseline load, e.g., a statistical based method proposed in Chen et al. (2017). In this paper, since the aggregate model of the GES cluster is available to the LA, it can estimate in each hour by solving the following optimization problem:
(35)  
Solution of Eq.(35) is referred to as the baseline case in this paper.
6.2 Case 1: only participate in the energy market
To evaluate the effectiveness of the aggregate dynamic model and the DoSequality control, an optimization problem simplified from Eq. (31) is solved which only considers the energy market:
(36)  
LA coordinates GESs to make the aggregate power track the scheduled power . As illustrated in Fig.8, the proposed method has high tracking accuracy. In addition, the aggregate power can vary up and down around the baseline power as the electricity price changes to reduce energy cost. Therefore, the GES cluster can be scheduled as virtual energy storage, since it can be charged by making higher than and discharged by making lower than .
Fig.9 demonstrates the performance of the DoSequality control. As shown in Fig.9(a), DoS of all CPGESs, i.e., IVAs and EESs, can track quite well as they are able to adjust their power continuously. For DPGESs (including FFAs and EVs), note again that the average DoS of a cluster rather than an individual’s DoS can follow . It can be seen in Fig.9(b) that of FFAs tracks well. However, for the EVs, slightly fluctuates around . This is because the number of EVs is small, making the statistical characteristics inconspicuous and the distribution of DoS not well aligned with the analysis in section 3.3. Therefore, a largerscale DPGES cluster yields better DoSequality control effect.
Fig.10 shows how well the aggregate dynamic model defined in Eq.(29) fits the cluster with heterogeneous GESs. As can be seen, of all GESs except EVs are very close to the aggregate state variable at the end of every optimization cycle. Since the number of EVs is small, especially during 6:009:00 and 18:0020:00 when some EVs are offgrid, fluctuates around with relatively large errors.
In addition to the small number of DPGESs, some other factors may also lead to errors in the aggregate dynamic model, e.g., the charge/discharge efficiency is not considered in EES’s dynamic model. In order to prevent the error from being accumulated, this paper adopts the rolling optimization method to mitigate impacts of these factors continually. As can be observed from Fig.10, when combined with the rolling optimization, the simple aggregate model in (29) could be a useful tool for an LA to capture the aggregate dynamic feature of largescale GESs.
6.3 Case 2: participate in both energy and regulation markets
In case 2, the GES cluster participates in both the energy and regulation markets. By solving the optimization problem in Eq.(31), the scheduled power and regulation capacity can be obtained, as illustrated in Fig.11. For comparison purposes, the scheduled power in case 1 is also plotted in the figure, which is denoted by . The scheduled power profile in case 2 has significant difference from that in case 1 because the LA should allocate the cluster’s flexibility to two markets according to both electricity price () and regulation price ( and ). Since a symmetric regulation signal is used in this paper, can reach its maximum value when . It can be seen that when is relatively high, e.g., during 14:0015:00, 19:0021:00, the LA tends to maximize to gain higher payments from regulation market.
The target power in this case is calculated by Eq.(30), and the tracking performance is shown in Fig.12. The hourly value of and are shown in Fig.17. According to the result, can basically reach 0.95 under the proposed control framework. In comparison, when responding to regD signal, of a hydroelectric generator can be 0.70.8, while that of an electric energy storage can be higher than 0.9 PJM (2015). Therefore, the simulation results demonstrate that the GESs discussed in this paper are very promising alternatives to provide fast and accurate frequency regulation services.
Trajectory of DoS and clearing price are illustrated in Fig.13. Compared with Fig.10, since the GES cluster also needs to respond to the rapidly changing regD signal, some fluctuations and sudden changes can be observed in , which makes DoS of GESs unable to exactly follow . However, DoS always tends to approach , and thus the DoSequality control is basically realized. In addition, at the end of some optimization cycles, e.g., at 10:00, 16:00, 24:00, the difference between and is a little larger than that in Fig.10. It is because regD signal is not exactly zeromean, which affects the actual energy consumption in each optimization cycle and enlarges the difference.
To analyse the economic benefits of the proposed method, the energy bill, regulation payments and total cost between the above cases (case 1 and case 2) and the baseline case are compared. Hourly calculated cost is shown in Fig.18, and Table.2 lists the daily results. As can be observed, case 1 which only considers energy market can significantly reduce the energy bill compared with the baseline case. By optimally allocating flexility in two markets, case 2 further reduce the total cost compared to case 1, even though it receives a higher energy bill in the energy market. By providing fast regulation service, an LA obtains high payments from the regulation market, leading to a significant reduction in the total cost. Therefore, our method could achieve great economic benefits.
Baseline Case  Case 1  Case 2  
Energy Bill/  1062.7  923.8  982.5 
Change Rate /%  /  13.1  7.5 
Regulation Payments/  0  0  595.2 
Total Cost/  1062.7  923.8  387.3 
Change Rate/%  /  13.1  63.6 
6.4 Response Performance of Individual GESs
The proposed control strategy ensures DoSequality among GESs. While for different types of GESs, their response behaviour may be different due to their distinguishing features. To demonstrate this, pick one GES randomly from each type of GESs, and plot their response power in Fig.14.
For CPGESs, this paper allows different response cycles. For example, an EES adjusts its response power each 10s (), while an IVA adjusts its power every 1min () considering its response ability, thus an IVA’s response power is stairshaped as shown in Fig.14(a).
DPGESs adjust their response power by regulating the duty cycle. Among them, an EV’s duty cycle at different time is basically similar, because its operation is irrelevant to external conditions and mainly depends on the user’s charging pattern. In contrast, the required power of an FFA varies over time as it is significantly affected by environment conditions, e.g., outdoor temperature. For example, it can be observed from Fig.14(b) that the duty cycle of an FFA during 13:0016:00 is higher than that in the rest of the day as more cooling energy is required during these time slots.
In addition, an EV should ensure that the electric energy reaches its target value at departure time. The change of electric energy is shown in Fig.15, where , . As we can see, the hysteretic model and control strategy used in this paper can guarantee that the difference between the energy at departure time and the target energy would not exceed .
7 Conclusions
In this paper, a unified coordination method is developed for largescale heterogeneous GESs to participate in both energy and regulation market.
A generalized state variable referred to as DoS is first defined for GESs. The dynamic models with a unified form are then developed for different GESs. In realtime control, a marketbased coordination framework is adopted, and a DoSequality control method is then developed by construction of generalized demand curves for both GESs operating at continuous power and GESs with discrete states. Based on the unified dynamic models and the DoSequality control feature, a lowdimensional aggregate dynamic model for a GES cluster is derived. At last, an optimization model aiming to allocate the flexibility of a GES cluster into both the energy market and the regulation market is developed, which uses the aggregate model to significantly reduce the mathematical complexity of the optimization problem.
The control framework has unified uplink/downlink information interfaces and supports a treelike structure in both realtime coordination stage and optimization control stage, which makes it flexible, scalable and suitable for the control of largescale GESs. Simulation results demonstrate that the aggregate model well describes the dynamic behaviour of a GES cluster. Additionally, the realtime control method can track the target power accurately while satisfying diversified requirements of different GESs and ensuring control fairness. It is also shown that an LA could gain considerable energy bill savings and high payments by participating in both energy and regulation markets.
However, simulations in this paper are based on ideal communication system and perfect model parameter identification. Future work would study the robustness of our method under communication problems and model errors.
Conceptualization and review: Peichao Zhang; simulation and writing: Yao Yao; review and editing: Sijie Chen.
This research was funded by National Key R&D Program of China (2018YFB0905000).
The authors declare no conflict of interest
The following abbreviations are used in this manuscript:
GES  Generalized Energy Storage 
EES  Electric Energy Storage 
EV  Electric Vehicle 
FFA  FixedFrequency Airconditioner 
IVA  Inverter Airconditioner 
TCL  Thermostatically Controlled Load 
DoS  Degree of Satisfaction 
LA  Load Aggregator 
no \appendixsectionsmultiple
Appendix A Derivation of the function
The analytical solution of Eq.9 can be described as:
(37) 
where and denote the current indoor air temperature and outdoor temperature of IVA ; denotes the indoor air temperature at time .
To derive the electric power required to make change from to over a certain period of time , we let , , and the required heat rate can be calculated by
(38) 
According to Eq.(11), the required electric power can be derived by
(39) 
Appendix B
\reftitleReferences
yes
References
 Hao et al. (2018) Hao, H.; Wu, D.; Lian, J.; Yang, T. Optimal Coordination of Building Loads and Energy Storage for Power Grid and End User Services. IEEE Transactions on Smart Grid 2018, 9, 4335 – 4345.
 Mingshen et al. (2018) Mingshen, W.; Yunfei, M.; Jiang, T.; Hongjie, J.; Xue, L.; Kai, H.; Tong, W. Load curve smoothing strategy based on unified state model of different demand side resources. Journal of Modern Power Systems and Clean Energy 2018, 6, 540–554.
 Vandael et al. (2013) Vandael, S.; Claessens, B.; Hommelberg, M.; Holvoet, T.; Deconinck, G. A Scalable ThreeStep Approach for Demand Side Management of Plugin Hybrid Vehicles. IEEE Transactions on Smart Grid 2013, 4, 720–728.
 Hao et al. (2014) Hao, H.; Sanandaji, B.M.; Poolla, K.; Vincent, T.L. Aggregate Flexibility of Thermostatically Controlled Loads. IEEE Transactions on Power Systems 2014, 30, 189–198.
 Song et al. (2018) Song, M.; Gao, C.; Yan, H.; Yang, J. Thermal Battery Modeling of Inverter Air Conditioning for demand response. IEEE Transactions on Smart Grid 2018, 9, 5522 – 5534.
 Namor et al. (2018) Namor, E.; Sossan, F.; Cherkaoui, R.; Paolone, M. Control of Battery Storage Systems for the Simultaneous Provision of Multiple Services. IEEE Transactions on Smart Grid 2018, PP, 1–1.
 Dubey et al. (2017) Dubey, A.; Chirapongsananurak, P.; Santoso, S. A Framework for StackedBenefit Analysis of DistributionLevel Energy Storage Deployment. Inventions 2017, 2, 6.
 Wu et al. (2015) Wu, D.; Jin, C.; Balducci, P.; KintnerMeyer, M. An energy storage assessment: Using optimal control strategies to capture multiple services. Power & Energy Society General Meeting, 2015 IEEE. IEEE, 2015, pp. 1–5.
 Cheng and Powell (2018) Cheng, B.; Powell, W.B. Cooptimizing battery storage for the frequency regulation and energy arbitrage using multiscale dynamic programming. IEEE Transactions on Smart Grid 2018, 9, 1997–2005.
 Anderson and El Gamal (2017) Anderson, K.; El Gamal, A. Cooptimizing the value of storage in energy and regulation service markets. Energy Systems 2017, 8, 369–387.
 Hao et al. (2013) Hao, H.; Sanandaji, B.; Poolla, K.; Vincent, T. A Generalized Battery Model of a Collection of Thermostatically Controlled Loads for Providing Ancillary Service. Communication, Control, &Computing, 2013.
 Mathieu et al. (2015) Mathieu.; Kamgarpour, M.; Lygeros, J.; Andersson, G.; Callaway, D.S. Arbitraging Intraday Wholesale Energy Market Prices With Aggregations of Thermostatic Loads. IEEE Transactions on Power Systems 2015, 30, 763–772.
 Sanandaji et al. (2014) Sanandaji, B.M.; Hao, H.; Poolla, K.; Vincent, T.L. Improved battery models of an aggregation of Thermostatically Controlled Loads for frequency regulation. 2014 American Control Conference  ACC 2014; IEEE, IEEE: Portland, OR, USA, 2014; pp. 38–45. doi:\changeurlcolorblack10.1109/ACC.2014.6858956.
 Hughes et al. (2016) Hughes, J.T.; DomínguezGarcía, A.D.; Poolla, K. Identification of Virtual Battery Models for Flexible Loads. IEEE Transactions on Power Systems 2016, 31, 4660–4669.
 Binetti et al. (2015) Binetti, G.; Davoudi, A.; Naso, D.; Turchiano, B.; Lewis, F.L. Scalable RealTime Electric Vehicles Charging With Discrete Charging Rates. IEEE Transactions on Smart Grid 2015, 6, 2211–2220.
 Floch et al. (2016) Floch, C.L.; Kara, E.; Moura, S. PDE Modeling and Control of Electric Vehicle Fleets for Ancillary Services: A Discrete Charging Case. IEEE Transactions on Smart Grid 2016, PP, 1–1.
 Callaway and Hiskens (2010) Callaway, D.S.; Hiskens, I.A. Achieving Controllability of Electric Loads. Proceedings of the IEEE 2010, 99, 184–199.
 Yao and Zhang (2017) Yao, Y.; Zhang, P. Transactive control of air conditioning loads for mitigating microgrid tieline power fluctuations. 2017 IEEE Power Energy Society General Meeting, 2017, pp. 1–5. doi:\changeurlcolorblack10.1109/PESGM.2017.8273853.
 Liu and Shi (2016) Liu, M.; Shi, Y. Model Predictive Control for Thermostatically Controlled Appliances Providing Balancing Service. IEEE Transactions on Control Systems Technology 2016, 24, 2082 – 2093.
 Tang et al. (2017) Tang, J.; Wang, D.; Jia, H.; Gao, S.; Song, Y.; Li, J.; Sun, C.; Fan, M. A Study of V2G Control Strategies of Aggregated Electric Vehicles for RealTime Demand Response Based on Hysteresis Model. Power System Technology 2017, pp. 2155–2164.
 Zhang et al. (2017) Zhang, T.; Chen, S.X.; Gooi, H.B.; Maciejowski, J.M. A Hierarchical EMS for Aggregated BESSs in Energy and Performancebased Regulation Markets. IEEE Transactions on Power Systems 2017, 32, 1751 – 1760.
 Xu et al. (2016) Xu, B.; Dvorkin, Y.; Kirschen, D.S.; SilvaMonroy, C.A.; Watson, J.P. A comparison of policies on the participation of storage in U.S. frequency regulation markets. Power and Energy Society General Meeting, 2016.
 Zechun et al. (2014) Zechun, H.U.; Xu, X.; Fang, Z.; Jing, Z.; Song, Y. Research on Automatic Generation Control Strategy Incorporating Energy Storage Resources. Proceedings of the CSEE 2014, 34.
 Aho et al. (2015) Aho, J.; Pao, L.Y.; Fleming, P.; Ela, E. Controlling Wind Turbines for Secondary Frequency Regulation: An Analysis of AGC Capabilities Under New Performance Based Compensation Policy: Preprint 2015.
 Yao et al. (2018) Yao, Y.; Zhang, P.; Wang, Y. A Twolayer Control Method for Thermostatically Controlled Loads to Provide Fast Frequency Regulation. Proceedings of the CSEE 2018, 38.
 PJM (2017) PJMRTO regulation Data. https://www.pjm.com/marketsandoperations/ancillaryservices.aspx, 2017.
 Chen et al. (2017) Chen, Y.; Xu, P.; Chu, Y.; Li, W.; Wu, Y.; Ni, L.; Bao, Y.; Wang, K. Shortterm electrical load forecasting using the Support Vector Regression (SVR) model to calculate the demand response baseline for office buildings. Applied Energy 2017, 195, 659–670.
 PJM (2015) Performance, Mileage and the Mileage Ratio. https://www.pjm.com/~/media/committeesgroups/taskforces/rmistf/20151111/20151111item05performancebasedregulationconcepts.ashx, 2015.