Ramp-Rate-Constrained Bidding of Energy and Frequency Reserves in Real Market Settings
The energetic flexibility of electric energy resources can be exploited when trading on wholesale energy and ancillary service markets. This paper considers the problem of a Balance Responsible Party to maximize its profit from trading on energy markets while simultaneously offering Secondary Frequency Reserves to the System Operator. However, the accurate provision of regulation power can be compromised by power ramp-rate limitations of the resources providing the service. To avoid this shortcoming, we take power ramp-rate constraints into account explicitly when computing optimal adjustable energy trading policies that are robust against uncertain activation of reserves. The approach proposed is applicable in real market settings because it models all the different timescales of the day-ahead, intra-day, and reserve markets. Finally, the effect of different market settings and energy trading policies on the amount of available Secondary Frequency Reserves is investigated.
Balance Responsible Parties (BRPs) act as intermediaries between electric energy resources, electricity markets, and Independent System Operators (ISOs). BRPs are legal entities representing a single or a group of electric energy systems. They are responsible for trading electric energy on energy markets to satisfy the needs of their balance group and to adjust energy schedules to match actual electricity production and/or consumption as accurately as possible. If the balance group of a BRP comprises systems whose electric energy generation and/or consumption is flexible, the BRP can offer this flexibility to the ISO in the form of ancillary services that are traded on dedicated flexibility markets . We consider a BRP of a single flexible system that wishes to maximize the economic profit made from trading energy on day-ahead and intra-day energy markets and offering Secondary Frequency Regulation (SFR) services to the ISO. We refer to the problem of making optimal decisions on energy and reserve markets as the energy and reserve bidding problem.
Characterizing and exploiting the energetic flexibility of various types of electric energy resources has recently attracted growing interest because the need for flexibility in power grids is expected to increase as less predictable renewable energy resources, such as wind and solar, are connected to the power grid. In particular, the provision of SFR has been studied for heating, ventilation, and air-conditioning systems [2, 3, 4], plug-in electric vehicles [5, 6], and systems behaving like a generalized battery [7, 8], among others.
The energy and reserve bidding problem faced by the BRP is to decide on the amount of SFR to offer on the reserve market and to optimally trade on energy markets. These decisions are mutually dependent and constitute a trade-off problem between the revenues from offering reserves and the costs of procuring energy. The problem has recently been formulated as a stochastic optimization problem [9, 10] and as a robust optimization problem [2, 11, 12, 13, 14]. The robust formulations rely on the concept of affinely adjustable robust control (AARC) policies, which have first been applied to the regulation power context in [13, 14, 15, 16], building on the initial work in  and the references therein.
The provision of SFR services requires the tracking of an activation signal broadcast by the ISO on the timescale of a few seconds. The ability of a system to accurately follow such signals is limited by the physical properties of the system, in particular by power, power-ramp rate, and energy constraints. While power and energy constraints are considered by most AARC approaches, all works mentioned above neglect power ramp-rate constraints. However, the crucial importance of ramping constraints has been recognized in the context of grid balancing  and unit commitment [19, 20], and has lead to the adoption of new performance-based schemes for the remuneration of SFR provision by several ISOs . But the methods developed in [18, 19, 20] cannot be directly applied to the energy and reserve bidding problem considered here.
In our formulation of the bidding problem, we adopt the AARC formulation used in [2, 11, 12, 13, 14], and make two main contributions. First, in contrast to the aforementioned works, we explicitly incorporate constraints on power-ramp rates in our modeling framework to guarantee the feasibility and accurate provision of SFR. Instead of the commonly used piece-wise constant power schedules, we base our formulation on piece-wise affine and continuous power trajectories, which makes it possible to accurately model power-ramp rates on arbitrary timescales. We then derive a reformulation of the AARC optimization problem which is necessary to guarantee the feasibility of the piece-wise affine solutions. Second, in our problem formulation we include all the different timescales and market lead times involved in the bidding problem. This allows us to study how different energy and reserve market parameters affect the amount of SFR capacity a system can provide. These two contributions make our approach versatile and adaptable, and pave the way for applying the energy and reserve bidding problem in real-world settings, where accurate modeling of system constraints and market specifications is indispensable.
The energy and reserve bidding problem of a BRP is introduced in Section II. In Section III, we illustrate how piece-wise affine power schedules can account for finite ramp rates. Section IV provides a robust reformulation of the the energetic flexibility of an energy resource. It is used in Section V to formulate and solve the energy and reserve bidding problem. A conclusion is provided in Section VI.
Ii Problem Description
We consider a BRP that participates in energy and reserve markets and whose balance group comprises a single system. The BRP strives to schedule the electric energy generation and/or consumption of the system so as to maximize the profit made from trading energy and from offering SFR services over a planning horizon of duration . The bidding problem involves different timescales and requires the BRP to take decisions on multiple stages both offline, i.e. , prior to the time of delivery, and online, i.e. , at delivery.
Ii-a Offline decisions
1) Reserve capacity market: An ISO is responsible for the safe and reliable operation of the transmission grid. In particular, it is accountable for keeping the supply and demand of electric energy balanced at all times. To be able to compensate fluctuating generation and consumption and other unexpected disturbances in a timely manner, it procures ancillary services on dedicated markets ahead of time. Different ancillary services exist, and their specifications vary among ISOs [22, 1, 23]. We focus on Secondary Frequency Regulation111Secondary Frequency Regulation is also referred to as Frequency Restoration Reserves, Spinning Reserves, or Load-Frequency Control. and adopt the product specifications that apply to the Swiss SFR market, cf. : every week on Tuesday not later than at 1 p.m., a BRP can bid a constant and symmetric capacity for the subsequent week .222More precisely, multiple bids consisting of capacity-price pairs can be submitted. If the bid is accepted, the ISO has the right, but not the obligation, to ask the BRP to deviate from the planned power reference by at most units of power at any time during the subsequent week. The SFR tendering period is denoted by (1 week in Switzerland). We choose the planning horizon to comprise a single SFR tendering period, i.e. , week. In return for keeping the reserve capacity available over the time , the BRP receives the capacity reservation payment , where is the reserve capacity price .
2) Day-ahead energy market: On the day-ahead energy market, a BRP can trade energy for every time interval of duration of the next day. The Swiss day-ahead market runs on an hourly timescale, i.e. , h, and closes at 11 a.m. on the day before delivery . The outcome of the day-ahead markets is the energy schedule of energy quantities to be produced or consumed during each of the time intervals in the planning horizon . The day-ahead energy procurement costs are , where denotes the day-ahead market clearing prices, which are unknown at the time of bidding.
3) Intra-day energy market: To make adjustments to the day-ahead energy schedule during the day of delivery, the BRP can trade energy continuously on the intra-day markets for time intervals of durations . We focus on quarter-hourly energy block products, i.e. , min, as traded on the Swiss intra-day market with a lead time of 1 hour  and denote by the number of intra-day time slots in the planning horizon , i.e. , . The energy procurement costs of the intra-day energy schedule are , where denotes the intra-day market prices, which are unknown at the time of bidding.
Ii-B Online decisions
As will be discussed in detail in Section III, the day-ahead and intra-day energy schedules of a BRP are translated into a continuous-time power reference. The BRP is responsible for ensuring that its balance group follows this power reference in order to comply with all energy contracts concluded on the markets. BRPs are held accountable for any mismatches between their actual and planned energy schedules that arise from unforeseen outages of generation units or inaccuracies of load forecasts. The ISO constantly observes the state of the power grid and, in case of imbalances, takes measures to reestablish the nominal operating state. SFR is activated to bring the grid frequency back to its nominal value and make actual energy profiles match their corresponding schedules . This is achieved by adjusting the power consumption or generation of all energy resources that have offered SFR according to an activation signal computed and broadcast by the ISO . Power deviations due to SFR activation can result in energy deviations from the energy schedule of a BRP. In Switzerland, this so-called up- and down-regulation energy is measured over each time interval separately for positive and negative activation, and is denoted by and , respectively. The BRP is remunerated for tracking the activation signal based on the amount of up- and down-regulation energy delivered, i.e. , the remuneration is where are the corresponding regulation energy prices .
Ii-C The energy and reserve bidding problem
The energy and reserve bidding problem of the BRP is to decide how much reserve capacity to offer on the SFR market over the planning horizon and how to trade day-ahead and intra-day energy and , respectively, such as to maximize the total expected profit. These decisions must be made subject to the constraints that the reference power trajectory associated with the energy schedules and i) meets the energy needs and physical constraints of the systems in the balance group, ii) complies with the energy contracts concluded on the energy markets, and iii) keeps the SFR power capacity offered to the ISO available at all times. The more reserve capacity the BRP offers, the more restrictive become the constraints on possible trades in the energy markets. Consequently, offering reserves and trading energy constitutes a trade-off between the reserve reward , the potential remuneration of regulation energy , and the energy procurement costs .
Ii-D Different timescales
The bidding problem takes place on different timescales. Offline decisions are made on the timescales , , and corresponding to the reserve market, and the day-ahead and the intra-day energy market, respectively. Online activation of SFR occurs at the faster timescale . The dynamics of the system are discretized on an intermediate timescale , which depends on the time constants of the system, the frequency with which set points can be changed, and possible computational challenges of the resulting problem. Commonly, will lie in the range of 5–15 min. In general, it holds that . For simplicity, we assume that longer time horizons are integer multiples of shorter ones, and for define , , and the continuous time interval . Different timescales are used in different market regions. All timescales are used as parameters in our approach making it versatile and applicable to the various market settings. Here we consider the Swiss market setting and use week, h, min, min, and s.
Iii Energy Baseline and Power Reference
Iii-a Energy baseline
The day-ahead and intra-day energy trades of a BRP result in the energy market baseline , which comprises the total energy trades in each interval of duration in the planning horizon. The energy baseline is defined as
and is written in vector form as with and .
Iii-B Power reference
that the BRP is committed to follow. Whereas the energy content of over each time interval matches as required by market contracts, implementing the instantaneous changes of at the interface of subsequent market time intervals would require infinite ramping capabilities. Inaccurate tracking of the power reference (2) can cause the grid frequency to deviate from its nominal value when switching between market time intervals, making it necessary to activate frequency reserves [18, 19]. To alleviate this difficulty, we propose to translate the energy baseline into a continuous-time piece-wise affine  and continuous power reference whose breakpoint intervals are each of duration . The continuous-time power reference is fully defined by its breakpoint values according to
for , . An example power reference is given in Fig. 1. In accordance with ENTSO-E standards , the power reference comprises periods during which the power is constant and equal to , . These periods of constant power are connected by linear ramps, each of duration as illustrated in Fig. 1. We assume that is an integer multiple of and that . For the Swiss case of min, min, and min, the relation between and is the following:
Thus, the power reference can be expressed compactly as
where we encode (4) by . Note that (4) reverts to (2) in the limit case . For all , our choice (3) is continuous and, thus, allows us to model the power-ramp rates. We make the assumption that a BRP with energy baseline will use the piece-wise affine continuous-time power trajectory (3) defined by its breakpoint values (5) as its power reference.
Iii-C Activation of SFR
The SFR service is activated by the ISO by sending out an activation signal to all BRPs that offer SFR for the corresponding time interval. The activation signal is a discrete-time signal with , . The signal is computed by the ISO and broadcast sequentially on the timescale (e.g. , s in Continental Europe , s in Switzerland , and s in the USA ). That is, the activation becomes available to the BRP only at time . We interpret the discrete signal as the continuous-time piece-wise affine and continuous activation signal
where , . We denote the set of all such functions by , which is a closed set of continuous functions.
The target power level that the BRP is responsible to track continuously is a function of the continuous time and the activation and is given by
Compared with the power reference , which is piece-wise affine on the timescale , the activation can vary at the higher rate . Thus, accurate tracking of the target power level (7) is challenging. The minimum required tracking accuracy of is specified by the ISOs individually (cf.  for details on the Swiss requirements). In the USA, ISOs are required to base their regulation compensation payment not only on the quantity of regulation provided, but must also take into account how accurately the activation signal has been tracked . This motivates the use of a continuous power reference according to (5) and the explicit consideration of ramp-rate constraints.
Iii-D Adjustable energy schedules
The decision on the amount of SFR to be offered is made once for each reserve tendering period and cannot be changed afterwards. In contrast, energy schedules can be adjusted from day to day or intra-day by trading energy on respective markets. These adjustments can depend on past SFR activation, for example. Let be the activation signal averaged over time intervals of duration , i.e. ,
with parameters , , , and . The matrices and are used to average over time intervals of duration and , respectively. For instance, is block-diagonal with each block given by a row-vector of length whose elements all equal . By plugging (9) into (5) we can write the power reference as an affine function of the averaged activation:
The policies (9) describe how much energy should be traded on the day-ahead and intra-day energy markets depending on the average amount of SFR activated during past time intervals. The policies must be causal, i.e. , they can only depend on past activation, and must respect the different lead times of the markets. These requirements can be incorporated by imposing a particular structure onto and . Figure 4 illustrates which elements of and can be nonzero. Because the day-ahead market has a lead time of 13 h, the trading decisions for the second day can only depend on the first 11 h of the first day. Similarly, the min lead time of the intra-day market forces the first 5 rows of to be zero. The size of the bidding problem can be reduced by not taking into account all the available historic activation data in (9) but only considering the and most recent measurements. The policy parameters and determine the width of the blocks in and the number of off-diagonals in the lower-triangular part of , respectively, cf. Fig. 4.
Iv Description of flexibility
The decisions a BRP makes on the different energy and reserve markets are restricted by the requirement that the target power trajectory (7) must be feasible for all realizations of the unknown activation signal and must satisfy all physical constraints of the system. We consider a system with power, power ramp-rate, and state (e.g. energy) constraints.
Iv-a Power constraints
The power the system can draw from or feed into the power grid is limited. For all , , we require that
where denote the piece-wise constant bounds on power. The constraints above are satisfied if and only if for all it holds that
The robust counterparts of the above constraints are derived according to  by computing the maximum and minimum of with regard to according to (10). An analytic solution exists because each element of is restricted to : the power constraints (11) are satisfied if and only if for all
where denotes a column vector of ones of appropriate size.
Iv-B Power ramp-rate constraints
Limits on the rate at which power can vary over time play an important role, in particular for providing ancillary services with high accuracy. For all , , we require that the rate of change of the target power be bounded, i.e. ,
where denote the piece-wise constant power ramp-rate limits. The above inequalities are satisfied if and only if for all it holds that
The above inequalities illustrate that ramp-rate constraints limit the sum of the rate of change of the power reference and the rate of change of activated regulation power. They are equivalent to
with . The case of maximum negative ramp rates is illustrated in Fig. 5, where SFR activation changes from maximum positive activation () to maximum negative activation () within one control time step . In this case, maximum negative ramp rates are required during the time interval .
Iv-C State constraints
Many types of flexible energy resources owe their flexibility to an energy buffer with limited capacity. Examples of such systems are pumped hydro-power plants, batteries, heating and cooling systems, capacitors, and flywheels. Here we consider systems whose energy buffer level is governed by the linear time-invariant dynamics
with , . Exogenous (uncontrollable) inputs, such as the weather conditions in the case of heating systems, or trips in the case of electric-vehicle batteries, are summarized by and are assumed to be known. Because the bidding problem has to be solved ahead of time, the initial state is not known precisely, but is known to lie in the interval . The dynamics (17) are characterized by the parameters , where can be interpreted as the self-dissipation rate of the energy buffer, and determine the conversion efficiencies of exogenous inputs and electric energy into buffered energy, respectively. Our sign convention implies . The behavior of an ideal battery, for instance, can be modeled by setting and .
We consider the piece-wise constant state constraints
for all , , where denote the physical limits of the system state. The initial state is assumed to be feasible, i.e. , .
The exact evolution of the state (17) is unknown because it depends on the uncertain activation . For our case of piece-wise affine power references, the state can evolve non-monotonically during a discretization interval . To ensure that (18) holds would require bounding the extreme values of for . This, however, would result in non-convex constraints. Instead, we approximate the right-hand side of (17) by affine functions of time and define for every interval , , the auxiliary dynamics
with , , , and . For brevity we use to denote
If for all and
then the state constraints (18) are satisfied for all .
and note that , which, via (21), implies that . The same line of arguments holds for all subsequent subintervals. Consequently, in the limit case , we have that , , . ∎
Note that are affine functions of that can be integrated explicitly. In addition, the and the in the definitions (19)–(20) ensure that the minimum value of and the maximum value of over are reached at the edges of the interval, i.e. , at . Thus, (21) holds if for all and
which is equivalent to requiring that for all , , and
Iv-D Description of flexibility
The power, ramp-rate, and state constraints (11), (14), and (18), respectively, are satisfied if the offered SFR capacity together with the energy trading policies , meet the constraints (13), (16), and the robust counterparts of (26)-(28). All these constraints are linear in the decision variables and thus define a convex polytope parameterized by the system constraints parameters . This polytope can serve as a concise and convenient description of the flexibility the system offers with regard to trading energy in the day-ahead and intra-day markets and offering regulation power on the SFR market.
V Solving the Bidding Problem
V-a Formulation of the bidding problem
The energy and reserve bidding problem consists of finding the energy trading policies (9) and the reserve capacity that maximize the expected profit while satisfying all system constraints. The profit
is uncertain because of its dependency on the SFR activation signal and the market clearing prices. Thus, we maximize the expectation of (29) over the uncertain parameters , and write the energy and reserve bidding problem as the linear program
V-B Computational study
The results of the economic bidding problem (30) do not only depend on the market characteristics and technical constraints of the system, but are also influenced by market prices. An economic analysis of (30) can be found in . To investigate the maximum available SFR capacity independent of economic considerations, we solve (30) with objective and investigate the effects that different market time scales, lead times, and trading policies have on the maximum available SFR capacity . To facilitate the interpretation of the results, we consider an ideal battery modeled according to (17) with parameters , , and no exogenous input, i.e. , . The physical constraints of the battery are similar to those of a Tesla Powerwall 2 battery : kW, kWh, kWh, , and kWh. We use min, s, h, and . Table I provides the market settings and trading policy parameters considered and the results obtained. The maximum available SFR capacity and the minimum ramp rates required to offer this service are provided as quantities relative to the rated power .
As a reference case, the first setting described in Tab. I considers the case where the power reference cannot be adjusted via day-ahead or intra-day energy trades; it remains fixed regardless of potential SFR activation. This setting results in the least amount of SFR capacity, which, for the simple system at hand, can be computed analytically by dividing the available up- and down energy buffer capacity by the duration of the planning horizon: 7.5 kWh / (724) h 0.0446 kW, which corresponds to 0.89% of . More SFR capacity can be made available by adjusting the power reference via day-ahead energy trades depending on past activation. The available capacity increases as trading decisions are based on more data on past activation, see settings 2–6, where the value of is gradually increased. Considering more than 24 h of past activation data, i.e. , , does not yield more SFR capacity because the activation that occurred more than 24 h in the past has already been compensated for in the previous day-ahead market.
Whereas day-ahead adjustments of the power reference yield only minor increases of SFR capacity over the reference setting, intra-day adjustments make more than 50% of the rated power available for SFR, see settings 7–9. Reducing the intra-day market lead time from 1 h to 1/4 h, however, has only minor effects on , see settings 11–13. Reducing the planning horizon from 7 d to 1 d results in an increase of from 50.26% to 51.87%, see settings 7 and 11. The results also illustrate the ramp-rate constraints (15). Consider setting 13, for instance: Tracking SFR activation of size % of around a constant would require (% of )/s, whereas in reality the slightly larger amount (% of )/s is needed.
The results highlight the importance of the intra-day market which operates at faster time scales than the day-ahead market. It allows energy-constrained systems such as batteries to compensate for the regulation energy provided. The introduction of energy markets operating at even shorter time intervals, e.g. 5 min, could further increase the amount of available reserves. Additional reserves can be made available by reducing the length of the tendering period and by relaxing the requirement that a constant amount of SFR must be offered.
The bidding problem (30) has been solved with IBM ILOG®333ILOG and CPLEX are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. CPLEX®33footnotemark: 3 v12.6 on a system featuring an Intel® Xeon® 12-core CPU @2.4 GHz and 20 GB of RAM. The solving times are provided in the rightmost column of Tab. I. For fixed discretization time , the problem size grows quadratically with the planning horizon length . An efficient means to reduce the problem size is to reduce the number of free variables in the energy trading policies (9) via the parameters and , cf. Section III-D.
|[d]||[h]||[% ]||[(% )/s]|
The provision of accurate and reliable SFR services is important to compensate for imbalances in the power grid. We showed how to implement energy schedules in a way that is compliant with ENTSO-E regulations and allows one to explicitly consider limitations of the power-ramp rates of the system providing SFR. Based on piece-wise affine and continuous power trajectories, we derived a reformulation of the energy and reserve bidding problem as a robust linear program that accurately models the different market timescales and lead times. Our approach is versatile and applicable in various market settings. Computations show how adjusting the power reference via day-ahead and intra-day energy markets significantly increases the amount of available SFR capacity.