Regulation, Volatility and Efficiency in Continuous-Time Markets

Regulation, Volatility and Efficiency in Continuous-Time Markets

Arman C. Kizilkale and Shie Mannor Arman C. Kizilkale is with the Department of Electrical and Computer Engineering at McGill University, Montreal, Canada. arman@cim.mcgill.caShie Mannor is with the Department of Electrical Engineering at the Technion, Haifa, Israel. shie@ee.technion.ac.il
Abstract

We analyze the efficiency of markets with friction, particularly power markets. We model the market as a dynamic system with the demand process and the supply process. Using stochastic differential equations to model the dynamics with friction, we investigate the efficiency of the market under an integrated expected undiscounted cost function solving the optimal control problem. Then, we extend the setup to a game theoretic model where multiple suppliers and consumers interact continuously by setting prices in a dynamic market with friction. We investigate the equilibrium, and analyze the efficiency of the market under an integrated expected social cost function. We provide an intriguing efficiency–volatility no-free-lunch trade-off theorem.

I Introduction

The first attempts of privatization and deregulation of power industry took place in the 1980s starting in Chile and the UK [1]. After the restructuring of power markets in California in the late 1990s, price fluctuations have resulted in an estimate of $45 billion in higher electricity costs, lost businesses due to long blackouts, and a weakening economic growth according to the Public Policy Institution of California [2]. Even though such events have been mostly considered as market failures [3, 4], it was shown in [5] that the occurrence of choke-up prices (the maximum price a consumer is willing to pay) is intrinsic to markets with friction, and the market mechanism is efficient in a stylized model. Choke-up prices are observed in current market mechanisms regardless of being efficient, intrinsic, or market failure; this is undesirable and costly.

Dynamic pricing in electricity markets have interesting characteristics. Locational Marginal Pricing (LMP) schemes may determine very high prices for a region while a neighbour region might be assigned a low or even a negative price for the same amount of energy, where the supplier is actually willing to pay to consumers for the power they use. The constraints due to transmission congestion, voltage and thermal constraints, Kirchoff’s Laws and start-up and shut-down costs are the main reasons behind excess and lack of supply which cause volatile prices [6].

As shown in [5], the current deregulated market mechanism is efficient with respect to the infinite horizon social cost. However, the definition of efficiency depends on the social cost function defined. Many models do not penalize volatility in their cost functions. One can define volatility as rapid or unexpected changes in the price process. Several models of deterministic and stochastic volatility have been studied in the economics literature including the famous deterministic Black-Scholes formula [7], and the stochastic Heston’s extension [8], SABR [9] and GARCH [10] models. We are going to adopt a much simpler definition of volatility since our goal is to give a specialized analysis of volatility in power markets.

Several authors studied efficiency in power markets. Even though most studies are based on static frameworks, it was shown in [11] that under ramping constraints, markets might face prices not necessarily equal to the marginal cost price. A dynamic game model based on duopoly markets is analyzed in [12], and a dynamic competitive equilibrium for a stochastic market model is formulated and the role of volatility for the value of wind generation is presented in [13].

We model the power market through continuous dynamics and an integrated undiscounted cost function. The problem is presented as an optimal control problem, and the control action is defined as an increment process applied by the regulator. The HJB equation is solved and the resulting optimal control is presented. As a special case, in the class of linear quadratic cost functions, we analytically show that there is a trade-off between efficiency and non-volatility. In the second part of the paper we take a decentralized approach and define the market as a dynamic linear quadratic game among individual decision maker supplier and consumer agents. The agents are coupled through the price process. We show that this price process can be estimated, and the agents can calculate their best response actions based on this estimation. We show that these best response actions constitute an equilibrium and the trade-off theorem between efficiency and non-volatility is shown to hold in this dynamic game model as well.

In the first part of the paper we suggest a dynamic optimization framework for power markets: in Sec. II, we introduce the model we are going to use for the centralized control model. Demand , supply and price processes are defined for the social cost optimizer regulator agent with the corresponding cost function. In Sec. III, we present the optimal control that leads to a volatile price process. In Sec. IV, we define volatility and modify the social cost function to account for it. We solve the dynamic stochastic optimization problem for linear dynamics and a quadratic cost function and present the closed form solution. We show that there is a trade-off that can be quantified between efficiency and non-volatility, and present supporting simulations. In the second part of the paper we suggest a dynamic game-theoretic optimization framework: in Sec. V, the consumer agents , with their dynamics , the supplier agents with their dynamics , and the price process are defined with the corresponding cost functions for the consumers and suppliers. In this framework there is no regulator agent: the price process is solely determined in the market mechanism through the actions of the consumers and suppliers [14]. In Sec. VI, we first show the existence of best response actions for the game model, we present the closed form solutions, and finally we analyze the equilibrium properties of the system. In Sec. VI-E, we define volatility for this model, and show that the trade-off theorem can be extended to the multi-player game setup. We present supporting simulations in Sec. VI-F and conclude in Sec. VII.

Ii Model

In this section we define the optimization problem for the social cost optimizer in power markets. Here we call the optimizer the “regulator” (agent ). We define the three dimensional state process . We have , the demand process, , the supply process, and , the price process. Demand and supply dynamics are defined as

(1)

using deterministic continuous functions and with and , standard Wiener processes. The function is allowed to be a function of and , values of demand and price, and is allowed to be a function of and , values of supply and price processes.

We employ the following assumptions on the functions and in (1). The first assumption AII reflects friction for power markets. This assumption ensures that the instantaneous change in demand and supply processes with respect to a price change is constrained. This is one of the key properties of power dynamics: the suppliers and consumers are unable to respond to abrupt changes in the system instantly. The reason for the supplier’s sluggishness is slow ramp up in power production, whereas for the consumers it is usually not handy or very complicated and costly to startup and shutdown a running machine or a household. The second assumption, AII, reflects natural characteristics of demand and supply dynamics: demand is a decreasing function of the price, whereas supply is an increasing function of the price.

A1: For constant , and

An immediate example is a linear function of the form with of class .

A2: is a strictly decreasing function of , whereas is strictly increasing.

This assumption ensures that an increase in price is reflected on the deterministic portion of decreasing demand dynamics and increasing supply dynamics.

We also adopt the assumption below for initial values of the processes and the disturbance process:

A3: are mutually independently distributed bounded initial conditions, and are mutually independent and independent of the initial conditions. Instantaneous variances of the disturbance processes, , are bounded.

We adopt the stepwise price adjustment model [15] for the optimizer (so called regulator agent ), where the bounded input control process controls the amount of the increment. The price process controlled by agent ’s input is defined as

(2)

The actions of is the set which is simply the constrained price adjustment. observes the demand and supply processes and taking into consideration their dynamics, cost function and the constraint on price increment, takes an action in terms of increasing or decreasing the power price. This action is intended to control market dynamics by only applying increments on the price process.

Following [5], the individual loss functions of the consumer and supplier are defined respectively:

Here, , with polynomial growth with power or less, i.e., , where denotes the family of all bounded functions which are twice differentiable. The function is the production cost, and is strictly convex and strictly increasing with respect to . One needs to work on a realistic production cost function in order to have a reasonable power market model. We note that in real power markets, production cost is not a convex function. The startup and shutdown costs, transmission line constraints, weather fluctuations all affect the production cost function. However, if one neglects the startup and shutdown costs, the cost function resembles a convex function [11, see Figure 1]. For our model we will assume a continuous convex cost. The constant is the value the consumer obtains for a unit of power. The blackout is denoted by , with polynomial growth with power or less, i.e., , is convex, zero on and strictly decreasing on , where denotes the reserve, . In other words, if the total consumption in the system can not be met, blackout cost is paid. In the spot market, the consumer, , pays , the price of all the supply bought, to the supplier, . Note that is multiplied by the supplied portion of his demand. Blackout cost is a function of the unmet demand. Further note that the supplier pays for all the cost of production, and gains unit price multiplied with all the units of supply bought by the consumer agent . Finally, we employ the following integrated expected social cost function that is simply the sum of the consumer and the supplier loss functions integrated in time:

(3)

In the section that follows, we consider the optimality of the cost function presented above with the dynamics (1), the control (2) and the cost function (3) under AII, AII and AII.

Iii Centralized Control Formulation

In this section we analyze the optimal control problem in terms of the state vector . As stated before, this is a centralized control problem for the regulator agent . In principle ’s objective is to regulate demand and supply processes using the price increments as the control tool, so that the best social outcome is achieved. In this section we show that the optimal control of the regulator is a “bang-bang” control, which leads to volatile prices. We write (1) and (2) in vector form with stochastic dynamics as

(4)

where is a standard Wiener process. We set and,

The loss function of (3) is rewritten here as . The admissible control for the regulator is specified as adapted to and . Therefore, the regulator can at most increase or decrease the price with unit and at each iteration. Finally, the cost associated with (4) and a control is specified to be . Further, we set the value function

(5)

The theorem that follows claims the existence and uniqueness of the optimal control to the problem (5).

Theorem III.1

There exists a unique such that , where is the initial state at time , and if is another control such that , then only on a set of times of Lebesgue measure zero.

Proof: The proof is given in Appendix A.

Now that we have shown the existence and uniqueness of a control, we check for approaches to compute the optimal solution. For a function class : (i) , (ii) where depend on , (iii) , we write the HJB Equation

(6)

A classical solution to the HJB Equation (6) does not exist as is not of full rank in (4) [16]. Therefore, viscosity solutions are adopted.

Definition III.1

Viscosity solution: [17, Sec. 4, Def. 5.1]

A function is a viscosity subsolution to the HJB equation (6) if , and for any , whenever obtains a local maximum at , we have

(7)

A function is called a viscosity supersolution to (6) if , and whenever takes a local minimum at , in (7) the inequality is changed to . A value function is a viscosity solution if it is a viscosity subsolution and a viscosity supersolution.

Theorem III.2

The value function defined in (5) is the unique viscosity solution to the HJB equation (6) in the class .

{proof}

H4.1 and H4.2 of [18] are satisfied. Theorem 4.1 of [18] proves that defined in (5) is a viscosity solution to the HJB equation (6), and Theorem 4.3 of [18] proves that the solution is a unique solution to (6) in the class .

Iii-a Perturbation Method

In order to make the matrix full rank, we add to (6) [19]. For a function class : (i) , (ii) where depend on , (iii) , we write the HJB Equation

(8)

where .

Lemma III.3

[20, Sec. 6, Theorem 4] For each

where the constant depends on , and .

Lemma III.4

[19, Lemma 6.2] Let be bounded, a solution of (8) in with continuous in and . Then there exists a constant such that

where the constant depends only on , the constant in AII and , defined for and .

Theorem III.5

The perturbed HJB equation (8) has a unique classical solution in the class for all .

{proof}

We employ an approximation approach. Let us first take . For integer , let be such that for , for , and . Let be the solution to

(9)

where .

For fixed and , for any , satisfies (8) for . Lemma III.4 ensures that are uniformly bounded on . For any , , by local estimates

is uniformly bounded, where denotes a Sobolev type norm, where denotes the space of -th power integrable functions on . Take , and by the Hölder estimates, satisfies a uniform Hölder condition on any compact subset of . Moreover, satisfy a uniform Hölder condition on such a . At this point we employ Arzela-Ascoli theorem and take a subsequence such that converge uniformly to on , respectively, as , where satisfies (8) and is in the class due to the growth condition on and the compactness of . In the next theorem, we use the Itō’s formula to show that is the value function to a related stochastic control system, and thus it is a unique solution to (8) in the class .

Theorem III.6

Let and . Define as solution to (8) and as solution to (6) for the admissible control set . Then uniformly on .

Proof:

For a standard Wiener process, we can define an alternative control action in the form of a stochastic differential equation . The resulting value function can be shown to be a viscosity solution to (8), and this solution is unique (see Chapter 4, [17]). For a fixed , we have . We recall from (5) that is the infimum of among non-anticipative controls in . Let be as in the polynomial growth conditions for and , Since is compact, AII together with Lemma III.3 imply that and are bounded uniformly with respect to and . It follows that is uniformly bounded. One can use Lebesgue’s dominated convergence theorem to obtain , and as follows. By adopting Arzela-Ascoli Theorem similar to the methodology that was employed in the proof of Theorem III.5, one can obtain uniformly on , as . \qed

This gives us the following result:

Corollary 1

For the function class the solution to the perturbed HJB Equation (8) is found as:

(10)

where was previously defined as .

When we look at the the perturbed HJB Equation (8), the bound is a direct estimate, the value function is differentiable everywhere in the function class , and due to the constraint defined on the control action, the optimal control is represented as a bang-bang control. Hence, the optimal control is found as a single switch. At the boundary we have . Therefore, one can numerically solve (8).

In Theorem III.1 we showed the existence of an optimal control to the problem (5). Due to the problematic nature of stochastic differential equations, we have seen that the solution of an optimal control in “classical sense” may not exist. This leads us to formulate a suboptimal approach. The convergence of the suboptimal solution to the optimal solution was shown.

The control is shown to be a simple single switch. This has significant consequences, i.e., we proved that the regulator needs to increase the price increment to the possible maximum or decrease it to the possible minimum depending on the value obtained from (10). Due to AII, the effect of price on demand and supply is constrained. Therefore, a certain amount of time is needed in order to adjust the levels of demand and supply in the system. For cases where demand is much bigger than supply or supply is much bigger than demand, the maximal increment has to be applied for a long period of time. Hence, volatile prices are the optimal outcome of the market with respect to the cost function (3).

Note that AII is important both for technical reasons and for modeling reasons. In addition to the fact that AII models friction, if AII is removed, the polynomial growth of the value function also may not be satisfied. Moreover, for a hypothetical frictionless market, a single increment on price would adjust demand and supply levels to the desired levels instantly; thus, less volatility would be expected. Indeed, for a completely deterministic frictionless system, volatility would be zero.

Iv Efficiency–volatility Trade-Off

Non-volatility and efficiency are two desirable properties of power markets. In this section we show that these two notions contradict each other in a market model with friction. Therefore, one has to trade-off non-volatility and efficiency in designing the market mechanism.

The optimal control policy for the system (1) and the price process due to the nature of the optimal control (10) were discussed in the previous section. Since the demand and supply processes are defined by stochastic differential equations, they fluctuate on their trajectories and the regulator modifies the price process for the optimal outcome. The highest cost is paid when the difference between demand and supply is the highest.

In this section we prove that no efficient regulation strategy can exist that maintains a smooth price process when supply and demand are defined by mean-reverting stochastic differential equations.

We form a function that penalizes the control action . Recall the loss function defined in (3). We adopt the stepwise price adjustment model defined in (2), where the input control process controls the amount of the increment. The cost associated with the system is defined as

(11)

where we add to the term (3) and is the volatility coefficient. We will prove that if the volatility coefficient decreases, the expected cost decreases. In other words, if high volatility is not allowed, the social cost defined in (3) increases.

We define efficiency as the quantity obtained when the expected cost is multiplied by -1 taken out the control action penalizing part: . Volatility on the other hand is defined by the price fluctuation measured by .

We require one more assumption here:

A4: The supply process and the demand process are linear mean-reverting processes that have bounded variances and admit stationary probability distributions in case of time invariant means.

As a special case, we study a linear quadratic cost function of the form

(12)

where , and , and are constant values. Employing AIV, we have the dynamics

(13)

where is a standard Wiener process, , and are in the form of

(14)

where ‘’ denotes a bounded constant.

Iv-a Existence and Uniqueness of the Optimal Control

From now on, we will work on (12) and (13). We take the admissible control set . The minimum cost-to-go from any initial state and any initial time is described by the value function which is defined by . The optimal control problem is well defined with the Hamilton-Jacobi-Bellman (HJB) Equation

(15)

where .

As discussed earlier in Sec. III, due to the lack of uniform parabolicity, standard solutions may be hard to obtain. Viscosity solutions are adopted in these circumstances. Therefore we add the term to (15) and obtain uniform parabolicity. Equation (15) then becomes

(16)

where .

Equation (16) has a unique solution as stated in the following theorem.

Theorem IV.1

Equation (16) has a unique classical solution for the admissible control set for all .

Proof: The proof is very similar to the Proof of Theorem III.5, therefore omitted.

In the theorem below, we prove that the solution to the perturbed value function (16) converges uniformly to the value function obtained from the HJB Equation (15).

Theorem IV.2

Let and . Define as solution to (16) and as solution to (15) for the admissible control set . Then uniformly on .

The proof is very similar to the proof of Theorem III.6, therefore omitted.

Iv-B Closed Form Solution

Standard arguments [21, Section 2.3] show that is quadratic in . Furthermore, at any point and the minimum cost-to-go is quadratic in . Consequently, one can model of the form that satisfies the boundary condition . Substituting in (15) and applying first order optimization gives

(17)

Solving the closed loop expression we get the ODEs:

(18)
(19)
(20)

with boundary conditions , and . The linear quadratic optimal control problem admits a unique optimum feedback controller given by (17) which obtains the minimum value of the cost function .

Iv-C Efficiency–Volatility Trade-off

We would like to look at the relation between , the volatility coefficient, and the state penalizing part of the cost function obtained when the volatility term is removed from the cost function. We define the state penalizing cost as

(21)

which is denoted as efficiency when multiplied by .

Theorem IV.3

Suppose AII-AIV hold. For all , the state penalizing cost portion (21) of the cost function (12) using optimal control is an increasing function of .

Proof: The proof is presented in Appendix B.

Increasing the volatility coefficient increases social cost, therefore decreases efficiency, while decreasing the coefficient decreases the cost, hence increases efficiency. On the other hand increasing the volatility coefficient decreases volatility, whereas decreasing volatility coefficient increases volatility. Therefore, there is a trade-off between social efficiency and non-volatility.

Iv-D Simulations

Iv-D1 Analytical Supportive Simulation

Here we simulate a power market. We use Euler-Maruyama Method [22] for discretization of the stochastic differential equations. The dynamics equations are , where , with the initial conditions . We use mean-reverting processes with time varying means. The power market we simulate consists of a demand process with mean MWh, and a supply process with mean MWh. Therefore for a price of $50 per MWh, the supplier is expected to produce MW of power, whereas the demand in the system is also expected to be 25 MW. In accordance with AII, the demand is an decreasing function of price, whereas supply is increasing. We calculate using Theorem IV.3 using a range of values of and present the result in Fig. 2, and as expected, it is always positive. Also, as expected it is a convex function; the value is very high for small values of and converges to 0 as increases. Increasing , the volatility coefficient, corresponds to decreasing volatility which ends up with a cost increase as for all . In Fig. 2 we present the trade-off between the efficiency and the non-volatility. The numbers are normalized, and one can see that in a market with higher volatility the efficiency is higher. Here, on the axis 0 corresponds to the situation where is very large and 1 corresponds to the situation where . On the axis, the corresponding values are normalized, so that 0 is the lowest and 1 is the highest efficiency that can be obtained.

Fig. 1:
Fig. 2: Trade-off

Iv-D2 Numerical Simulation

Here we present a couple of simulations showing the dynamics when and . The high volatility in Fig. 4 compared to the low volatility in Fig. 4 can be observed. One can also notice the effect of volatility on stability.

Also in Fig. 4 the optimal actions of the regulator agent can be observed at 4 points on the trajectory. At P1, the demand goes up due to stochasticity and the regulator acts with full force to increase the price, so that stability can be obtained. At P2, price gets high, and the demand is taken under control; gradually the regulator decreases the price. Between 60 seconds and 80 seconds, we see that supply follows a higher level than the demand. The regulator acts to take the price down to a local minimum at P3. Then, until P4 the regulator gradually increases the price until it comes to a local maximum at P4.

Fig. 3: Dynamics when r = 0.01
Fig. 4: Dynamics when r = 1000

Now we present two more simulations with . The effect of the initial state on the trajectory is observed here. In Fig. 6 initially, demand is higher than the supply, whereas in Fig. 6 demand is lower than the supply. As expected, the price process becomes very volatile in early stages to stabilise the market.

Fig. 5: Dynamics when initial supply is higher than demand
Fig. 6: Dynamics when initial supply is lower than demand

Finally, we present an experimental result showing the relation between and the average absolute difference between supply and demand dynamics. Recall that high costs are paid when this difference is high, and as seen in Fig. 7, as increases the average absolute difference increases. The axis is drawn on a logarithmic scale in order to capture the graph on lower values of .

Fig. 7: Average absolute difference between demand and suply

V Decentralized Control Formulation

We define a continuous dynamic game for consumers and suppliers. The agents continuously submit their bids as price-quantity graphs, and the system announces the resulting price. Agents buy or sell corresponding shares of supplies according to their bids. One important notion is that future demand and supply processes are dependent on the price process, which is determined instantly by the agents’ price-quantity graphs shaped by their actions.

We have the set of agents . We define the family of three dimensional state processes for the consumers and two dimensional state processes for the suppliers. The initial conditions are mutually independently distributed bounded random variables which are independent of the standard Wiener processes . The process is the demand dynamics for agent , the process is the supply it receives, and the process is the parameter it applies to its pre-announced price-quantity graph function . For the supplier side is the current supply and is the parameter for the price-quantity graph . Here and are the price-quantity graphs that the consumers and the suppliers submit to the market clearing price functional for the instant price determination. The dynamics for the consumers and the suppliers for are given as

(22)

The actions of the agents control the size of the increments for . The functional is allowed to be a function of and , values of the demand of the consumer agent , the price and its price-quantity graph; and is allowed to be a functional of , values of the supply of the supplier , the price and its price-quantity graph.

Following [5], the individual loss function of a consumer and a supplier are defined respectively:

(23)

Finally, the cost functions associated with each consumer, each supplier and corresponding control actions , and , are specified to be

(24)

We employ AII for initial values and the disturbance processes, and AII on the functions and . Moreover,

A5: , is a strictly decreasing function of , whereas , is strictly increasing. The price-quantity graphs for the consumers are decreasing functions of in the form of , whereas the price-quantity graphs are increasing in the form of , for the suppliers. Functions and are Lipschitz continuous on with Lipschitz constants , and . Consequently, for some , the market clearing price function is a linear function in the form of .

This assumption limits the model to a price process parameterized by and obtained by price-quantity graph functions submitted by the consumer and supplier agents: