A Derivation of Equation (18)

Adaptive Electricity Scheduling in Microgrids


Microgrid (MG) is a promising component for future smart grid (SG) deployment. The balance of supply and demand of electric energy is one of the most important requirements of MG management. In this paper, we present a novel framework for smart energy management based on the concept of quality-of-service in electricity (QoSE). Specifically, the resident electricity demand is classified into basic usage and quality usage. The basic usage is always guaranteed by the MG, while the quality usage is controlled based on the MG state. The microgrid control center (MGCC) aims to minimize the MG operation cost and maintain the outage probability of quality usage, i.e., QoSE, below a target value, by scheduling electricity among renewable energy resources, energy storage systems, and macrogrid. The problem is formulated as a constrained stochastic programming problem. The Lyapunov optimization technique is then applied to derive an adaptive electricity scheduling algorithm by introducing the QoSE virtual queues and energy storage virtual queues. The proposed algorithm is an online algorithm since it does not require any statistics and future knowledge of the electricity supply, demand and price processes. We derive several “hard” performance bounds for the proposed algorithm, and evaluate its performance with trace-driven simulations. The simulation results demonstrate the efficacy of the proposed electricity scheduling algorithm.


mart grid, Microgrids, distributed renewable energy resource, Lyapunov optimization, stability.

1 Introduction

Smart grid (SG) is a modern evolution of the utility electricity delivery system. SG enhances the traditional power grid through computing, communications, networking, and control technologies throughout the processes of electricity generation, transmission, distribution and consumption. The two-way flow of electricity and real-time information is a characteristic feature of SG, which offers many technical benefits and flexibilities to both utility providers and consumers, for balancing supply and demand in a timely fashion and improving energy efficiency and grid stability. According to the US 2009 Recovery Act [2], an SG will replace the traditional system and is expected to save consumer cost and reduce America’s dependence on foreign oil. These goals are to be achieved by improving efficiency and spurring the use of renewable energy resources.

Microgrid (MG) is a promising component for future SG deployment. Due to the increasing deployment of distributed renewable energy resources (DRERs), MG provides a localized cluster of renewable energy generation, storage, distribution and local demand, to achieve reliable and effective energy supply with simplified implementation of SG functionalities [3, 4]. A typical MG architecture is illustrated in Fig. 1, consisting of  DRERs (such as wind turbines and solar photovoltaic cells), energy storage systems (ESS), a communication network (e.g., wireless or powerline communications) for information delivery, an MG central controller (MGCC), and local residents. The MG has centralized control with the MGCC [4], which exchanges information with local residents, ESS’s, and DRERs via the information network. There is a single common coupling point with the macrogrid. When disconnected, the MG works in the islanded mode and DRERs and ESS’s provide electricity to local residents. When connected, the MG may purchase extra electricity from the macrogrid or sell excess energy back to the market [5].

Figure 1: Illustrate the microgrid architecture.

The balance of electricity demand and supply is one of the most important requirements in MG management. Instead of matching supply to demand, smart energy management matches the demand to the available supply using direct load control or off-peak pricing to achieve more efficient capacity utilization [3]. In this paper, we develop a novel control framework for MG energy management, exploiting the two-way flows of electricity and information. In particular, we consider two types of electricity usage: (i) a pre-agreed basic usage that is “hard”-guaranteed, such as basic living usage, and (ii) extra elastic quality usage exceeding the pre-agreed level for more comfortable life, such as excessive use of air conditioners or entertainment devices. In practice, residents may set their load priority and preference to obtain the two types of usage [6]. The basic usage should be always satisfied, while the quality usage is controlled by the MGCC according to the grid status, such as DRER generation, ESS storage levels and utility prices. The MGCC may block some quality usage demand if necessary. This can be implemented by incorporating smart meters, smart loads and appliances that can adjust and control their service level through communication flows [5]. To quantify residents’ satisfaction level, we define the outage percentage of the quality usage as Quality of Service in Electricity (QoSE), which is specified in the service contracts [7]. The MGCC adaptively schedules electricity to keep the QoSE below a target level, and accordingly dynamically balance the load demand to match the available supply.

In this paper, we investigate the problem of smart energy scheduling by jointly considering renewable energy distribution, ESS management, residential demand management, and utility market participation, aiming to minimize the MG operation cost and guarantee the residents’ QoSE. The MGCC may serve some quality usage with supplies from the DRERs, ESS’s and macrogrid. On the other hand, the MG can also sell excessive electricity back to the macrogrid to compensate for the energy generation cost. The electricity generated from renewable sources is generally random, due to complex weather conditions, while the electricity demand is also random due to the random consumer behavior, and so do the purchasing and selling prices on the utility market. It is challenging to model the random supply, demand, and price processes for MG management, and it may also be costly to have precise, real-time monitoring of the random processes. Therefore, a simple, low cost, and optimal electricity scheduling scheme that does not rely on any statistical information of the supply, demand, and price processes would be highly desirable.

We tackle the MG electricity scheduling problem with a Lyapunov optimization approach, which is a useful technique to solve stochastic optimization and stability problems [8]. We first introduce two virtual queues: QoSE virtual queues and battery virtual queues to transform the QoSE control problem and battery management problem to queue stability problems. Second, we design an adaptive MG electricity scheduling policy based on the Lyapunov optimization method and prove several deterministic (or, “hard”) performance bounds for the proposed algorithm. The algorithm can be implemented online because it only relies on the current system status, without needing any future knowledge of the energy demand, supply and price processes. The proposed algorithm also converges exponentially due to the nice property of Lyapunov stability design [9]. The algorithm is evaluated with trace-driven simulations and is shown to achieve significant efficiency on MG operation cost while guaranteeing the residents’ QoSE.

The remainder of this paper is organized as follows. We present the system model and problem formulation in Section 2. An adaptive MG electricity scheduling algorithm is designed and analyzed in Section 3. Simulation results are presented and discussed in Section 4. We discuss related work in Section 5. Section 6 concludes the paper.

Symbol Description
total number of residents
total number of batteries
total number of slots
energy level for battery at time slot
recharging energy for battery at time slot
discharging energy for battery at time slot
maximum battery energy level for battery
minimum battery energy level for battery
maximum supported recharging energy for batter in a slot
maximum supported discharging energy for battery in a slot
average quality usage arrival rate for resident
average outage rate of quality usage for resident in MG
target QoSE for resident in MG
quality usage of residents in time slot
maximum quality usage of resident in a single slot
basic electricity usage of resident in time slot
available electricity from DRERs to supply quality usage in
time slot
electricity generated from DRERs in time slot
electricity purchased from macrogrid in time slot
electricity sold on the market in time slot
electricity to the resident
purchasing price on the utility market in time slot
selling price ob the utility market in time slot
indicator function for outage events of quality usage of
resident in time slot
minimum purchasing price of utility from macrogrid
maximum purchasing price of utility from macrogrid
minimum selling price of utility to macrogrid
maximum selling price of utility to macrogrid
battery virtual queue for the battery
QoSE virtual queue for the resident
states of the virtual queues and
Lyapunov function
Lyapunov one step drift
proposed scheduling policy including ,
optimal objective value of problem (9)
relaxed scheduling policy for problem 25
optimal objective value of problem (25)
Table 1: Notation

2 System Model and Problem Formulation

2.1 System Model


We consider the electricity supply and consumption in an MG as shown in Fig. 1. We assume that the MG is properly designed such that a portion of the electricity demand related to basic living usage (e.g., lighting) from the residents, termed basic usage, can be guaranteed by the minimum capacity of the MG. There are randomness in both electricity supply (e.g., weather change) and demand (e.g., entertainment usage in weekends). To cope with the randomness, the MG works in the grid-connected mode and is equipped with ESS’s, such as electrochemical battery, superconducting magnetic energy storage, flywheel energy storage, etc. The ESS’s store excess electricity for future use.

The MGCC collects information about the resident demands, DRER supplies, and ESS levels through the information network. When a resident demand exceeds the pre-agreed level, a quality usage request will be triggered and transmitted to the MGCC. The MGCC will then decide the amount of quality usage to be satisfied with energy from the DRERS, the ESS’s, or by purchasing electricity from the macrogrid. The MGCC may also decline some quality usage requests. The excess energy can be stored at the ESS’s or sold back to the macrogrid for compensating the cost of MG operation.

Without loss of generality, we consider a time-slotted system. The time slot duration is determined by the timescale of the demand and supply processes.

Energy Storage System Model

The system model is shown in Fig. 2. Consider a battery farm with independent battery cells, which can be recharged and discharged. We assume that the batteries are not leaky and do not consider the power loss in recharging and discharging, since the amount is usually small. It is easy to relax this assumption by applying a constant percentage on the recharging and discharging processes. For brevity, we also ignore the aging effect of the battery and the maintenance cost, since the cost on the utility market dominates the operation cost of MGs.

Figure 2: The system model considered in this paper.

Let denote the energy level of the the th battery in time slot . The capacity of the battery is bounded as


where is the maximum capacity, and is the minimum energy level required for battery , which may be set by the battery deep discharge protection settings. The dynamics over time of can be described as


where and are the recharging and discharging energy for battery in time slot , respectively. The charging and discharging energy in each time slot are bounded as


In each time slot , and are determined such that (1) is satisfied in the next time slot.

Usually the recharging and discharging operations cannot be performed simultaneously, which leads to


Energy Supply and Demand Model

Consider residents in the MG; each generates basic and quality electricity usage requests, and each can tolerate a prescribed outage probability for the requested quality usage part. The MGCC adaptively serves quality usage requests at different levels to maintain the QoSE as well as the stability of the grid. The service of quality usage can be different for different residents, depending on individual service agreements.

Let be the average quality usage arrival rate, and a prescribed outage tolerance (i.e., a percentage) for user . The average outage rate for the quality usage, , should satisfy


At each time , the quality usage request from resident is units, which is an i.i.d random variable with a general distribution and mean . The average rate is according to the Law of Large Numbers.

The DRERs in the MG generate units of electricity in time slot . can offer enough capacity to support the pre-agreed basic usage in the MG, which is guaranteed by islanded mode MG planning. The electricity is transmitted over power transmission lines. Without loss of generality, we assume the power transmission lines are not subject to outages and the transmission loss is negligible. Let be the pre-agreed basic usage for resident in time slot , which can be fully satisfied by , i.e., , for all . In addition, some quality usage request may be satisfied if . Let be the energy allocated for the quality usage of resident . We have


We define a function to indicate the amount of quality usage outage for resident , as . Then the average outage rate can be evaluated as .

The MGCC may purchase additional energy from the macrogrid or sell some excess energy back to the macrogrid. Let denote the energy purchased from the macrogrid and the energy sold on the market in time slot , where and are determined by the capacity of the transformers and power transmission lines. Since it is not reasonable to purchase and sell energy on the market at the same time, we have the following constraints


To balance the supply and demand in the MG, we have


Utility Market Pricing Model

The price for purchasing electricity from the macrogrid in time slot is per unit. The purchasing price depends on the utility market state, such as peak/off time of the day. We assume finite , which is announced by the utility market at the beginning of each time slot and remains constant during the slot period [10]. Unlike prior work [10], we do not require any statistic information of the process, except that it is independent to the amount of energy to be purchased in that time slot.

If the MGCC determines to sell electric energy on the utility market, the selling price from the market broker is denoted by in time slot , which is also a stochastic process with a general distribution. We also assume is known at the beginning of each time slot and independent to the amount of energy to be sold on the market. We assume , and for all . That is, the MG cannot make profit by greedily purchasing energy from the market and then sell it back to the market at a higher price simultaneously.

2.2 Problem Formulation

Given the above models, a control policy is designed to minimize the operation cost of the MG and guarantee the QoSE of the residents. We formulate the electricity scheduling problem as

minimize: (9)
s.t. (1), (3), (4), (5), (6), (7), (8)
battery queue stability constraints.

Problem (9) is a stochastic programming problem, where the utility prices, generation of DRERs, and consumption of residents are all random. The solution also depends on the evolution of battery states. It is challenging since the supply, demand, and price are all general processes.

Virtual Queues

We first adopt a battery virtual queue that tracks the charge level of each battery :


where is a constant for the trade-off between system performance and ensuring the battery constraints. This constant is carefully selected to ensure the evolution of the battery levels always satisfy the battery constraints (1), which will be examined in Section 3.3. The virtual queue can be deemed as a shifted version of the battery dynamics in (2) as


These queues are “virtual” because they are maintained by the MGCC control algorithm. Unlike an actual queue, the virtual queue backlog may take negative values.

We next introduce a conceptual QoSE virtual queue , whose dynamics are governed by the system equation as


where .

Theorem 1.

If an MGCC control policy stabilizes the QoSE virtual queue , the outage quality usage of resident will be stabilized at the average QoSE rate .


According to the system equation (12), we have


Summing up the inequalities in (13), we have


Dividing both sides by and letting go to infinity, we have

Note that is finite. If is rate stable by a control policy , it is finite for all . We have , which yields due to the definitions of and . ∎

Problem Reformulation

With Theorem 1, we can transform the original problem (9) into a queue stability problem with respect to the QoSE virtual queue and the battery virtual queues, which leads to a system stability design from the control theoretic point of view. We have a reformulated stochastic programming problem as follows.

minimize: (15)
s.t. (3), (4), (6), (7), (8)
Battery and QoSE virtual queue stability

Theorem 1 indicates that QoSE provisioning is equivalent to stabilizing the QoSE virtual queue , while stabilizing the virtual queues (11) ensures that the battery constraints (1) are satisfied. We then apply Lyapunov optimization to develop an adaptive electricity scheduling policy for problem (15), in which the policy greedily minimize the Lyapunov drift in every slot to push the system toward stability.

2.3 Lyapunov Optimization

We define the Lyapunov function for system state with dimension as follows, in which and .


which is positive definite, since when and . We then define the conditional one slot Lyapunov drift as


With the drift defined as in (17), it can be shown that


where is a constant. The derivation of (18) is given in Appendix A.

To minimize the operation cost of the MG, we adopt the drift-plus-penalty method [11]. Specifically, we select the control policy to minimize the bound on the drift-plus-penalty as:


where is defined in Section 2.2.1 for the trade-off between stability performance and operation cost minimization. Given the current virtual queue states and , market prices and , available DRERs energy , and the resident quality usage request , the optimal policy is the solution to the following problem.

minimize: (20)

Since the control policy is only applied to the last three terms of (20), we can further simplify problem (20) as

minimize: (21)

which can be solved based on observations of the current system state .

3 Optimal Electricity Scheduling

3.1 Properties of Optimal Scheduling

With the Lyapunov penalty-and-drift method, we transform problem (15) to problem (21) to be solved for each time slot. The solution only depends on the current system state; there is no need for the statistics of the supply, demand and price processes and no need for any future information. The solution algorithm to this problem is thus an online algorithm. We have the following properties for the optimal scheduling.

Lemma 1.

The optimal solution to problem (21) has the following properties:

  1. If , we have ,

    1. If , the optimal solution always selects ; if , the optimal solution always selects .

    2. If , the optimal solution always selects ; if , the optimal solution always selects .

  2. When , we have ,

    1. If , the optimal solution always selects ; if , the optimal solution always selects .

    2. If , the optimal solution always selects ; if , the optimal solution always selects .

The proof of Lemma 1 is given in Appendix B.

Lemma 2.

The optimal solution to the battery management problem has the following properties:

  1. If , the optimal solution always selects .

  2. If , the optimal solution always selects .

The proof of Lemma 2 is given in Appendix C.

Lemma 3.

The optimal solution to the QoSE provisioning problem has the following properties:

  1. If , the optimal solution always selects .

  2. If , the optimal solution always selects .

The proof directly follows Lemma 1 and is similar to the proof of Lemma 2. We omit the details for brevity.

Lemma 1 provides useful insights for simplifying the algorithm design, which will be discussed in Section 3.2. The intuition behind these lemmas is two-fold. On the ESS management side, if either the purchasing price or the selling price is low, the MG prefers to recharge the ESS’s to store excess electricity for future use. On the other hand, if either or is high, the MG is more likely to discharge the ESS’s to reduce the amount of energy to purchase or sell more stored energy back to the macrogrid. On the QoSE provisioning side, if either or is high and the quality usage is low, the MG is apt to decline the quality usage for lower operation cost. On the other hand, if either or is low and is high, the quality usage are more likely to be granted by purchasing more energy or limiting the sell of energy.

3.2 MG Optimal Scheduling Algorithm

In this section, we present the MG control policy to solve problem (21). Given the current virtual queue state , market prices and , quality usage and available energy from the DRERS for serving quality usage, problem (21) can be decomposed into the following two linear programming (LP) sub-problems (since one of and must be zero, see (7)).

minimize: (22)
minimize: (23)

In sub-problem (22), we set if , and if according to Lemma 1. Also, if , we set ; otherwise, we reset constraint (6) to a smaller search space of . We take a similar approach for solving sub-problem (23) by replacing with . Then we compare the objective values of the two sub-problems and select the more competitive solution as the MG control policy. The complete algorithm is presented in Algorithm 3.2.


[!t]\SetAlgoLinedMGCC initializes the QoSE target to and the virtual queues backlogs and , for all and \WhileTRUE Residents send usage request (with basic and quality usage) to MGCC via the information network   MGCC solves LPs (22) and (23)   MGCC selects the optimal solution comparing the solutions to (22) and (23)   MGCC updates the virtual queues and according to (11) and (12), for all and Adaptive Electricity Scheduling Algorithm

3.3 Performance Analysis

The proposed scheduling algorithm dynamically balances cost minimization and QoSE provisioning. It only requires current system state information (i.e., as an online algorithm) and requires no statistic information about the random supply, demand, and price processes. The algorithm is also robust to non-i.i.d. and non-ergodic behaviors of the processes [11].

Theorem 2.

The constraint on the ESS battery level , , is always satisfied for all and .


From the battery virtual queue definition (10), the constraint is equivalent to

We assume all the batteries satisfy the battery capacity constraint at the initial time , i.e., , for all . Supposing the inequalities hold true for time , we then show the inequalities still hold true for time .

First, we show . If , then with from Lemma 2, we have . If , then the largest value is . For any , we have

It follows that .

Next, we show . Assuming , then from Lemma 2, we have . It follows that

If , following (10), we have

Therefore, we have . Thus the inequalities also hold true for time .

It follows that is satisfied under the optimal scheduling algorithm for all , . ∎

Theorem 3.

The worst-case backlogs of the QoSE virtual queue for each resident is bounded by , for all , . Moreover, the worst-case average amount of outage of quality usage for resident in a period is upper bounded by .


(i) We first prove the upper bound . Initially, we have . Assume that in time slot the backlog of the QoSE virtual queue of resident satisfies . We then check the backlog at time and show the bound still holds true.

If , following Lemma 3, the optimal scheduling for the quality usage of resident satisfies . From the virtual queue dynamics (12), we have

If , we have ; otherwise, it follows that .

If , we have . If , we have ; otherwise, we have .

Thus we have . The proof of the QoSE virtual queue backlog bound is completed.

(ii) Consider an interval with length of . Summing (12) from to , we have . It follows that

Theorem 4.

The average MG operation cost under the adaptive electricity scheduling algorithm in Algorithm3.2, , is bounded as , where is optimal operating cost and .


From Theorem 2, the battery capacity constraints is met in each time slot with the adaptive control policy. Take expectation on (2) and sum it over the period :

Since , we divide both sides by and let go to infinity, to obtain


Consider the the following relaxed version of problem (9).

minimize: (25)

Since the constraints in problem (25) are relaxed from that in problem (9), the optimal solution to problem (9) is also feasible for problem (25). The solution of (25) does not depend on battery energy levels. Let the optimal solution for problem (25) be and the corresponding object value is . According to the properties of optimality of stationary and randomized policies [12], the optimal solution satisfies and .

We substitute solution into the right-hand-side of the drift-and-penalty (19). Since our proposed policy minimizes the right-hand-side of (19), we have

The second inequality is due to , , , , and . Taking expectation and sum up from to , we obtain