ClosedForm Analysis of NonLinear AgeofInformation in Status Updates with an Energy Harvesting Transmitter
Abstract
Timely status updates are crucial to enabling applications in massive Internet of Things (IoT). This paper measures the datafreshness performance of a status update system with an energy harvesting transmitter, considering the randomness in information generation, transmission and energy harvesting. The performance is evaluated by a nonlinear function of age of information (AoI) that is defined as the time elapsed since the generation of the most uptodate status information at the receiver. The system is formulated as two queues with status packet generation and energy arrivals both assumed to be Poisson processes. With negligible service time, both FirstComeFirstServed (FCFS) and LastComeFirstServed (LCFS) disciplines for arbitrary buffer and battery capacities are considered, and a method for calculating the average penalty with nonlinear penalty functions is proposed. The average AoI, the average penalty under exponential penalty function, and AoI’s threshold violation probability are obtained in closed form. When the service time is assumed to follow exponential distribution, matrix geometric method is used to obtain the average peak AoI. The results illustrate that under the FCFS discipline, the status update frequency needs to be carefully chosen according to the service rate and energy arrival rate in order to minimize the average penalty.
I Introduction
Satisfying strict realtime requirements in wireless communication systems is of extensive concerns. A typical realtime application in Internet of Things (IoT) is remote monitoring and control, which requires timely status update to the fusion center, i.e., status information of the objects should be refreshed at the fusion center in a timely manner. However, due to the inevitable delays in queuing and transmission, the received status packets do not carry the present status information. To characterize the lag in status update, age of information (AoI) has been proposed in [1] as a metric for information freshness. It is defined as the time that has elapsed since the generation of the most uptodate status information at the receiver, as is illustrated in Fig. 1. The AoI at time t is expressed as
\Delta(t)=tU(t), 
where U(t) represents the time stamp at the generation epoch of the most uptodate status information that has been received before time t. In [1], the status update process is formulated as a queuing system, in which the traffic arrivals correspond to the generations of status packets, and the service times correspond to transmissions and the time waiting for medium access. The timeaveraged AoI of M/M/1, M/D/1 and D/M/1 queues under the firstcomefirstserved (FCFS) discipline is obtained, and the comparison among the three queues indicates that a more regular update brings a smaller average AoI.
Meanwhile, numerous nodes in IoT will be powered by renewable sources using energy harvesting techniques [2]. For example, to sustainably update the environmental conditions (temperature, air humidity and etc.), motion (position and velocity) or other characters to a fusion center, energy harvesting transmitters can free the monitored object from limited power backup or cables. Wireless nodes powered by renewable energy are faced with the randomness in energy supplies, which makes guaranteeing realtime communications more challenging compared with gridpowered or solely batterypowered systems, and further complicates the problem of keeping status update in a regular fashion. Therefore, the performance analysis of a status update system with an energy harvesting transmitter is essential in addition to the analysis of a system powered by a stable energy source.
The average AoI is considered a key indicator to the performance of a status update system. However, it cannot straightly describe the performance degradation caused by the lag in status update. A thorough survey on the performance degradation caused by information staleness can be found in [3]. The performance degradation of a system, such as the inaccuracy in monitoring and the invalidity of control decisions, varies based on the scenario and is generally nonlinear on AoI. Ref. [4] shows that the characteristics of the average AoI and the violation probability of peak AoI, which is defined as the AoI right before the reception of a new status information, can be noticeably different. Therefore, the actual performance of a status update system, which is nonlinear on AoI, should be further investigated. The nonlinear AoI penalty function is introduced in [5] to evaluate information staleness. Ref. [6] extends the work on M/M/1 in [1], and obtains the average penalty under exponential and logarithmic AoIpenalty functions. The case where AoI is expected to be within a certain threshold is considered in [7], and scheduling algorithms to allocate channel access among multiple users are proposed in order to reduce violation probability. Other work mainly focuses on the analysis on average AoI or average peak AoI[8]–[18]. Two special cases of multiple servers, M/M/\infty and M/M/2 systems, are investigated in [8]. Timeaveraged peak AoI is considered for multiclass M/G/1 and M/G/1/1 queuing systems in [9] and M/M/1 queues with delivery errors in [10]. Ref. [11] studies the average AoI of a G/G/1/1 status update system. Ref. [12] studies the packet management policies for a status update system and shows that preemptive LCFS discipline is ageoptimal, throughputoptimal and delayoptimal, given that the service times are independent and identically distributed (i.i.d.) exponential random variables. Ref. [13] compares the performance of M/M/1, M/M/1/1 and M/M/1/2*, in which the backlogged packet is replaced if a new packet arrives. The stationary distribution of AoI in GI/GI/1 system is derived in [14]. Recently, Ref. [15] introduces a method named stochastic hybrid systems (SHS) to the analysis of status updating systems. For M/M/1 queues with multiple sources, the performance under FCFS, preemptive and nonpreemptive lastcomefirstserved (LCFS) policy is investigated with SHS method. Ref.[16]–[18] investigate userscheduling policies to minimize overall average AoI in multiuser scenarios.
Scheduling schemes to reduce latency in an energy harvesting communication system can be found in [19]–[22]. Efforts on the AoI in energy harvesting powered status update systems began to emerge recently. Existing work mainly focuses on the scheduling of status packet transmissions subject to energy constraints [23]–[28], where the generation of status packets can be fully controlled. Ref. [23] considers scheduling policies for the minimization of average AoI under energy replenishment constraints in the systems where the queuing delay and service time are neglected. Ref. [24] proposes and compares three intuitive status update policies, which try to equalize update interval, to equalize update delay, and to reduce packet queuing, respectively. Ref. [25] analyzes energy harvesting systems with different battery capacities, and discusses their AoIoptimal transmission scheduling policies by which the average AoI is minimized. A twohop status update system with energy harvesting transmitter and relay is investigated in [26]. Ref. [27] analyzes the batterythreshold policy in energy harvesting system, and finds the condition for one to minimize the average AoI among all threshold policies. The optimality of energy dependent AoIthreshold policies in a system with finite battery and zero service time is proved in [28]. Ref. [29] and [30] explore M/M/1/1 energy harvesting status update systems, where there are at most one status packet buffered in the system. They investigate the average AoI in the system with SHS method, and analyze the asymptotic cases where status packet arrival rate, energy arrival rate, or service rate is relatively large, respectively.
In this paper, we jointly consider the randomness in status packet generation and energy harvesting in the analysis of nonlinear AoIbased performance for an energy harvesting wireless communication system. The randomness in transmission and MAC delay is also considered to investigate the overall impact of status packet generation, energy arrivals and services on the AoI of a status update system. The status update problem is characterized as a queuing system with a finite data queue (buffer), where the status packets are stored after their generations, and a finite energy queue (battery). Both the status packet arrivals and energy packet arrivals are assumed to be Poisson processes. The problem considered is similar to the one in [29] and [30], and they focus solely on average AoI in a system with unit buffer size, and investigate the average AoI by SHS. Different from [29] and [30], this paper applies the conventional stochastic analysis with queuing model, and analyzes the nonlinear AoIbased performance arbitrary buffer and battery capacity. The contribution of this paper is summarized as follows:

A method to obtain the closedform AoIbased penalty is established for both FCFS and LCFS disciplines when the service time is negligible, since the service time is usually much smaller than packet generation intervals and energy arrival intervals in realworld applications. Explicitly, the closedform expressions of the cumulative probability distributions (CDF) of the peak AoI and the sojourn time, and the rate of valid updates^{1}^{1}1A valid update is defined as a status packet that is the most uptodate status packet upon reception in Sec. II. are obtained. The average nonlinear penalty of the system can be derived by this method for integrable AoIpenalty functions.

Average penalties under three typical AoIpenalty functions are obtained and analyzed. They corresponds to the average AoI, average exponential penalty of AoI and AoI’s threshold violation probability in the system. The results are further compared under different buffer capacities, battery capacities and service disciplines. Results shows that the exponential penalty is extremely sensitive to the ratio between status generation frequency and energy arrival rate, especially under the FCFS discipline when the buffer capacity is large.

To consider a more general case and incorporate nonnegligible service time, the service time is assumed to be independent and identically distributed (i.i.d.) random variables following exponential distribution, and the status update is formulated as a quasibirthanddeath (QBD) process with finite battery capacity. The explicit expression for the average peak AoI is obtained after the stationary distribution of system states is computed by the matrix geometric method [33].
The remainder of the paper is organized as follows. Section II explains the basic notations and the problem formulation. Results in the negligibleservicetime regime are described in Section III. Section IV formulates the problem as a QBD process and describes how to compute the average peak AoI. Section V illustrates the results with figures. Section VI concludes the paper.
II System Model
As depicted in Fig. 2, a status update system powered by renewable energy is modeled as a queuing system with a data buffer and a battery. The arrival of a status packet corresponds to the generation of a status information from the source, while each departure represents the successful reception of the status information at the receiver side.
IIA Status Update Model
To capture the randomness in the generation of status packets, the arrival of status packets is assumed to be a Poisson process with rate \lambda. After its generation, a status packet first arrives at the data buffer, which can store at most K status packets. Thus, if there are already K status packets in the data buffer upon the arrival of a new status packet, one status packet must be dropped.
The server is assumed to be workconserving, i.e., the server is idle only when at least one of the two queues is empty. Two kinds of service disciplines are considered. The first one is FCFS, with which a newlyarrived status packet waits at the end of the data queue, and will not be served until all the status packets ahead of it depart. Under the FCFS discipline, older status packets have higher priority, and the newlyarrived status packet will be blocked out of the data buffer if there are already K status packets in the data buffer. The other service discipline is LCFS, which allows the latest status packet to wait at the front of the queue and the oldest packet in the buffer to be discarded if the buffer is full.
It is possible that a newlydelivered status packet is older than the status packets at the receiver side. However, the reception of old information does not change the AoI. To clarify this, a valid update is defined as a status packet that is delivered and is the most uptodate packet upon its reception. Thus, the AoI is reset to the age of the received packet only when it is a valid update. In [14], the authors define informative packets and noninformative packets to distinguish the status packets that reset the AoI from those that do not. The informative packets in [14] corresponds to the valid updates. Under FCFS discipline, any status packet that enters the data buffer is a valid update, while under LCFS discipline, a status packet is a valid update if and only if there is no status packet arrival between its arrival and departure.
IIB Energy Model
The energy model in this paper is similar to those in [24]–[30]. The transmitter is powered by an energy harvesting module, which consistently harvests energy and stores it in a battery with limited capacity. Assume that the service of a status packet requires E_{0} Joules of energy, which is also referred to as an energy packet. Discretizing the battery by E_{0} Joules of energy, the capacity of the battery is denoted as B energy packets. Therefore, the arrival of energy packets represents the accumulation of integral multiples of E_{0} Joules in the battery. The arrival of energy packets is modeled as a Poisson process with rate r, which characterizes the randomness and unpredictability in energy harvesting. The energy arrival rate r is assumed to be greater than the status packet arrival rate \lambda to ensure the stability of the data queue. A new energy packet is immediately discarded if the battery is full.
IIC Objective Functions
IIC1 Linear functions
The performance of a status update system is commonly evaluated by the longterm averaged AoI or the longterm averaged peak AoI. Denote the interarrival time between the (i1)th and the ith valid update by X_{i}, the interdeparture time between the ith and the (i+1)th valid update by D_{i} and the ith valid update’s sojourn time by T_{i}, as is depicted in Fig. 1. The ith peak AoI A_{i}, which is the AoI right before the reception of the ith valid update [13], can be written as
A_{i}=X_{i}+T_{i}=D_{i1}+T_{i1}.  (1) 
Eq. (1) can be proved straightforward by the evolution curve of AoI in Fig. 1. Taking expectation, letting i\to\infty at both sides of Eq. (1) and denoting the limits of the variables as the ones without subscripts, the average peak AoI follows
\displaystyle\mathbb{E}\left[A\right]=\mathbb{E}\left[X\right]+\mathbb{E}\left% [T\right]=\mathbb{E}\left[D\right]+\mathbb{E}\left[T\right].  (2) 
According to [1], the average AoI of a stationary and ergodic system is defined as
\bar{\Delta}=\lim_{\mathcal{T}\to\infty}\frac{1}{\mathcal{T}}\int_{0}^{% \mathcal{T}}\Delta(t)\,\mathrm{d}t, 
in which the integral equals the area below the AoI curve. Denote the area below the AoI curve between the delivery of the (i1)th and the ith valid update as Q_{i}, as is shadowed in Fig. 1. The average AoI is expressed as
\bar{\Delta}=\lim_{\mathcal{T}\to\infty}\frac{1}{\mathcal{T}}\sum_{i=1}^{N(% \mathcal{T})}Q_{i}, 
where N(\mathcal{T}) represents the number of valid updates delivered before t=\mathcal{T}. Defining the arrival rate of valid updates as \tilde{\lambda}=\lim_{\mathcal{T}\to\infty}\frac{N(\mathcal{T})}{\mathcal{T}}, the average AoI becomes \bar{\Delta}=\tilde{\lambda}\lim_{\mathcal{T}\to\infty}\mathbb{E}\left[Q_{i}% \right]. Since Q_{i}=\left(A_{i}^{2}T_{i1}^{2}\right)/2, the average AoI is given by
\displaystyle\bar{\Delta}  \displaystyle=  \displaystyle\frac{\tilde{\lambda}}{2}\left(\mathbb{E}\left[A^{2}\right]% \mathbb{E}\left[T^{2}\right]\right).  (3) 
IIC2 Nonlinear functions
Considering the nonlinear performance degradation caused by outdated data, AoI is further generalized as an AoIrelated penalty function, which quantizes the performance based on AoI. A general way to characterize the nonlinear penalty is by defining a nonlinear function g(\Delta) that maps AoI to the penalty as introduced in [5], so that the average penalty of a stationary and ergodic system is
\displaystyle C  \displaystyle=  \displaystyle\lim_{\mathcal{T}\to\infty}\frac{1}{\mathcal{T}}\int_{0}^{% \mathcal{T}}g\left(\Delta\left(t\right)\right)\,\mathrm{d}t.  (4) 
In [14, Example 1], the authors provide how to obtain the average penalty with the CDF of AoI given, without further analysis on the average penalty. It is also proved that the CDF of AoI can be obtain with the CDFs of the peak AoI A and the sojourn time T, and the arrival rate \tilde{\lambda} of valid updates. Next, we are going to provide another method to show how to derive the average penalty directly with the CDFs of the peak AoI A and the sojourn time T, and the arrival rate \tilde{\lambda} of valid updates.
The righthand side of Eq. (4) is rearranged as the summation of integrals over the intervals between sequential deliveries of valid updates:
\displaystyle\lim_{\mathcal{T}\to\infty}\frac{1}{\mathcal{T}}\int_{0}^{% \mathcal{T}}g\left(\Delta\left(t\right)\right)\,\mathrm{d}t  \displaystyle=  \displaystyle\lim_{\mathcal{T}\to\infty}\frac{N(\mathcal{T})}{\mathcal{T}}\lim% _{\mathcal{T}\to\infty}\frac{1}{N(\mathcal{T})}\sum_{i=0}^{N(\mathcal{T})1}% \int_{\sum_{j=0}^{i1}D_{j}}^{\sum_{j=0}^{i}D_{j}}g\left(\Delta\left(t\right)% \right)\,\mathrm{d}t. 
According to the definition, AoI is set to sojourn time T_{i} upon the ith valid update’s delivery, and grows linearly with unit slope before the delivery of the (i+1)th valid update. Therefore,
\displaystyle\lim_{\mathcal{T}\to\infty}\frac{1}{\mathcal{T}}\int_{0}^{% \mathcal{T}}g\left(\Delta\left(t\right)\right)\,\mathrm{d}t  \displaystyle=  \displaystyle\tilde{\lambda}\lim_{\mathcal{T}\to\infty}\frac{1}{N(\mathcal{T})% }\sum_{i=0}^{N(\mathcal{T})1}\int_{T_{i}}^{A_{i}}g\left(\Delta\right)\,% \mathrm{d}\Delta  
\displaystyle=  \displaystyle\tilde{\lambda}\left(\mathbb{E}\left[G(A)\right]\mathbb{E}\left[% G(T)\right]\right), 
in which G(x)=\int_{0}^{x}g(\Delta)\,\mathrm{d}\Delta.
Thus, given the CDFs of the peak AoI A and the sojourn time T, and the arrival rate \tilde{\lambda} of valid updates, the average penalty of the system can be determined by
C=\tilde{\lambda}\int_{0}^{\infty}G(a)\,\mathrm{d}\mathrm{P}\left\{A\leq a% \right\}\tilde{\lambda}\int_{0}^{\infty}G(t)\,\mathrm{d}\mathrm{P}\left\{T% \leq t\right\}.  (5) 
The average penalty indicates the longterm average performance of a status update system. Our objective is to investigate how system parameters, such as status packet arrival rate, buffer capacity, battery capacity and service disciplines, affect the average penalty.
III Negligible Service Time Regime
To gain more insights, in this section, we first explore the asymptotic results where the service time is negligible compared to the status packet arrival intervals and the energy packet arrival intervals. The scenario where the average service time is much shorter than status packet arrival intervals and energy arrival intervals commonly exists in practical IoT applications. The service time mainly incorporates transmission and the MAC (Medium Access Control) delay. Since status update packets are usually small in data size, the time for transmission (even if retransmission is considered) or MAC delay is relatively short. For example, a subframe in FDDLTE is 1ms, so the service time of a short packet can be several milliseconds. On the contrary, the need for status update (e.g., temperature) and energy harvesting are mostly in a larger time scale than milliseconds. Therefore, the asymptotic regime where the service time is negligible is reasonable to consider and significant in offering insight on the performance of status update, especially on the impact of energy provision in such a system.
Denote the data queue length by q_{1} and the energy queue length by q_{2}. The state transition in the negligibleservicetime regime is illustrated in the left diagram in Fig. 3. If data queue length q_{1}<K, each state (q_{1},q_{2}) transits to state (q_{1}+1,q_{2}) with rate \lambda; If energy queue length q_{2}<B, each state (q_{1},q_{2}) transits to state (q_{1},q_{2}+1) with rate r. Note that when the service time is negligible, the service for a status packet can be completed instantly when the energy queue is not empty. Therefore, every state (q_{1},q_{2}) with both q_{1}>0 and q_{2}>0 transits to state (q_{1}1,q_{2}1) with rate \infty. In other words, any state (q_{1},q_{2}) with q_{1}q_{2}\neq 0 is a transient state that will never be visited thereafter. In the right diagram of Fig. 3, an equivalent state transition is drawn with all the transient states shadowed in gray. As shown in the diagram, the state transition among the recurrent states, i.e., q_{1}q_{2}=0, is identical to an M/M/1 queue. Based on this, the system states are indexed as S=q_{1}q_{2}\in\left\{B,\left(B1\right),\cdots,1,0,1,\cdots,K\right\}, which distinctively maps a recurrent state to S:

If S<0, data queue length q_{1}=0 and energy queue length q_{2}=S;

If S=0, both queues are empty;

If S>0, data queue length q_{1}=S and energy queue length q_{2}=0.
For each state S\in\left\{(B1),\cdots,1,0,1\cdots,K\right\}, the system transits to state S1 when there is an energy arrival; for S\in\left\{B,\cdots,1,0,1,\cdots,K1\right\}, system state becomes S+1 when there is a status packet arrival. Thus, as is shown in the bottom diagram of Fig. 3, the system is equivalent to an M/M/1 queue with status packet arrival rate \lambda, service rate r and buffer size K+B. Let \theta denote the ratio of the status packet arrival rate to the energy packet arrival rate, i.e., \theta=\frac{\lambda}{r}, and the utilization of the M/M/1 queue is \theta. Next, we first derive the stationary probability distribution of the peak AoI A and the sojourn time T, and the arrival rate \tilde{\lambda} of valid updates, then obtain the average penalty under three types of penalty functions (depicted in Fig. 4) by Eq. (5). The three penalty functions are:

Linear function g(\Delta)=\Delta: The average penalty equals the longterm average AoI. This penalty function is suitable for the systems in which the influence of the delay in information grows approximately linearly with time.

Exponential function g(\Delta)=\alpha^{1}\left(e^{\alpha\Delta}1\right),\alpha\neq 0: We let g(0)=0 and g^{\prime}(0)=1, so that the penalty and the slope when \Delta=0 are identical to the ones in the linear function, for the simplicity of comparison. When exponent \alpha>0, the exponential penalty function is more sensitive to large AoI compared to the linear function, and is more reasonable in the scenarios where the timecorrelation among status is relatively small. Taking the finitestate Markov channel[31], which is a widely adopted channel model, as an example, the probability of correctly estimating the current channel state based on delayed channel state exponentially decays to the limiting distribution. Additionally, note that the exponential penalty function is increasing for both \alpha>0 and \alpha<0. Exponent \alpha being negative can be useful in systems such as [32].

Shifted unit step function g(\Delta)=\mathbf{1}\left\{\Delta\beta\right\}: Unit step function \mathbf{1}\left\{\cdot\right\} is defined as
\displaystyle\mathbf{1}\left\{x\right\}=\left\{\begin{aligned} &\displaystyle 1% ,\mathrm{~{}if~{}}x>0,\\ &\displaystyle 0,\mathrm{~{}if~{}}x\leq 0.\end{aligned}\right. Under the shifted unit step function, the longterm average penalty gives the probability on AoI exceeding threshold \beta. This function should be considered when there is a certain upperbound constraint of AoI that the system tries not to violate.
In the following two subsections, the formulas for the average penalty under the three types of penalty functions will be listed with preliminary intuition towards the expressions. The results will be thoroughly analyzed together with figures in Section V.
IIIA FCFS Discipline
For FCFS descipline, we have the following results.
Lemma 1
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, the arrival rate \tilde{\lambda} of valid updates is
\tilde{\lambda}_{\mathrm{FCFS}}=\lambda\frac{1\theta^{K+B}}{1\theta^{K+B+1}}. 
Proof: See Appendix A.
The proof for Lemma 1 is straightforward: Since all the status packets admitted to the data buffer are valid updates under FCFS, the arrival rate of valid update equals the arrival rate of status packets that enter the buffer. Therefore, the arrival rate \tilde{\lambda} of valid update under the FCFS discipline equals the throughput.
Lemma 2
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, the CDFs of the peak AoI and the sojourn time of valid updates are
\displaystyle\mathrm{P}\left\{A\leq a\right\}  \displaystyle=  \displaystyle 1\frac{e^{\lambda a}\theta^{B}}{\theta^{B}\theta^{K}}\frac% {\theta^{1}e^{ra}}{\theta^{B}\theta^{K}}\sum_{n=1}^{K}\frac{\left(\lambda a% \right)^{n}}{n!}+\frac{\theta^{K}e^{ra}}{\theta^{B}\theta^{K}}\sum_{n=0}^{K% }\frac{\left(ra\right)^{n}}{n!},  (6) 
and
\displaystyle\mathrm{P}\left\{T\leq t\right\}  \displaystyle=  \displaystyle 1\frac{1}{\theta^{B}\theta^{K}}\sum_{n=0}^{K}\frac{\left(% \lambda t\right)^{n}}{n!}e^{rt}+\frac{\theta^{K}}{\theta^{B}\theta^{K}}\sum% _{n=0}^{K}\frac{\left(rt\right)^{n}}{n!}e^{rt},  (7) 
respectively.
Proof: See Appendix B.
According to Eq. (5), the average penalty under FCFS discipline can be obtained by Lemma 1 and 2. For the three special cases of penalty function being g(\Delta)=\Delta, g(\Delta)=\alpha^{1}\left(e^{\alpha\Delta}1\right), and g(\Delta)=\mathbf{1}\left\{\Delta\beta\right\}, we obtain the longterm average penalty by Eq. (5) and summarize the results in Theorem 1–3.
Theorem 1
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, the average AoI is
C_{\mathrm{FCFS,~{}linear}}=\lambda^{1}+r^{1}\frac{\theta}{\theta^{B}% \theta^{K+1}}\left[K\theta^{K}+\frac{1+\theta^{K1}3\theta^{K}+\theta^{K+1}}% {1\theta}\right].  (8) 
Consider an averagepowerconstrained scenario, where the average energy usage rate is limited by the energy arrival rate r. Since \theta is less than 1, each status packet can be served as soon as it arrives at the system, thus the average AoI is \lambda^{1}. It is exactly the first term at the righthand sides of Eq. (8). As battery capacity B increases, the average AoI decreases and approaches the average AoI under the averagepowerconstrained scenario. This result offers guidance to the selection of the battery capacity in an energy harvesting status update system. For example, given the system requirement that the average AoI should not exceed \Delta_{\mathrm{max}}, the battery capacity must be greater than
B_{\mathrm{min}}=\log_{\theta}\frac{\lambda\Delta_{\mathrm{max}}1}{\lambda% \theta^{K}\Delta_{\mathrm{max}}+\left(K\theta^{K}\theta^{K}+\frac{1\theta^{% K}}{1\theta}\right)\theta^{2}}. 
Similar characteristics can be found in Theorems 2–6 for the other penalty functions as well as under the LFCS discipline.
Additionally, the average AoI under the FCFS discipline is derived in [29][30] by SHS for a system where only one status packet is allowed in the system if the battery is not empty (identical to the case K=0 if service rate \mu\to\infty). The average AoI (see [29, Eq. (7)] and [30, Eq. (9)]) matches Eq. (8) under the conditions K=0 and \mu\to\infty, which further verifies Theorem 1.
Theorem 2
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, with penalty function g\left(\Delta\right)=\alpha^{1}\left(e^{\alpha\Delta}1\right),\alpha<\lambda, the average penalty is
\displaystyle C_{\mathrm{FCFS,~{}exp}}=\left\{\begin{aligned} &\displaystyle% \frac{1}{\lambda\alpha}+\frac{r\alpha^{1}}{\theta^{B}\theta^{K+1}}\left[% \frac{\theta^{K+2}}{\lambda\alpha}+\left(1\theta\right)\frac{1\frac{\lambda% ^{K+1}}{\left(r\alpha\right)^{K+1}}}{r\alpha\lambda}\frac{1}{r\alpha}% \right],\mathrm{if}~{}\alpha\neq r\lambda,\\ &\displaystyle\frac{1}{\lambda\alpha}+\frac{r^{1}}{\theta^{B}\theta^{K+1}}% \left[\frac{\theta^{K+2}2\theta+1}{(2\theta1)(1\theta)}+\frac{K}{\theta}% \right],\mathrm{if}~{}\alpha=r\lambda.\end{aligned}\right.  (9) 
Note that Eq. (9) holds for \alpha\in[0,\lambda) as well as \alpha\in(\infty,0). The exponent \alpha needs to be strictly smaller than status packet arrival rate \lambda, or there is an unbounded average penalty. In this sense, the average penalty under an exponential penalty function with exponent \alpha>0 is highly sensitive to the status packet arrival rate \lambda.
Theorem 3
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, with penalty function g\left(\Delta\right)=\mathbf{1}\left\{\Delta\geq\beta\right\}, the average penalty is
\displaystyle C_{\mathrm{FCFS,~{}step}}=e^{\lambda\beta}+\frac{e^{r\beta}}{% \theta^{B}\theta^{K+1}}\sum_{i=0}^{K}\frac{(\lambda\beta)^{i}}{i!}\frac{% \theta^{K+1}e^{r\beta}}{\theta^{B}\theta^{K+1}}\sum_{i=0}^{K}\frac{(r\beta)% ^{i}}{i!}+\frac{e^{\lambda\beta}\theta^{K+1}e^{r\beta}}{\theta^{B}\theta^% {K+1}}.  (10) 
The average penalty with penalty function g\left(\Delta\right)=\mathbf{1}\left\{\Delta\geq\beta\right\} equals the probability that the AoI exceeding threshold \beta. In [13], the peak AoI is proposed as a metric that is suitable in the applications where the AoI is supposed to be lower than a given bound. However, comparing the CDF of peak AoI in Eq. (6) and AoI’s threshold violation probability in Eq. (10), it is observed that with the same threshold \beta, we have
\mathrm{P}\left\{A>\beta\right\}\geq C_{\mathrm{FCFS,~{}step}}, 
which implies that the peak AoI violation probability does not equal the AoI violation probability. Therefore, the peak AoI violation probability can only serve as an upper bound of the AoI violation probability.
IIIB LCFS Discipline
Under the LCFS discipline, the average penalty is expected to be smaller than its FCFS counterpart, since the latest status packet is delivered first. In a queuing system with the LCFS discipline, the probability distribution of sojourn time and peak AoI are obtained and summarized in the following lemmas:
Lemma 3
Under the LCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda and r, and the service time being negligible, the arrival rate \tilde{\lambda} of valid updates is
\tilde{\lambda}_{\mathrm{LCFS}}=\frac{1\frac{\theta^{B+1}}{1+\theta}\frac{% \theta^{B+K+1}}{1+\theta}}{1\theta^{B+K+1}}.  (11) 
Proof: See Appendix C.
Under the LCFS discipline when the buffer capacity K>1, outdated status packets is going to be delivered to the receiver unless they are pushed out of the buffer by fresher status packets. Therefore, the arrival rate of valid updates is smaller than the throughput of the system. The valid updates under the LCFS discipline consists of all status packets that arrive at the buffer when the battery is not empty, and those delivered before any other status packets arrive. It is observed from Eq. (11) that as the buffer capacity K increases, there is a decrease in valid updates. The reason is that outdated status packets in the buffer is going to consume energy packets, which reduces the probability of a newly arrived status packet being a valid update.
Lemma 4
Proof: See Appendix D.
\displaystyle\mathrm{Prob}\left[T\leq t\right]=1e^{\left(\lambda+r\right)t}% \frac{\left(1\theta^{K+1}\right)}{\left(\theta^{B}1\right)\left(1+\theta% \right)+\left(1\theta^{K+1}\right)}  (12) 
\displaystyle\mathrm{Prob}\left[A\leq a\right]  (13)  
\displaystyle=  \displaystyle 1e^{\lambda a}\frac{\left(\theta^{B}\theta^{K+1}\right)\left% (1+\theta\right)}{\left(\theta^{B}1\right)\left(1+\theta\right)+\left(1% \theta^{K+1}\right)}e^{ra}(1+\theta)\frac{K\theta^{1}+\frac{\theta^{K+1}2+% \theta^{1}}{1\theta}}{\left(\theta^{B}1\right)\left(1+\theta\right)+\left(% 1\theta^{K+1}\right)}  
\displaystyle+\frac{e^{\left(\lambda+r\right)a}(1+\theta)}{\left(\theta^{B}% 1\right)\left(1+\theta\right)+\left(1\theta^{K+1}\right)}\left\{\frac{1+% \theta^{K+1}}{1+\theta}+(K\theta^{1}+\theta^{1}\frac{\theta}{1\theta})\sum% _{k=0}^{K}\frac{\left(r\theta a\right)^{k}}{k!}\right.  
\displaystyle\left.\theta^{1}\sum_{k=0}^{K1}\frac{\left(r\theta a\right)^{k% +1}}{k!}+\frac{\theta^{K+2}}{1\theta}\sum_{k=0}^{K}\frac{\left(ra\right)^{k}}% {k!}\right\} 
For the penalty function being g(\Delta)=\Delta, g(\Delta)=\alpha^{1}\left(e^{\alpha\Delta}1\right), and g(\Delta)=\mathbf{1}\left\{\Delta\beta\right\}, the longterm average penalty under LFCS is obtained by Eq. (5) and given in Theorem 4–6.
Theorem 4
Under the LCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, the average AoI is
C_{\mathrm{LCFS,~{}linear}}=\lambda^{1}+\frac{r^{1}}{\theta^{B}\theta^{K+1% }}\left[\frac{(1\theta)\theta^{K+1}}{(1+\theta)^{K+1}}\theta^{K+1}+\theta% \right].  (14) 
Theorem 5
Under the LCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, with penalty function g\left(\Delta\right)=\alpha^{1}\left(e^{\alpha\Delta}1\right),\alpha<\lambda, the average penalty is
\displaystyle C_{\mathrm{LCFS,~{}exp}}=\frac{1}{\lambda\alpha}+\frac{1}{(r% \alpha)^{2}}\frac{\lambda}{\left(\theta^{B}\theta^{K+1}\right)}\left[1+\left% (\frac{\lambda}{\lambda+r\alpha}\right)^{K+1}\frac{r\lambda}{\lambda\alpha}% \theta^{K+1}\frac{r\alpha}{\lambda\alpha}\right].  (15) 
Theorem 6
Under the LCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being negligible, with penalty function g\left(\Delta\right)=\mathbf{1}\left\{\Delta\geq\beta\right\}, the average penalty is
\displaystyle C_{\mathrm{LCFS,~{}step}}  \displaystyle=  \displaystyle e^{\lambda\beta}+\frac{e^{(r+\lambda)\beta}}{\theta^{B}% \theta^{K+1}}\left\{e^{\lambda\beta}\frac{\theta^{K+2}}{1\theta}+\left(K+% \frac{12\theta}{1\theta}\right)\left(e^{\lambda\beta}\sum_{i=0}^{K}\frac{(% \lambda\beta)^{i}}{i!}\right)\right.  (16)  
\displaystyle\left.+\lambda\beta\sum_{i=0}^{K1}\frac{(\lambda\beta)^{i}}{i!}% \frac{\theta^{K+2}}{1\theta}\sum_{i=0}^{K}\frac{(r\beta)^{i}}{i!}\right\}. 
IIIC Asymptotic Regime: K\to\infty
When the buffer size is large enough, all the status packets is delivered to the receiver, and thus the throughput is \lambda. Next, we look into the average penalty when K\to\infty.
Corollary 1
The average penalty of an energy harvesting status update system, with Poisson status packet arrivals and energy packet arrivals of rate \lambda and r=\lambda/\theta, and an infinitesized buffer, is summarized in TABLE I.
Penalty functions  FCFS  LCFS 

g(\Delta)=\Delta  \frac{1}{\lambda}+\frac{1}{\lambda}\frac{\theta^{B+2}}{1\theta}  \frac{1}{\lambda}+\frac{1}{\lambda}\theta^{B+2} 
g(\Delta)=\alpha^{1}\left(e^{\alpha\Delta}1\right)  \frac{1}{\lambda\alpha}+r\alpha^{1}\left(\frac{1\theta}{r\alpha\lambda}% \frac{1}{r\alpha}\right)\theta^{B},~{}\alpha<r\lambda  \frac{1}{\lambda\alpha}+\frac{\lambda}{(r\alpha)^{2}}\theta^{B} 
g\left(\Delta\right)=\mathbf{1}\left\{\Delta\geq\beta\right\}  e^{\lambda\beta}+\left(e^{\lambda\beta}1\right)e^{r\beta}\theta^{B}  e^{\lambda\beta}+\lambda\beta e^{r\beta}\theta^{B} 
As mentioned before, the difference in the average penalty between the energy harvesting case and the averagepowerconstrained case gradually diminishes with the growth of battery capacity B. When the K\to\infty, it is observed in TABLE I that the average penalty decays exponentially to its limit with constant rate \theta as the battery capacity B increases, regardless of the service discipline or the penalty function.
IV NonNegligible Service Time Regime
In [29], the authors analyze the average AoI of the model when the buffer capacity is 1, and obtain the average AoI under the asymptotic region where the ratio between status packet arrival rate \lambda and service rate \mu goes to infinity. However, it is difficult to obtain the result when the service time is not negligible.
In this section, the service time of a status packet is assumed to be an i.i.d exponential random variable with mean \mu^{1}. We show that the queues evolve as a QBD process, and compute the average peak AoI by matrix geometric method[33]. Analysis of the results in this section will be discussed at the beginning of Section V with illustrations.
The states of the system are represented by tuple \left(q_{1},q_{2}\right). Note that when q_{1}>0 and q_{2}>0, the server is serving a status packet with an energy packet until the status packet leaves the system, such that the state transits to \left(q_{1}1,q_{2}1\right). At each state \left(q_{1},q_{2}\right), there are several possible transitions, summarized as follows:

From state \left(q_{1},q_{2}\right) to state \left(q_{1}+1,q_{2}\right) with transition rate \lambda;

From state \left(q_{1},q_{2}\right) to state \left(q_{1},q_{2}+1\right) with transition rate r;

From state \left(q_{1},q_{2}\right) to state \left(q_{1}1,q_{2}1\right) with transition rate \mu if \min\{q_{1},q_{2}\}\geq 1.
The corresponding Markov transition graph is illustrated in Fig. 5. As the graph indicates, the twodimensional Markov chain is a QBD process, of which the transition matrix is written as:
\displaystyle\mathbf{Q}=\left(\begin{array}[]{ccccc}\mathbf{\tilde{V}}&\mathbf% {W}&\mathbf{0}&\mathbf{0}&\cdots\\ \mathbf{U}&\mathbf{V}&\mathbf{W}&\mathbf{0}&\cdots\\ \mathbf{0}&\mathbf{U}&\mathbf{V}&\mathbf{W}&\cdots\\ \mathbf{0}&\mathbf{0}&\mathbf{U}&\mathbf{V}&\cdots\\ \vdots&\vdots&\vdots&\ddots&\ddots\\ \end{array}\right), 
in which \mathbf{\tilde{V}},\mathbf{U},\mathbf{W},\mathbf{V}\in\mathbb{R}^{\left(B+1% \right)\times\left(B+1\right)} are
\displaystyle\mathbf{\tilde{V}}=\lambda\mathbf{I}+\left(\begin{array}[]{ccccc% }r&r&&&0\\ &r&r&&\\ &&\ddots&\ddots&\\ &&&r&r\\ 0&&&&0\\ \end{array}\right), 
\displaystyle\mathbf{V}=\lambda\mathbf{I}+\left(\begin{array}[]{ccccc}r&r&&&% 0\\ &(r+\mu)&r&&\\ &&\ddots&\ddots&\\ &&&(r+\mu)&r\\ 0&&&&\mu\\ \end{array}\right), 
\displaystyle\mathbf{U}=\left(\begin{array}[]{ccccc}0&&&&0\\ \mu&0&&\\ &\mu&\ddots&&\\ &&\ddots&0&\\ 0&&&\mu&0\\ \end{array}\right), 
and \mathbf{W}=\lambda\mathbf{I}.
Since the system is ergodic, there exists a stationary solution \{\mathbf{p}_{i}\}, where \mathbf{p}_{i}=\left\{p_{i,0},p_{i,1},\cdots,p_{i,B}\right\} denotes the probability of the system being at state (i,\cdot), such that \mathbf{p}_{i} satisfies the following recursive relationship
\mathbf{p}_{{i+1}}=\mathbf{p}_{i}\mathbf{R},i\in\mathbb{N}.  (17) 
Since
\left(\mathbf{p}_{0},\mathbf{p}_{1},\cdots\right)\mathbf{Q}=\mathbf{0},  (18) 
by substituting (17) into (18), we have
\mathbf{R}^{2}\mathbf{U}+\mathbf{R}\mathbf{V}+\mathbf{W}=\mathbf{0}  (19) 
and \mathbf{p}_{0}\left(\mathbf{\tilde{V}}+\mathbf{R}\mathbf{U}\right)=\mathbf{0}, along with constraint \mathbf{p}_{0}\sum_{i=0}^{\infty}\mathbf{R}^{i}\mathbf{1}^{\mathrm{T}}=\mathbf% {p}_{0}\left(\mathbf{I}\mathbf{R}\right)^{1}\mathbf{1}^{\mathrm{T}}=1, in which \mathbf{1}=(1,\cdots,1)\in\mathbb{R}^{B+1}. Based on (19), by iteratively computing \mathbf{R}_{n+1}=(\mathbf{R}^{2}_{n}\mathbf{U}+\mathbf{W})\mathbf{V}^{1} with initial condition \mathbf{R}_{0}=\mathbf{0}, \mathbf{R} can be approached, and the stationary distribution of system states is obtained. The algorithm to compute the stationary distribution is summarize in Algorithm 1.
According to Little’s law, the mean sojourn time of a status packet is
\mathbb{E}\left[T\right]=\frac{\bar{q_{1}}}{\lambda}=\frac{1}{\lambda}\sum_{i=% 1}^{+\infty}i\mathbf{p}_{0}\mathbf{R}^{i}\mathbf{1}^{\mathrm{T}}=\frac{1}{% \lambda}\mathbf{p}_{0}(\mathbf{I}\mathbf{R})^{2}\mathbf{R}\mathbf{1}^{% \mathrm{T}}. 
With the stationary distribution \{\mathbf{p}_{0}\mathbf{R}^{i}\}_{i\geq 0}, the expression for average peak AoI is obtained by Eq. (2), and summarized in the following theorem:
Theorem 7
Under the FCFS discipline, with status packet arrivals and energy packet arrivals being Poisson processes of rate \lambda=r\theta and r, and the service time being i.i.d. random variable following exponential distribution with mean \mu^{1}, the average peak AoI is
\displaystyle\mathbb{E}\left[A\right]=\frac{1}{\lambda}+\frac{1}{\lambda}% \mathbf{p}_{0}(\mathbf{I}\mathbf{R})^{2}\mathbf{R}\mathbf{1}^{\mathrm{T}},  (20) 
where \mathbf{p}_{0} and \mathbf{R} are computed by Algorithm 1.
V Numerical Analysis
Fig. 6 illustrates the average peak AoI versus the ratio of status packet arrival rate to service rate, and the ratio of energy arrival rate to service rate, respectively. The service rate \mu is set to 1\mathrm{s}^{1}, and the battery capacity B is set to 5 units. As the figure shows, the average peak age first decreases then rises with the growth of \frac{\lambda}{\mu}. The reason is intuitive: When the status packet arrival rate is small, the lack of status information generation leads to a low update frequency, while when the status packet arrival rate is large, long queuing delay becomes the dominant factor that aggravates the freshness of status information. Therefore, there exists an optimal status update frequency that achieves the minimum average peak AoI, which infers the optimal sensing rate for remote status update. It is also observed from the figure that the energy arrival rate r should be larger than the status packet arrival rate \lambda. Otherwise, the status packet queue is not stable.
Next, we focus on the negligibleservicetime regime to get insights into the problem.
Fig. 7 plots the three types of average penalty under the FCFS and LCFS disciplines. The average penalty under exponential penalty function is more sensitive to the change in datatoenergy ratio \theta, and the average penalty of shifted unit step function, i.e., the violation probability, is the least sensitive. The overall characteristics of the three penalty functions have several similarities:

Under the FCFS discipline, the average penalty first decreases then increases with the growth of \theta. This result is consistent with the one in average peak AoI when service time cannot be neglected.

Under the LCFS discipline, a larger \theta always gives a lower penalty, owing to a larger update frequency.

The LCFS discipline always outperforms the FCFS discipline. This result is consistent with the work in [12].
VA Impact from Buffer Capacity K under FCFS
Fig. 8(a) depicts the average AoI under different buffer capacities and ratio \theta with constant energy arrival rate. It is shown that the average AoI is significantly lower, if newly arrived status packets are dropped when the status packet arrival rate is large. However, as observed in Fig. 8(b), blocking more status packets cannot always reduce the average AoI when status packet arrival rate is small. The optimal buffer capacity under given battery capacity and status packet arrival rate can be found by numerical methods. Illustrations of average penalties under exponential penalty function and violation probability are shown in Fig. 8(c) and Fig. 8(d).
Moreover, as shown in Fig. 8(c), the average penalty under the exponential penalty function is extremely sensitive to status packet arrival rate when the buffer capacity is large. The sensitivity of exponential penalty function is previously observed in [6] for K=\infty and in the packet management problem in [5]. Especially, when the buffer capacity goes to infinity, the average penalty exists only if \lambda<r\alpha. The reason is the under the exponential penalty function, the penalty rises sharply when AoI is large. However, the upper bound condition on status packet arrival rate is not necessary under the LCFS discipline.
VB Impact from Battery Capacity B under FCFS
Fig. 9 compares average penalty under different battery capacities and ratio \theta with constant energy arrival rate r=1\mathrm{s}^{1}. As shown in the figure, when \theta is small, the increase in battery capacity does not noticeably reduce the average penalty. As the status packet arrival rate grows, different from the average power constrained case, the average penalty first drops then increases. The optimal status packet arrival rate can be found by bisection method.
VC Impact from the LCFS Discipline
According to Theorem 4–6, the average penalty under LCFS discipline also decays as battery capacity increases. Fig. 10 plots the average AoI under LCFS discipline, with data buffer K=1 and K=100. When K=100, little status packets is dropped, and the average AoI first drops and then increases as the status packet arrival rate grows, while with K=1, i.e., only the latest status packet is kept in the queue at each time, a higher status update frequency always gives a lower average age. Additionally, it is noticed from the figure and can be easily proved that LCFS discipline with K=1 leads to the lowest average AoI if the server is workingconserving. The same result also holds in the cases with exponential penalty function and shifted unit step function.
VI Conclusions and Future Work
In this paper, we investigate the nonlinear AoI for a status update system powered by renewable energy sources. With packet generation and energy arrival both being a Poisson process and service time following exponential distribution, a method to compute the average peak AoI is proposed. To gain more insights into the problem, a special case in which the service time is small enough is analyzed. The closedform expressions of average penalty under linear penalty function, exponential penalty function, and shifted unit step function for both FCFS and LCFS systems are obtained. Results show that under the FCFS discipline, the average penalty under the exponential penalty function is extremely sensitive to the status information generation frequency, especially when the buffer capacity or the parameter \alpha in exponent is large. As battery capacity increases, the difference in the average penalty between the energyharvesting case and the averagepowerconstrained case exponentially decays with rate equal to the ratio of status information generation frequency to energy arrival rate. For FCFS systems, there exists a unique minimum average penalty that can be achieved by a proper status information generation frequency, while in a LCFS system with unit buffer capacity, the minimum penalty is achieved, and a larger status information generation frequency always gives a smaller average penalty. It is also noticed that blocking more status packets can reduce the average penalty when the status generation frequency is large. The performance of the LCFS discipline and finite buffer capacity in a system with nonnegligible service time needs to be investigated in future work.
Appendix A Proof of Lemma 1
Under FCFS, a data packet is a valid update if and only if the data buffer is not full upon its arrival. Since the queuing system is equivalent to an M/M/1 queue, we have
\lim_{n\to\infty}\mathrm{P}\left\{S_{n}=j\right\}=\frac{\left(1\theta\right)% \theta^{j+B}}{1\theta^{K+B+1}}. 
According to the PASTA (Poisson Arrivals See Time Average) property, we have
\tilde{\lambda}_{\mathrm{FCFS}}=\lambda\sum_{j=B}^{K1}\lim_{n\to\infty}% \mathrm{P}\left\{S_{n}=j\right\}=\lambda\frac{1\theta^{K+B}}{1\theta^{K+B+1}}.  (21) 
Appendix B Proof of Lemma 2
By the total probability formula, we have that
\mathrm{P}\left\{A_{n}\leq a\right\}=\mathrm{P}\left\{X_{n}+T_{n}\leq a\right% \}=\int_{0}^{\infty}\mathrm{P}\left\{T_{n}\leq ax\big{}X_{n}=x\right\}f_{X_{% n}}(x)\mathrm{d}x, 
and that
\mathrm{P}\left\{T_{n}\leq t\right\}=\int_{0}^{\infty}\mathrm{P}\left\{T_{n}% \leq t\big{}X_{n}=x\right\}f_{X_{n}}(x)\mathrm{d}x. 
Therefore, we first obtain the CDF of interarrival time X of valid updates, and the conditional probability distribution of sojourn time T given interarrival time X.
Denoting the system state right before the arrival of the nth valid update as S_{n}^{}, since valid updates are the status packets that enter the data buffer, we get S_{n}^{}\leq K1. Applying the total probability formula again, we have
\displaystyle\mathrm{P}\left\{T_{n}\leq tX_{n}\right\}  \displaystyle=  \displaystyle\sum_{i=B}^{K1}\mathrm{P}\left\{S_{n}^{}=iX_{n}\right\}% \mathrm{P}\left\{T_{n}\leq tS_{n}^{}=i,X_{n}\right\}  (22)  
\displaystyle\overset{(a)}{=}  \displaystyle\sum_{i=B}^{K1}\mathrm{P}\left\{S_{n}^{}=iX_{n}\right\}% \mathrm{P}\left\{T_{n}\leq tS_{n}^{}=i\right\}, 
where (a) uses the fact that the sojourn time T_{n} of the nth valid update and the interarrival time X_{n} between the (n1)th and the nth valid update are conditionally independent given S_{n}^{}.
The second term on the righthand side of Eq. (22) is expressed as Eq. (23): when the battery is empty and there are already i status packets waiting to be served, i.e., S_{n}^{}\geq i, the new status packets will not be served until the arrival of i+1 energy packets; when the battery is not empty (S_{n}^{}<0), the new status packets will be delivered instantly.
\displaystyle\mathrm{P}\left\{T_{n}\leq tS_{n}^{}=i\right\}=\left\{\begin{% aligned} &\displaystyle\sum_{j=i+1}^{+\infty}\frac{\left(rt\right)^{j}}{j!}e^{% rt},&\displaystyle\mathrm{~{}if~{}}i\geq 0;\\ &\displaystyle 1,&\displaystyle\mathrm{~{}if~{}}i<0.\end{aligned}\right.  (23) 
Using the total probability formula in the first term on the righthand side of Eq. (22) , we have
\displaystyle\mathrm{P}\left\{S_{n}^{}=iX_{n}\right\}=\sum_{j=\max\left\{i1% ,B\right\}}^{K1}\mathrm{P}\left\{S_{n1}^{}=jX_{n}\right\}\mathrm{P}\left% \{S_{n}^{}=iX_{n},S_{n1}^{}=j\right\},  (24) 
where uses the condition that S_{n1}^{}+1\geq S_{n}^{} since there are only one valid update between S_{n1}^{} and S_{n}^{}.
To get Eq. (24), we first obtain the CDFs of interarrival time X_{n} and system state S_{n}^{} before arrival. According to the PASTA property, \mathrm{P}\left\{S_{n}^{}=i\right\} approaches the stationary probability of the system being at state i when n goes to infinity given the condition that the state is not K. Since the queuing system is equivalent to an M/M/1 queue, which gives \lim_{n\to\infty}\mathrm{P}\left\{S_{n}=i\right\}=\frac{\left(1\theta\right)% \theta^{i+B}}{1\theta^{K+B+1}} the probability of state being i is
\displaystyle\lim_{n\to\infty}\mathrm{P}\left\{S_{n}^{}=i\right\}=\frac{\lim_% {n\to\infty}\mathrm{P}\left\{S_{n}=i\right\}}{1\lim_{n\to\infty}\mathrm{P}% \left\{S_{n}=K\right\}}=\frac{1\theta}{1\theta^{K+B}}\theta^{i+B}.  (25) 
Next, we apply the total probability formula to obtain the CDF of interarrival time X_{n}:
\displaystyle\mathrm{P}\left\{X_{n}\leq x\right\}  \displaystyle=  \displaystyle\mathrm{P}\left\{X_{n}\leq xS_{n1}^{}=K1\right\}\mathrm{P}% \left\{S_{n1}^{}=K1\right\}  (26)  
\displaystyle+\sum_{j=B}^{K2}\mathrm{P}\left\{X_{n}\leq xS_{n1}^{}=j% \right\}\mathrm{P}\left\{S_{n1}^{}=j\right\}. 
When S_{n1}^{}<K1, the data buffer cannot be full at the arrival of the next status packet. Therefore, any status packet arrives right after the (n1)th valid update is a valid update, which gives
\mathrm{P}\left\{X_{n}\leq xS_{n1}^{}=j\right\}=1e^{\lambda x},j\leq K2;  (27) 
When S_{n1}^{}=K1, there are at least one energy packet arrival before the next status packet entering the buffer. Thus, we have
\displaystyle\mathrm{P}\left\{X_{n}\leq xS_{n1}^{}=K1\right\}  \displaystyle=  \displaystyle 1\frac{1}{1\theta}e^{\lambda x}+\frac{\theta}{1\theta}e^{rx}.  (28) 
Substituting Eq. (25), (27) and (28) into the righthand side of Eq. (26), we have
\mathrm{P}\left\{X_{n}\leq x\right\}=1+e^{rx}\frac{\theta^{K+B}}{1\theta^{K+% B}}e^{\lambda x}\frac{1}{1\theta^{K+B}}.  (29) 
Next we obtain the first term on the righthand side of Eq. (24) by Bayes’ theorem:
\displaystyle\mathrm{P}\left\{S_{n1}^{}=jX_{n}=x\right\}=\frac{f_{X_{n}S_{% n1}^{}=j}(x)\mathrm{P}\left\{S_{n1}^{}=j\right\}}{f_{X_{n}}(x)}. 
With Eq. (25), (27)–(29), the above equation can be expressed as:
\displaystyle\mathrm{P}\left\{S_{n1}^{}=jX_{n}=x\right\}=\left\{\begin{% aligned} &\displaystyle\frac{\left(1\theta\right)\theta^{j+B+1}e^{\lambda x}% }{e^{rx}\theta^{K+B}+\theta e^{\lambda x}},&\displaystyle\mathrm{~{}if~{}}j% <K1;\\ &\displaystyle\frac{e^{rx}\theta^{K+B}+e^{\lambda x}\theta^{K+B}}{e^{rx}% \theta^{K+B}+\theta e^{\lambda x}},&\displaystyle\mathrm{~{}if~{}}j=K1.\end{% aligned}\right.  (30) 
Now we obtain the second term on the righthand side of Eq. (24). If S_{n1}^{}\leq K2 and S_{n}^{}>B, then there are (S_{n1}^{}S_{n}^{}+1) energy packets arrived between the arrivals of the (n1)th and the nth valid update, which gives
\mathrm{P}\left\{S_{n}^{}=iX_{n}=x,S_{n1}^{}=j\right\}=\frac{\left(rx% \right)^{j+1i}}{\left(j+1i\right)!}e^{rx},\mathrm{~{}if~{}}j\leq K2,i>B.  (31) 
If S_{n1}^{}\leq K2 and S_{n}^{}=B, then there are at least (S_{n1}^{}+B+1) energy packets arrived between the arrivals of the (n1)th and the nth valid update:
\displaystyle\mathrm{P}\left\{S_{n}^{}=iX_{n}=x,S_{n1}^{}=j\right\}=\sum_{% m=j+B+1}^{+\infty}\frac{\left(rx\right)^{m}}{m!}e^{rx},\mathrm{~{}if~{}}j\leq K% 2,i=B.  (32) 
If S_{n1}^{}=K1, by the definition, we have
\displaystyle\mathrm{P}\left\{S_{n}^{}=iX_{n}=x,S_{n1}^{}=K1\right\}=% \frac{f_{S_{n}^{},X_{n}S_{n1}^{}=K1}(i,x)}{f_{X_{n}S_{n1}^{}=K1}(x)}.  (33) 
Further, if S_{n}^{}>B, then what happens between the arrivals of the (n1)th and the nth valid update is that the first energy packet arrives at t seconds after the arrival of the (n1)th valid update, and that there are (K1S_{n}^{}) energy packets and no status packet arrived between the arrivals of the first energy packet and the nth valid update. Thus, we have
\displaystyle f_{S_{n}^{},X_{n}S_{n1}^{}=K1}(i,x)=\int_{0}^{x}re^{r(xt)% }\frac{(rt)^{Ki1}}{(Ki1)!}e^{rt}\lambda e^{\lambda t}\,\mathrm{d}t,% \mathrm{~{}if~{}}S_{n}^{}>B.  (34) 
If S_{n}^{}=B, then between the arrivals of the (n1)th and the nth valid update, the first energy packet arrives at t seconds after the arrival of the (n1)th valid update, and there are at least K+B1 energy packets and no status packet arrived between the arrivals of the first energy packet and the nth valid update:
\displaystyle f_{S_{n}^{},X_{n}S_{n1}^{}=K1}\left(B,x\right)=\int_{0}^{x% }re^{r(xt)}\sum_{l=K+B1}^{+\infty}\frac{(rt)^{l}}{l!}e^{rt}\lambda e^{% \lambda t}\,\mathrm{d}t.  (35) 
Substituting Eq. (28), (34) and (35) into Eq. (33), we get
\displaystyle\mathrm{P}\left\{S_{n}^{}=iX_{n}=x,S_{n1}^{}=K1\right\}  (36)  
\displaystyle=  \displaystyle\left\{\begin{aligned} &\displaystyle\frac{1\theta}{\theta^{Ki}% }\frac{e^{rx}}{e^{\lambda x}e^{rx}}\sum_{m=Ki}^{+\infty}\frac{(\lambda x)% ^{m}}{m!}e^{\lambda x},&\displaystyle\mathrm{~{}if~{}}S_{n}^{}>B;\\ &\displaystyle\frac{e^{r(1+\theta)x}}{e^{\lambda x}e^{rx}}\left(\sum_{i=K+% B}^{+\infty}\frac{(rx)^{i}}{i!}\theta^{KB+1}\sum_{i=K+B}^{+\infty}\frac{(% \lambda x)^{i}}{i!}\right),&\displaystyle\mathrm{~{}if~{}}S_{n}^{}=B.\end{% aligned}\right. 
Substituting Eq. (30), (31), (32) and (36) into Eq. (24), we get
\displaystyle\mathrm{P}\left\{S_{n}^{}=iX_{n}=x\right\}=\left\{\begin{% aligned} &\displaystyle\frac{(1\theta)\theta^{B+i}e^{rx}}{e^{rx}\theta^{K+% B}+\theta e^{\lambda x}},&\displaystyle\mathrm{~{}if~{}}i>B;\\ &\displaystyle\frac{(e^{\lambda x}e^{rx})\theta}{e^{rx}\theta^{K+B}+% \theta e^{\lambda x}},&\displaystyle\mathrm{~{}if~{}}i=B.\end{aligned}\right.  (37) 
Combining Eq. (37) with Eq. (23), Eq. (22) becomes
\displaystyle\mathrm{P}\left\{T_{n}\leq tX_{n}=x\right\}=1  \displaystyle  \displaystyle\frac{\theta^{B}e^{rx}}{e^{rx}\theta^{K+B}+\theta e^{\lambda x% }}\sum_{n=0}^{K1}\frac{\left(\lambda t\right)^{n}}{n!}e^{rt}  
\displaystyle+  \displaystyle\frac{\theta^{K+B}e^{rx}}{e^{rx}\theta^{K+B}+\theta e^{% \lambda x}}\sum_{n=0}^{K1}\frac{\left(rt\right)^{n}}{n!}e^{rt}. 
Together with Eq. (29), by the total probability formula, the lemma is proved.
Appendix C Proof of Lemma 3
Under the LCFS discipline, if a status packet arrives when the battery is not empty or afterwards a energy packet arrives before another status packet enters the data buffer, the status packet is a valid update. Thus, by the PASTA property, the probability of a status packet being a valid update is
\displaystyle\sum_{i=B}^{1}\mathrm{Prob}\left\{S_{n}=i\right\}+\frac{1}{1+% \theta}\sum_{i=0}^{K}\mathrm{Prob}\left\{S_{n}=i\right\}=\frac{(\theta^{B}1)% \left(1+\theta\right)+(1\theta^{K+1})}{\left(\theta^{B}\theta^{K+1}\right)% \left(1+\theta\right)}, 
and the arrival rate of valid update is \tilde{\lambda}=\lambda\frac{(\theta^{B}1)\left(1+\theta\right)+(1\theta^{K% +1})}{\left(\theta^{B}\theta^{K+1}\right)\left(1+\theta\right)}.
Appendix D Proof of Lemma 4
We first obtain the probability distribution of the state S_{n}^{} before valid updates’ arrivals. By the definition of conditional probability and the PASTA property, we have
\displaystyle\mathrm{Prob}\left\{S_{n}^{}=i\right\}  \displaystyle=  \displaystyle\frac{\mathrm{Prob}\left\{S_{n}=i\mathrm{~{}upon~{}arrival},% \mathrm{valid~{}update}\right\}}{\mathrm{P}\left\{\mathrm{valid~{}update}% \right\}}  (38)  
\displaystyle=  \displaystyle\left\{\begin{aligned} &\displaystyle\frac{\left(1\theta\right)% \theta^{i}}{\left(\theta^{B}1\right)\left(1+\theta\right)+\left(1\theta^{K+% 1}\right)},&\displaystyle\mathrm{~{}if~{}}0\leq i\leq K;\\ &\displaystyle\frac{\left(1\theta^{2}\right)\theta^{i}}{\left(\theta^{B}1% \right)\left(1+\theta\right)+\left(1\theta^{K+1}\right)},&\displaystyle% \mathrm{~{}if~{}}B\leq i\leq1.\end{aligned}\right. 
Since the status packets arriving when the battery is not empty have zero sojourn time, we have
\displaystyle\mathrm{Prob}\left\{T_{n}\leq t\right\}=\sum_{i=B}^{1}\mathrm{% Prob}\left\{S_{n}^{}=i\right\}+\sum_{i=0}^{K}\mathrm{Prob}\left\{S_{n}^{}=i% \right\}\mathrm{Prob}\left\{T_{n}\leq tS_{n}^{}=i\right\},  (39) 
where \mathrm{Prob}\left\{T_{n}\leq tS_{n}^{}=i\geq 0\right\} is the conditional probability distribution of sojourn time given that an energy packet comes before another status packet after the nth valid update, which leads to
\mathrm{Prob}\left\{T_{n}\leq tS_{n}^{}=i\right\}=(1+\theta)\int_{0}^{t}re^{% r\tau}e^{\lambda\tau}\mathrm{d}\tau=1e^{\left(\lambda+r\right)t},\mathrm{~% {}if~{}}i\geq 0.  (40) 
Substituting Eq. (38) and (40) into Eq. (39), we have
\displaystyle\mathrm{Prob}\left[T_{n}\leq t\right]  \displaystyle=  \displaystyle 1e^{\left(\lambda+r\right)t}\frac{\left(1\theta^{K+1}\right)}% {\left(\theta^{B}1\right)\left(1+\theta\right)+\left(1\theta^{K+1}\right)}  (41) 
For the CDF of peak age A, we first obtain the conditional probability of the interdelivery time D_{n} between the delivery of the nth and the (n+1)th valid update given the sojourn time T_{n} of the nth valid update. After that, the CDF of peak age A can be given by the total probability formula:
\mathrm{P}\left\{A\leq a\right\}=\int_{0}^{a}f_{T_{n}}(ad)\mathrm{P}\left\{D_% {n}\leq dT_{n}=ad\right\}\,\mathrm{d}d  (42) 
Denote the system state right after the delivery of the nth valid update as S_{n}^{+}. By the total probability formula, we have
\displaystyle\mathrm{Prob}\left\{D_{n}\leq dT_{n}=t\right\}=\sum_{i=B+1}^{K}% \mathrm{Prob}\left\{D_{n}\leq dT_{n}=t,S_{n}^{+}=i\right\}\mathrm{Prob}\left% \{S_{n}^{+}=iT_{n}=t\right\}. 
Due to the fact that the interdelivery time D_{n} between the delivery of the nth and the (n+1)th valid update and the sojourn time T_{n} of the nth valid update are conditionally independent given the state S_{n}^{+} right after the nth valid update’s delivery, the former equation can be simplified as
\displaystyle\mathrm{Prob}\left\{D_{n}\leq dT_{n}=t\right\}=\sum_{i=B+1}^{K}% \mathrm{Prob}\left\{D_{n}\leq dS_{n}^{+}=i\right\}\mathrm{Prob}\left\{S_{n}^{% +}=iT_{n}=t\right\}.  (43) 
First, we obtain the first term on the righthand side of Eq. (43). If S_{n}^{+}<0, the interdelivery time equals the interarrival time of two successive status packets, which gives
\mathrm{Prob}\left\{D_{n}\leq dS_{n}^{+}=i\right\}=1e^{\lambda d},\mathrm{~{% }if~{}}i<0.  (44) 
If S_{n}^{+}\geq 0, the complement of D_{n}\leq d is that in d after the nth valid update’s delivery, either there is no status packet arrival or there is no energy arrival, or that no more than S_{n}^{+} energy packets arrive and all the energy packets arriving in d after the nth valid update’s delivery come before status packets. Therefore, if S_{n}^{+}=0, we have
\displaystyle\mathrm{Prob}\left\{D\leq dS_{n}^{+}=0\right\}  \displaystyle=  \displaystyle\mathrm{Prob}\left\{\mathrm{both~{}data~{}arrivals~{}and~{}energy% ~{}arrivals~{}in~{}}d\right\}  (45)  
\displaystyle=  \displaystyle 1e^{\lambda d}e^{rd}+e^{r\left(1+\theta\right)d}. 
If S_{n}^{+}>0, we have
\displaystyle\mathrm{Prob}\left\{D_{n}\leq dS_{n}^{+}=i\right\}  (46)  
\displaystyle=  \displaystyle\mathrm{Prob}\left\{\mathrm{both~{}data~{}arrivals~{}and~{}energy% ~{}arrivals~{}in~{}}d\right\}  
\displaystyle\sum_{k=1}^{i}\mathrm{Prob}\left\{k\mathrm{~{}energy~{}arrivals~% {}in~{}}d,\mathrm{~{}all~{}energy~{}arrivals~{}before~{}the~{}first~{}data~{}% arrival}\right\}  
\displaystyle=  \displaystyle 1e^{\lambda d}\frac{\theta^{i}\theta}{1\theta}e^{rd}e^{% r\left(1+\theta\right)d}\left[\frac{\theta}{1\theta}\sum_{k=0}^{i}\frac{\left% (rd\right)^{k}}{k!}\frac{\theta^{i}}{1\theta}\sum_{k=0}^{i}\frac{\left(r% \theta d\right)^{k}}{k!}\right]. 
Next, we obtain the second term on the righthand side of Eq. (43). If T_{n}=0, we have that the state S_{n}^{} right before the nth valid update’s arrival is negative, and that S_{n}^{+}=S_{n}^{}+1. Therefore, we get
\displaystyle\mathrm{Prob}\left\{S_{n}^{+}=iT_{n}=0\right\}  \displaystyle=  \displaystyle\mathrm{Prob}\left\{S_{n}^{}=i1S_{n}^{}<0\right\}=\left\{% \begin{aligned} &\displaystyle 0,&\displaystyle\mathrm{~{}if~{}}i\geq 1;\\ &\displaystyle\frac{\left(1\theta\right)\theta^{i+B1}}{1\theta^{B}},&% \displaystyle\mathrm{~{}if~{}}i\leq 0.\end{aligned}\right.  (47) 
The sufficient and necessary condition of T_{n}>0 is S_{n}^{}\geq 0. If T_{n}>0, the valid update first enters the data buffer, and is deliveried when an energy packet arrives. Therefore, S_{n}^{}=S_{n}^{+}, and
\displaystyle\mathrm{Prob}\left\{S_{n}^{+}=iT_{n}=t\right\}  \displaystyle=  \displaystyle\mathrm{Prob}\left\{S_{n}^{}=iS_{n}^{}\geq 0\right\}=\left\{% \begin{aligned} &\displaystyle 0,&\displaystyle\mathrm{~{}if~{}}i<0;\\ &\displaystyle\frac{\left(1\theta\right)\theta^{i}}{1\theta^{K+1}},&% \displaystyle\mathrm{~{}if~{}}i\geq 0.\end{aligned}\right.  (48) 
Substituting Eq. (44)–(48) into Eq. (43), we get
\displaystyle\mathrm{Prob}\left[D_{n}\leq dT_{n}=t\right]  (49)  
\displaystyle=  \displaystyle\left\{\begin{aligned} &\displaystyle\left(1e^{\lambda d}\right% )\left[1e^{rd}\frac{\theta^{B1}\theta^{B}}{1\theta^{B}}\right],&% \displaystyle t=0;\\ &\displaystyle 1e^{\lambda d}\frac{K\frac{\theta^{2}\theta^{K+2}}{1% \theta}}{1\theta^{K+1}}e^{rd}&\\ &\displaystyle~{}~{}\frac{e^{r\left(1+\theta\right)d}}{1\theta^{K+1}}\left[% \sum_{k=0}^{K1}(Kk)\frac{\left(r\theta d\right)^{k}}{k!}+\frac{\theta^{2}}{% 1\theta}\sum_{k=0}^{K1}\frac{\left(r\theta d\right)^{k}}{k!}\frac{\theta^{K% +2}}{1\theta}\sum_{k=0}^{K1}\frac{\left(rd\right)^{k}}{k!}\right]&% \displaystyle t>0.\end{aligned}\right. 
References
 [1] S. Kaul, R. Yates, and M. Gruteser, “Realtime status: How often should one update?” in 2012 IEEE Conference on Computer Communications (INFOCOM), Mar. 2012.
 [2] S. Sudevalayam and P. Kulkarni, “Energy harvesting sensor nodes: Survey and implications,” IEEE Communications Surveys & Tutorials, vol. 13, no. 3, pp. 443–461, Third quarter 2011.
 [3] Y. Sun and B. Cyr, “Sampling for data freshness optimization: Nonlinear age functions,” arXiv:1812.07241, 2018.
 [4] R. Devassy, G. Durisi, G. C. Ferrante, O. Simeone, and E. UysalBiyikoglu, “Delay and peakage violation probability in shortpacket transmissions,” arXiv:1805.03271, 2018.
 [5] Y. Sun, E. UysalBiyikoglu, R. D. Yates, C. E. Koksal and N. B. Shroff, “Update or wait: How to keep your data fresh,” IEEE Trans. Inf. Theory, vol. 63, no. 11, pp. 74927508, Nov. 2017.
 [6] A. Kosta, N. Pappas, A. Ephremides, and V. Angelakis, “Age and value of information: Nonlinear age case,” in 2017 IEEE International Symposium on Information Theory (ISIT), June 2017.
 [7] X. Guo, R. Singh, P. R. Kumar, and Z. Niu, “A risksensitive approach for packet interdelivery time optimization in networked cyberphysical systems,” IEEE/ACM Trans. Netw., vol. 26, pp. 1976–1989, Aug. 2018.
 [8] C. Kam, S. Kompella, G. D. Nguyen, and A. Ephremides, “Effect of message transmission path diversity on status age,” IEEE Trans. Inf. Theory, vol. 62, pp. 1360–1374, Mar. 2016.
 [9] L. Huang and E. Modiano, “Optimizing ageofinformation in a multiclass queueing system,” in 2015 IEEE International Symposium on Information Theory (ISIT), June 2015.
 [10] K. Chen and L. Huang, “Ageofinformation in the presence of error,” in 2016 IEEE International Symposium on Information Theory (ISIT), July 2016.
 [11] A. Soysal and S. Ulukus, “Age of information in G/G/1/1 systems,” arXiv: 1805.12586, 2018.
 [12] A. M. Bedewy, Y. Sun, and N. B. Shroff, “Optimizing data freshness, throughput, and delay in multiserver informationupdate systems,” in 2016 IEEE International Symposium on Information Theory (ISIT), July 2016.
 [13] M. Costa, M. Codreanu, and A. Ephremides, “On the age of information in status update systems with packet management,” IEEE Trans. Inf. Theory, vol. 62, pp. 1897–1910, Apr. 2016.
 [14] Y. Inoue, H. Masuyama, T. Takine, and T. Tanaka, “A general formula for the stationary distribution of the age of information and its application to singleserver queues,” arXiv:1804.06139, 2018.
 [15] R. D. Yates and S. K. Kaul, “The age of information: Realtime status updating by multiple sources,” IEEE Trans. Inf. Theory, vol. 65, no. 3, pp. 1807–1827, Mar. 2019.
 [16] I. Kadota, A. Sinha, E. UysalBiyikoglu, R. Singh, and E. Modiano, “Scheduling policies for minimizing age of information in broadcast wireless networks,” arXiv:1801.01803, 2018.
 [17] Z. Jiang, B. Krishnamachari, S. Zhou, and Z. Niu, “Can decentralized status update achieve universally nearoptimal ageofinformation in wireless multiaccess channels?” 2018 30th International Teletraffic Congress (ITC 30), Sept. 2018.
 [18] Z. Jiang, B. Krishnamachari, X. Zheng, S. Zhou, and Z. Niu, “Decentralized status update for ageofinformation optimization in wireless multiaccess channels,” IEEE International Symposium on Information Theory (ISIT), June 2018.
 [19] S. Ulukus, A. Yener, E. Erkip, O. Simeone, M. Zorzi, P. Grover, and K. Huang, “Energy harvesting wireless communications: A review of recent advances,” IEEE J. Sel. Areas Commun., vol. 33, pp. 360–381, Mar. 2015.
 [20] O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, and A. Yener, “Transmission with energy harvesting nodes in fading wireless channels: Optimal policies,” IEEE J. Sel. Areas Commun., vol. 29, pp. 1732–1743, Sept. 2011.
 [21] V. Sharma, U. Mukherji, V. Joseph, and S. Gupta, “Optimal energy management policies for energy harvesting sensor nodes,” IEEE Trans. Wireless Commun., vol. 9, pp. 1326–1336, Apr. 2010.
 [22] X. Zheng, S. Zhou, and Z. Niu, “On the online minimization of completion time in an energy harvesting system,” in International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), May 2016.
 [23] B. T. Bacinoglu, E. T. Ceran, and E. UysalBiyikoglu, “Age of information under energy replenishment constraints,” in Information Theory and Applications Workshop (ITA), Feb. 2015.
 [24] R. D. Yates, “Lazy is timely: Status updates by an energy harvesting source,” in 2015 IEEE International Symposium on Information Theory (ISIT), June 2015.
 [25] X. Wu, J. Yang, and J. Wu, “Optimal status update for age of information minimization with an energy harvesting source,” IEEE Trans. Green Commun. Netw., vol. 2, no. 1, pp. 193–204, Mar. 2018.
 [26] A. Arafa and S. Ulukus, “Ageminimal transmission in energy harvesting twohop networks,” arXiv: 1704.08679, 2017.
 [27] B. T. Bacinoglu and E. UysalBiyikoglu, “Scheduling status updates to minimize age of information with an energy harvesting sensor,” in IEEE International Symposium on Information Theory (ISIT), June 2017.
 [28] A. Arafa, J. Yang, S. Ulukus, and H. V. Poor, “Ageminimal transmission for energy harvesting sensors with finite batteries: Online policies,” arXiv: 1806.07271, 2018.
 [29] S. Farazi, A. G. Klein, and D. R. Brown, “Average age of information for status update systems with an energy harvesting server,” in IEEE INFOCOM Workshops, Apr. 2018.
 [30] S. Farazi, A. G. Klein, and D. R. Brown, “Age of information in energy harvesting status update systems: When to preempt in service?” in IEEE International Symposium on Information Theory (ISIT), June 2018.
 [31] H. S. Wang and N. Moayeri, “Finitestate markov channela useful model for radio communication channels,” IEEE Trans. Veh. Technol., vol. 44, pp. 163–171, Feb. 1995.
 [32] T. Z. Ornee and Y. Sun, “Sampling for remote estimation through queues: Age of information and beyond,” arXiv:1902.03552, 2019.
 [33] M. Neuts, Matrixgeometric Solutions in Stochastic Models: An Algorithmic Approach. Algorithmic Approach, Dover Publications, 1994.