# Age-Optimal Transmission of Rateless Codes

in an Erasure Channel

###### Abstract

In this paper, we examine a status updating system where updates generated by the source are sent to the monitor through an erasure channel. We assume each update consists of symbols and the symbol erasure in each time slot follows an independent and identically distributed (i.i.d.) Bernoulli process. We assume rateless coding scheme is adopted at the transmitter and an update can be successfully decoded if coded symbols are received successfully. We assume perfect feedback available at the source, so that it knows whether a transmitted symbol has been erased instantly. Then, at the beginning of each time slot, the source has the choice to start transmitting a new update, or continue with the transmission of the previous update if it is not delivered yet. We adopt the metric “Age of Information” (AoI) to measure the freshness of information at the destination, where the AoI is defined as the age of the latest decoded update at the destination. Our objective is to design an optimal online transmission scheme to minimize the time-average AoI. The transmission decision is based on the instantaneous AoI, the age of the update being transmitted, as well as the number of successfully delivered symbols of the update. We formulate the problem as a Markov Decision Process (MDP) and identify the monotonic threshold structure of the optimal policy. Numerical results corroborate the structural properties of the optimal solution.

## I Introduction

In recent years, the pervasiveness and availability of wireless connectivity and the proliferation of smart mobile devices have ushered in the era of Internet of Everything, where the cyber, physical, and human worlds are intertwined and continuously interacting with each other. In such complex systems, continuous data streams are generated by a large variety of sources (sensors, humans, electronic records, etc.), transferred through different communication pipelines, delivered to data warehouses (central controllers, edge/cloud servers, etc.), and eventually processed and analyzed, facilitating real-time information sharing and decision-making. The applications range from automatic robotic control, remote healthcare and medical support, to high-frequency trading, social media trending topics tracking, to name just a few. In such applications, the timeliness of the data delivered to the data consumer is of paramount importance: While a millisecond delay of the stock price may result in million-dollar loss of revenue in high-frequency trading, a delayed reporting of heart attack symptoms in a few seconds could lead to loss of lives. Therefore, there is a compelling need to quantify and study the “freshness” of information, and to develop information gathering and processing techniques to ensure information freshness in real-time systems.

Recently, a new metric called Age of Information (AoI) has been introduced [1] to measure the timeliness of information in status monitoring systems. Originally proposed for a simple monitoring system where the destination keeps track of the system status through time-stamped status updates, the metric is proved to be of fundamental importance for quantifying the freshness of information from the destination’s perspective. Specifically, at time , the AoI at the destination is defined as , where is the time-stamp of the latest received update at the destination, i.e., the time at which it was acquired at the source. Thanks to its simple and elegant form and its effectiveness in connecting data generation with its transmission for timely information delivery, AoI has attracted growing attention from different research communities.

The time average of AoI has been analyzed in different queueing models, such as single-source single-server queues [1], the Last-Come First-Served (LCFS) queue with preemption in service [2], the queue with multiple sources [3, 4, 5], the LCFS queue with gamma-distributed service time and Poisson update packet arrivals [6]. The AoI in systems with multiple servers has been evaluated in [7, 8, 9]. For more complicated multi-hop networks, reference [10] introduces a novel stochastic hybrid system (SHS) approach to derive explicit age distributions. The optimality properties of a preemptive Last Generated First Served (LGFS) service discipline in a multi-hop network are established in [11]. With the knowledge of the state of the server, an AoI optimization problem is studied in [12] and it shows that the zero-wait policy does not always minimize the AoI. In communication networks, AoI is evaluated in a CSMA system in [13] and has also been examined in multiple-access channels [14, 15] and broadcast channels [16, 17, 18, 19].

AoI of coded status updating for erasure channels have been studied recently [20, 21]. The long-term average AoI under two different coding strategies, i.e., rateless codes and maximum distance separable (MDS) codes, are characterized for single and multiple monitor systems in [20]. It is shown that MDS coding can match the AoI performance of rateless coding if the redundancy is carefully optimized in response to the channel erasure rate. In [21], it considers an energy harvesting erasure channel and shows that rateless coding with save-and-transmit scheme outperforms MDS based schemes.

In this paper, we focus on AoI optimization for an erasure channel with rateless codes. Instead of characterizing the long-term average AoI with the given coding scheme, our objective is to design online transmission scheduling policies for the rateless codes so that the long-term average AoI is minimized. Specifically, during the transmission of an update, the source has the choice to preempt the current transmission and switch to a new update, or to continue transmitting the current update until it is successfully decoded, based on the instantaneous feedback from the destination. We focus on stationary Markovian policies, and show that the optimal policy exhibits threshold structures through theoretical analysis.

Age-dependent preemptive policies have been studied in [22, 23, 24] for various system settings. It is shown in [22] that within a class of stationary Markov policies where the decision only depends on the instantaneous AoI at the destination, persistent policies such as to always drop the new update or the old update is optimal for certain service time distributions. Our setup is different from [22] in the sense that: 1) the service time (i.e., transmission time of each update) has a specific distribution that is not captured in [22]. 2) Our policy is not only dependent on the AoI at the destination, but also on the age of the unfinished update and the number of successfully delivered symbols of it.

## Ii System Model and Problem Formulation

We consider a status monitoring system where a source and a monitor is connected by an erasure channel. The source continuously transmits updates to the monitor, where each update consists of symbols. To combat symbol erasures, we assume each update is encoded by a rateless code such that when coded symbols are correctly received by the monitor, the update is successfully decoded. We assume the source transmits one coded symbol in each time slot, which can be successfully delivered with probability . With instantaneous transmission feedback at the end of each time slot, the source has to decide whether to start transmitting a new packet, or to continue transmitting the previous one if it has not been successfully decoded yet.

Denote the decision of the source at the beginning of time slot as , where represents transmitting a new update and represents transmitting the unfinished update (if there is one) or being idle. As illustrated in Fig.1, whenever , the source starts sending a new update, whose age will continuously grow (as indicated by the red dotted curve), and the AoI at the destination will grow simultaneously as well (as indicated by the blue curve). Once the update is successfully decoded, the AoI at the destination will be reset to the age of the delivered update. Due to the random erasures happening during the transmission, each update may take a different duration to get delivered.

Define as the total age of information experienced by the system over . We focus on a set of online policies , in which the information available for determining includes the decision history , the up-to-date erasure pattern, as well as the system parameters (i.e., , in this scenario). The optimization problem can be formulated as

(1) |

where the expectation in the objective function is taken over all possible erasure patterns.

### Ii-a MDP Formulation

The optimization problem in (1) is very challenging to solve in general, due to the random erasures and the temporal dependency in the AoI evolution. In order to make the problem analytically tractable, in the following, we focus on stationary Markovian policies where the decision only depends on the state tuple . Here and are the AoI at the destination and the age of unfinished update at the beginning of time slot , respectively. If , it indicates that an update has just been decoded. is the number of successfully delivered symbols of the unfinished update at the beginning of time slot . When , we have as well. We note that .

Assume . Then, under decision , the system will transit from state to according to following equation:

(2) |

If and ,

(3) |

and if and ,

(4) |

Let be the instantaneous AoI at the beginning of time slot under state . Then, .

Let be the set of stationary Markovian policies where only depends on . Then, the long-term average AoI under scheduling policy is given by

(5) |

where represents the expectation when scheduling policy is employed.

In order to obtain the optimal solution to (5), we first define the corresponding discounted-cost problem with a discount factor , and obtain the dynamic programming formulation

(6) |

The expectation in (6) is taken over all possible state transiting from state with action , and for all . As , , which is the unique solution of the following optimal equation

(7) |

This is a three-dimensional MDP, which is difficult to solve in general. In the following, we first determine some structural properties of the optimal policy for the -discounted problem. We can then prove that the structure still exists when , i.e., for the average cost problem.

## Iii Structure of the Optimal Policy

Before we proceed, we introduce the following state-action value functions

Then the optimality conditions in (6) and (7) are equivalent to

(8) | ||||

(9) |

We note that for any valid state , we must have , and . Define the set of valid states as .

###### Lemma 1

For any , is monotonically increasing in at every iteration . Therefore, monotonically increases in .

###### Proof.

We prove it by induction. It is obviously true when . We assume it is true for an . Then, we will prove it holds for the th iteration as well.

If , we have

(10) | ||||

(11) |

Based on the assumption, and are both non-decreasing in . Thus, after taking the minimum of them, is increasing in as well.

Similarly, we can prove that, when , is increasing in . The proof is complete after we combine both cases. ∎

###### Lemma 2

For any , is monotonically increasing in at every iteration . Therefore, monotonically increases in .

The proof of Lemma 2 is similar to the proof of Lemma 1, and is omitted for the brevity of this paper.

We have established the monotonicity of in and in Lemma 1 and Lemma 2, respectively. In the following, we aim to show the monotonicity of in as well. Before we prove it in Lemma 5, we will first introduce Lemma 3 and Lemma 4, respectively.

###### Lemma 3

For any , at every iteration . Therefore, .

###### Proof.

It holds when . Assume it is true for an . We aim to show that it holds for as well.

According to Eqn. (8), we have

where the last inequality is based on the assumption that Lemma 3 holds for an .

Besides, based on the definition of , we have

Therefore, . ∎

Roughly speaking, Lemma 3 implies that is the “worst” state among the set of states in the form of . Based on Lemma 3, we also have the following corollary.

###### Corollary 1

Whenever the source successfully delivers a update, it should start transmitting a new update immediately.

Corollary 1 indicates that the zero-wait policy in this scenario is actually optimal, which is in contrast to the result in [12]. This is because under our setup, the source is allowed to preempt the transmission of an update at any time, while in [12], the server will finish the transmission once it starts.

###### Corollary 2

Whenever the source starts transmitting a new update and the first symbol is erased, it should discard it and start transmitting a new update immediately. Besides, for any , at every iteration .

The next lemma characterizes the relationship between the value function at state and state .

###### Lemma 4

For any , at every iteration . Therefore, .

###### Proof.

The statement is true for . Assume for an , for any state in . We aim to show the inequality holds for as well.

Meanwhile,

Therefore, based on Lemma 1 and the fact that , we have

(15) |

Now, it is ready to establish the monotonicity structure of on state value function .

###### Lemma 5

For any , is monotonically decreasing in at every iteration . Therefore, monotonically decreases in .

###### Proof.

We prove it through induction. Similar to previous proofs, we note that it is true when and we assume it holds for an . Then, we show that it holds for the th iteration as well.

We note that . Besides, for any valid state , , we have

where the inequality is based on the assumption that decreases in .

Since takes the minimum of and , after combining both cases, we have . ∎

In the following, we will prove two threshold structures of the optimal policy based on Lemma 1 through Lemma 5. The first one is the threshold structure with respect to the age of unfinished update, as described in Theorem 1, while the second one is the threshold structure on the number of successfully delivered symbols of the update being transmitted, as detailed in Theorem 2.

###### Theorem 1

Under the optimal policy in , if the source decides to discard the unfinished update and start transmitting a new update at state , it should make the same decision at state .

###### Proof.

We note that for any valid state . Therefore, it is equivalent to show that is non-decreasing in .

Combining both cases, we have and monotonically increases in for all . ∎

###### Theorem 2

Under the optimal policy in , if the source decides to keep sending the unfinished update at state , it should make the same decision at state .

###### Proof.

To show Theorem 2, it suffices to show that is monotonically decreasing in for every . We prove this through contradiction. Assume , however, . Then, we must have

(21) |

Hence,

(22) | |||

(23) | |||

(24) |

which contradicts the fact that is decreasing in , as shown in Lemma 5. Therefore is monotonically decreasing in . ∎

###### Theorem 3

Under the optimal policy in , if the source decides to keep sending the unfinished update at state , it should make the same decision at state .

###### Proof.

To prove Theorem 3, we will prove its contrapositive instead, i.e., if , we must have . We focus on the states with .

Applying Theorem 3 recursively and combining with Corollary 2, Theorem 1, Theorem 2, we characterize the structure of the optimal policy in the following theorem.

###### Theorem 4

Consider the optimal policy in . Starting at a state , there exists a threshold for every , such that if the source attempts to transmit an update for time slots, and coded symbols are successfully delivered, the source should quit the unfinished update and start transmitting a new update at state ; Otherwise, it will keep transmitting until the update is successfully decoded. Besides, monotonically increases in .

We point out that all of the structural properties of the optimal policy derived for the -discounted problem hold when [25]. Thus, the optimal policy for the time-average problem also exhibits similar threshold structure.

## Iv Numerical Results

In this section, we numerically search for the optimal thresholds and evaluate the performance of the corresponding transmission policy.

We propose a structured value iteration algorithm to obtain the thresholds, as detailed in Algorithm 1. In order to reduce the computational complexity, we define an approximate MDP as follows: We define as the boundary AoI, and truncate the state space of the original MDP as . If under the action , becomes outside , we will let it be the corresponding capped boundary state. We can show that the approximate MDP is identical to the original MDP when . Besides, we also leverage the structure of the optimal policy to reduce the computational complexity.

Fig. 6 shows the structure of the optimal policy. In Fig. 6(a), we observe the threshold structure on for fixed . Fig. 6(b) indicates the threshold structure on . Finally, Fig. 6(b)-(d) exhibits the properties described in Theorem 3 and Theorem 4.

Next, we evaluate the performances of the optimal policy and a persistent policy in Fig. 7. Under the persistent policy, the source keeps transmitting an update until it is delivered. We fix update size and vary in . We note that the optimal policy outperforms the persistent policy and the performance gap decreases as . This is intuitive since when the erasure probability (i.e.,) decreases, the source has less incentive to discard an unfinished update, thus it becomes identical to the persistent policy over more time slots.

## V Conclusions

In this paper, we considered the optimal transmission scheduling of rateless codes in an erasure channel for AoI optimization. Theoretical analysis indicates that the optimal policy has a monotonic threshold structure.

## References

- [1] S. K. Kaul, R. D. Yates, and M. Gruteser, “Real-time status: How often should one update?” in IEEE INFOCOM, Orlando, FL, USA, Mar. 2012, pp. 2731–2735.
- [2] ——, “Status updates through queues,” in Conference on Information Sciences and Systems (CISS), Princeton, NJ, USA, Mar. 2012, pp. 1–6.
- [3] R. D. Yates and S. K. Kaul, “Real-time status updating: Multiple sources,” in IEEE International Symposium on Information Theory (ISIT), Cambridge, MA, USA, Jul. 2012, pp. 2666–2670.
- [4] ——, “The age of information: Real-time status updating by multiple sources,” ArXiv e-prints, 2016. [Online]. Available: http://arxiv.org/abs/1608.08622
- [5] N. Pappas, J. Gunnarsson, L. Kratz, M. Kountouris, and V. Angelakis, “Age of information of multiple sources with queue management,” in IEEE International Conference on Communications (ICC), Jun. 2015, pp. 5935–5940.
- [6] E. Najm and R. Nasser, “Age of information: The gamma awakening,” in IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain, Jul. 2016, pp. 2574–2578.
- [7] C. Kam, S. Kompella, and A. Ephremides, “Age of information under random updates,” in IEEE International Symposium on Information Theory (ISIT), Istanbul, Turkey, Jul. 2013, pp. 66–70.
- [8] ——, “Effect of message transmission diversity on status age,” in IEEE International Symposium on Information Theory (ISIT), Honolulu, HI, USA, Jun. 2014, pp. 2411–2415.
- [9] C. Kam, S. Kompella, G. D. Nguyen, and A. Ephremides, “Effect of message transmission path diversity on status age,” vol. 62, no. 3, pp. 1360–1374, Mar. 2016.
- [10] R. D. Yates, “The Age of Information in Networks: Moments, Distributions, and Sampling,” ArXiv e-prints, Jun. 2018.
- [11] A. M. Bedewy, Y. Sun, and N. B. Shroff, “Optimizing data freshness, throughput, and delay in multi-server information-update systems,” in IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain, Jul. 2016, pp. 2569–2573.
- [12] Y. Sun, E. Uysal-Biyikoglu, R. D. Yates, C. E. Koksal, and N. B. Shroff, “Update or wait: How to keep your data fresh,” in IEEE INFOCOM, San Francisco, CA, USA, Apr. 2016, pp. 1–9.
- [13] S. Kaul, M. Gruteser, V. Rai, and J. Kenney, “Minimizing age of information in vehicular networks,” in 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, Jun. 2011, pp. 350–358.
- [14] S. K. Kaul and R. D. Yates, “Status updates over unreliable multiaccess channels,” ArXiv e-prints, 2017. [Online]. Available: http://arxiv.org/abs/1705.02521
- [15] I. Kadota, A. Sinha, and E. Modiano, “Optimizing age of information in wireless networks with throughput constraints,” in IEEE INFOCOM, Apr. 2018.
- [16] I. Kadota, E. Uysal-Biyikoglu, R. Singh, and E. Modiano, “Minimizing the age of information in broadcast wireless networks,” in 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Sep. 2016, pp. 844–851.
- [17] I. Kadota, A. Sinha, E. Uysal-Biyikoglu, R. Singh, and E. Modiano, “Scheduling Policies for Minimizing Age of Information in Broadcast Wireless Networks,” ArXiv e-prints, Jan. 2018.
- [18] Y. P. Hsu, E. Modiano, and L. Duan, “Age of information: Design and analysis of optimal scheduling algorithms,” in IEEE International Symposium on Information Theory (ISIT), June 2017, pp. 561–565.
- [19] Y.-P. Hsu, “Age of Information: Whittle Index for Scheduling Stochastic Arrivals,” ArXiv e-prints, Jan. 2018.
- [20] R. D. Yates, E. Najm, E. Soljanin, and J. Zhong, “Timely updates over an erasure channel,” in 2017 IEEE International Symposium on Information Theory (ISIT), Jun. 2017, pp. 316–320.
- [21] A. Baknina and S. Ulukus, “Coded status updates in an energy harvesting erasure channel,” in Conference on Information Sciences and Systems (CISS), Mar. 2018.
- [22] V. Kavitha, E. Altman, and I. Saha, “Controlling Packet Drops to Improve Freshness of information,” ArXiv e-prints, Jul. 2018.
- [23] B. Wang, S. Feng, and J. Yang, “When to Preempt? Age of Information Minimization under Link Capacity Constraint,” arXiv e-prints, p. arXiv:1812.05670, Dec 2018.
- [24] ——, “To Skip or to Switch? Minimizing Age of Information under Link Capacity Constraint,” arXiv e-prints, p. arXiv:1806.08698, Jun 2018.
- [25] L. I. Sennott, “Average cost optimal stationary policies in infinite state markov decision processes with unbounded costs,” Operations Research, vol. 37, no. 4, pp. 626–633, 1989.