Self-Triggered Time-Varying Convex Optimization
In this paper, we propose a self-triggered algorithm to solve a class of convex optimization problems with time-varying objective functions. It is known that the trajectory of the optimal solution can be asymptotically tracked by a continuous-time state update law. Unfortunately, implementing this requires continuous evaluation of the gradient and the inverse Hessian of the objective function which is not amenable to digital implementation. Alternatively, we draw inspiration from self-triggered control to propose a strategy that autonomously adapts the times at which it makes computations about the objective function, yielding a piece-wise affine state update law. The algorithm does so by predicting the temporal evolution of the gradient using known upper bounds on higher order derivatives of the objective function. Our proposed method guarantees convergence to arbitrarily small neighborhood of the optimal trajectory in finite time and without incurring Zeno behavior. We illustrate our framework with numerical simulations.
Time-varying optimization, self-triggered control, adaptive step size
In this paper, we address a class of time-varying optimization problems where the goal is to asymptotically track a unique, time-varying optimal trajectory given by
Problems of this form are generally referred to as time-varying optimization or parametric programming in the literature, and often arise in dynamical systems that involve an objective function or a set of constraints that have a dependence on time or a dynamic parameter, in general. Particular examples include fast model predictive control using online convex optimization , real time convex optimization in signal processing , distributed optimization of time-varying functions , time-varying pose estimation , traffic engineering in computer networks , neural network learning [6, 7], and dynamic density coverage for mobile robots .
From an optimization perspective, a general framework for solving problem (1) is to sample the objective function at particular times of interest, and solve the corresponding sequence of stationary optimization problems by standard iterative algorithms such as gradient or Newton’s methods. However, these algorithms clearly ignore the dynamic aspect of the problem which means they yield solutions with a final steady-state error whose magnitude is related to the time-varying aspects of the problem .
From a dynamical systems perspective, one could perform time sensitivity analysis of the optimal solution to propose a continuous-time dynamical system whose state is asymptotically driven to the optimal solution [7, 10]. The resulting dynamics is a combination of standard descent methods and a prediction term which tracks the drift in the optimal solution. For error-free tracking, however, we need to solve the dynamics continuously, implying that we need continuous access to the objective function and all of its derivatives that appear in the continuous-time dynamics. A natural solution to this is to implement the continuous-time dynamics periodically. In a recent work , the authors proposed a periodic sampling strategy in which the objective function is periodically sampled with a constant period , and a single step of prediction along with multiple iterations of standard gradient or Newton’s algorithm are combined to achieve an asymptotic error bound that depends on and the number of descent steps taken between the sampling times.
Instead, we are interested in utilizing self-triggered control strategies [12, 13, 14, 15] to adaptively determine when samples of the objective function are needed without sacrificing the convergence; see  for a survey. From a dynamical systems perspective, this strategy plays a similar role as step size selection in stationary optimization, where a proper continuous-time dynamics for instance) is discretized aperiodically using a backtracking line search method . In time-varying optimization, however, the line search method is no longer applicable as time and space become entangled. In this context, we can view our self-triggered sampling strategy as a way of adaptively choosing a proper step size in both time and space together. There are similar works that propose event-triggered broadcasting strategies to solve static distributed optimization problem [18, 19, 20, 21], but to the knowledge of the authors, no work has been reported on an aperiodic discretization of continuous time-varying optimization problems.
Statement of contributions: In this work we are interested in developing a real-time algorithm that can asymptotically track the time-varying solution to a time-varying optimization problem. Our starting point is the availability of a continuous-time dynamics such that the solutions to this satisfy as . Then, we are interested in a real-time implementation such that is to be updated at discrete instants of time and is held constant between updates. In contrast to standard methods that consider periodic samples, our contribution is the development of a self-triggered control strategy that autonomously determines how often should be updated. Intuitively, the self-triggered strategy determines how long the current control input can be applied without negatively affecting the convergence. Our algorithm guarantees that the state can asymptotically track an arbitrarily small neighborhood around while ensuring Zeno behavior is avoided. Simulations illustrate our results.
Notation Let , , and be the set of real, nonnegative, and strictly positive real numbers. and denote nonnegative and positive integers, respectively. is the space of -dimensional vectors and is the space of by symmetric matrices. The one-norm and two-norm of is denoted by and , respectively. The gradient of the function with respect to is denoted by . The partial derivatives of with respect to and are denoted by and , respectively. Higher order derivatives are also defined similarly.
Ii Preliminaries and Problem Statement
Let be a decision variable, a time index, and a real-valued convex function taking values . We interpret as a time-varying objective and consider the corresponding time-varying optimization problem in which we want to find the argument that minimizes the objective at time , Consider the function , and define the following optimization problem,
We impose the following assumption on .
The objective function is uniformly strongly convex in , i.e., satisfies for some , and for all .
By virtue of Assumption 1, is unique for each . The optimal trajectory is then implicitly characterized by for all . Using the chain rule to differentiate this identity with respect to time yields
Since the left-hand side of the above equation is identically zero for all , it follows that the optimal point obeys the dynamics
The above dynamics suggests that the optimizer needs to follow the minimizer with the same dynamics, in addition to taking a descent direction in order to decrease the suboptimality. Choosing Newton’s method as a descent direction yields the following continuous-time dynamics,
where the vector field is given by,
Here is arbitrary. An appropriate Lyapunov function for (5) is
which is zero along the optimal path, i.e., . It can be verified that under the continuous-time dynamics (5), the Lyapunov function evaluated at satisfies the ODE
Solving the latter ODE yields the closed form solution , where is the initial point, and is the initial time assumed to be zero. This implies that exponential convergence of to requires continuous evaluation of the gradient and the inverse Hessian of the objective function, according to (5) and (6), which is computationally expensive and is not amenable to digital implementation. Instead, we can use a simple Euler method to discretize (5). More precisely, suppose we use a sequence of periodic sampling times with period , i.e., for any to arrive at the following piece-wise affine state update law,
where is the estimate of obtained from the ideal dynamics (5). Now if the vector field is uniformly Lipschitz in , i.e., for all , we have the inequality for some , and that the initial condition satisfies , the discretization error at time would satisfy the bound 
The above inequality implies that as , i.e., we can control only the order of magnitude of the discretization error by . Moreover, this upper bound is very crude and is of no use for quantifying the suboptimality of . Motivated by this observation, we are interested in a sampling strategy that autonomously adapts the times at which it makes computations about the objective function in order to control the suboptimality. We formalize the problem next.
Given the dynamics (9), find a strategy that determines the least frequent sequence of sampling times such that:
for each , is determined without having access to the objective function for ,
converges to any neighborhood of the optimal trajectory after a finite number of samples, and remains there forever, and
for some and all .
The first property guarantees that the proposed method is completely online. The second property enables the optimizer to arbitrarily bound the discretization error. The last property ensures Zeno behavior is avoided. In order to develop the main results, we make the following Assumption about the objective function.
The objective function is thrice continuously differentiable. Furthermore,
The second-order derivatives are bounded by known positive constants, i.e.,
The third-order derivatives are bounded by known positive constants, i.e.,
The first bound in Assumption 2-(i) and the last bound in Assumption 2-(ii) are equivalent to uniform Lipschitz continuity of the gradient and the Hessian in , respectively. Both assumptions are standard assumptions in second-order methods . All other bounds are related to the time-varying aspect of the objective function and bound the rate at which the gradient and Hessian functions vary with time. Notice that except for the bound , all the other bounds are required for to be uniformly Lipschitz .
Iii Self-Triggered Strategy
In this section, we design a self-triggered sampling strategy that meets the desired specifications defined in Problem 1. Consider the discrete implementation of the ideal dynamics (5) at a sequence of times that is to be determined,
Recalling the Lyapunov function (7), the instantaneous derivatives of at the discrete sampling times are precisely
In other words, the property (8) that holds at all times in the continuous-time framework is now only preserved at discrete sampling times. This means in general there is no guarantee that remains negative between sampling times , as the optimizer is no longer updating its dynamics during this time interval. Given these observations, we need to predict, without having access to the objective function or its derivatives for , the earliest time after at which the Lyapunov function could possibly increase, and update the state dynamics at that time, denoted by . Consequently, we desire a tight upper bound on so that we are taking samples as conservatively as possible. Mathematically speaking, for each , we can characterize the upper bound as follows,
where is the class of all strongly convex objective functions such that
satisfies Assumption 2.
In words, is the set of all possible objective functions that agree with and it’s first and second-order derivatives at , and satisfy the bounds in Assumption 2. Intuitively, the set formalizes, in a functional way, the fact that we find without having access to the objective function for . The above definition implies that . In particular, we have that by (13) and (12). Once is characterized at time as a function of , the next sampling time is set as the first time instant at which crosses zero, i.e.,
where is the inverse of the map . This choice ensures that for . With this policy, the evaluated Lyapunov function becomes a piece-wise continuously differentiable monotonically decreasing function of with discontinuous derivatives at the sampling times. We can view as a triggering function which triggers the optimizer to sample when the event occurs for some . This concept is illustrated in Figure 1. In the next Subsection, we characterize in closed-form.
Iii-a Triggering Function
Lemma 1 (Second-order self-triggered strategy)
See Appendix A-A. ∎
A careful investigation into the Proof of Lemma 1 reveals the fact that can still be characterized without having access to the bounds in Assumption-(i), i.e., we assume known bounds on the third-order derivatives only. By virtue of this relaxation, our strategy prescribes less conservative sampling times. We will see in the next lemma that the triggering function, in this case, is a third-order polynomial.
Lemma 2 (Third-order self-triggered strategy)
See Appendix A-B. ∎
It can be observed from (1) and (2) that is fully characterized at time without having access to the objective function for . In this context, the self-triggered strategy is online, implying the property (i) in Problem 1. Moreover, has a unique root on the interval when , implying that the sampling time is well-defined and the step size satisfies for all .
Iii-B Asymptotic Convergence
The triggering functions developed in the previous lemmas have the following properties by construction:
is convex in and strictly increasing on .
We establish in the next theorem that the above properties are enough to secure asymptotic monotone convergence of the Lyapunov function to zero.
See Appendix A-C. ∎
Remark 2 (Role of )
In the proof of Theorem 1, we showed that the Lyapunov function at the sampling times satisfies the inequality
Combining this inequality with the trivial inequality lets us conclude that for all , the step sizes are bounded as . Therefore, increasing will reduce the step sizes such that the effective step size is bounded by one. This observation is consistent with backtracking line search method in stationary optimization in which the step sizes are bounded by one.
We have the following corollary as an immediate consequence of Theorem 1.
Next, we discuss the implementation issues of the self-triggered strategy proposed above.
It can be seen from Theorem 1 and the expression of in (1) or (2) that as , , and therefore , i.e., the step sizes vanish asymptotically. This might cause Zeno behavior, i.e., the possibility for infinitely many samples over a finite interval of time. To avoid this possibility, we need to modify the algorithm to ensure that the step sizes are lower bounded by a positive constant all the time; a stronger property than no Zeno behavior. For this purpose, we implement the algorithm in two phases: In the first phase, we use the sampling strategy developed in Subsection III-A until the state reaches within a pre-specified neighborhood around . In the second phase, we switch the triggering strategy so as to merely maintain in that neighborhood forever. More specifically, for the sequence of sampling times and any , define
In words, is the first sampling time at which the Lyapunov function is below the threshold . By Corollary 1, is finite. Now for , we propose another self-triggered sampling strategy such that the Lyapunov function satisfies for all . Recalling the inequality , we can obtain an upper bound for as follows,
The right-hand side is a polynomial in which can be fully characterized at . Now for , we set the next sampling time as the first time instant after at which the upper bound function in the right-hand side crosses , i.e., we select according to the following rule,
where the new triggering function is defined as
will hold for all .
According to (19), the sampling time is obtained by solving the equation for . By (20), it can be verified that for . In the limiting case , there are two solutions to ; the first one is which implies zero step size and is ignored; the other one is strictly greater that , and is selected as the next sampling time.
We summarize the proposed implementation in Table 1, where we use the notation and .
Second-order self-triggered strategy
Third-order self-triggered strategy
We close this section by the following theorem which accomplishes the main goals defined in Problem 1.
Let be the sequence of sampling times generated by Algorithm 1. Then, for any , there exists a nonnegative integer such that: (i) for all ; and (ii) for all and some .
In this section, we perform numerical experiments to illustrate our results. For simplicity in our exposition, we consider the following convex problem in one-dimensional space
where , , , , and . For these numerical values, we have that , , and . We solve this problem for the time interval via Algorithm 1 using the triggering function (2), and setting and . The total number of updates are , with the step sizes having a mean value of and standard deviation . For comparison, we also solve the optimization problem by a more standard periodic implementation. We plot all the solutions in Figure 2 along with the of the total number of samples required in each execution. It can be observed that small sampling periods, e.g., , yield a convergence performance similar to the self-triggered strategy, but uses a far higher number of updates. On the other hand, larger sampling periods, e.g., , result in comparable number of samples as the self-triggering strategy at the expense of slower convergence. It should also be noted that we do not know a priori what sampling period yields good convergence results with a reasonable number of requires samples; however, the self-triggered strategy is capable of automatically tuning the step sizes to yield good performance while utilizing a much smaller number of samples.
Effect of : Next, we study the effect of the design parameter on the number of samples and the convergence performance of the self-triggered strategy. More specifically, we run Algorithm 1 with all the parameters as before, and with different values of . Figure 3 shows the resulting trajectories for various values of . It is observed that does not change neither the transient convergence phase, but rather affects the steady state tracking phase. Moreover, the number of samples are almost unaffected by changing .
Effect of : Finally, we study the performance of the self-triggered strategy as changes. Intuitively, higher values of puts more weight on the descent part of the dynamics () than the tracking part (), according to (5). Hence, we expect that we get more rapid convergence to the -neighborhood of the optimal trajectory by increasing . Figure 4 illustrates the resulting trajectories for different values of . As expected, as we increase , the trajectory converges faster to the optimal trajectory. The number of samples, however, are not affected by . This observation is in agreement with Remark 2, where we showed that the effective step sizes are between zero and one, independent of . In the limiting case , the step sizes get arbitrarily small which is not desirable.
In this paper, we proposed a real-time self-triggered strategy to aperiodically implement a continuous-time dynamics that solves continuously time-varying convex optimization problems. The sampling times are autonomously chosen by the algorithm to ensure asymptotic convergence to the optimal solution while keeping the number of updates at the minimum. We illustrated the effectiveness of the proposed method with numerical simulations. In future work, we will consider the case where the term in (6) is not available, and needs to be estimated with backward difference in time.
Appendix A Appendix
A-a Proof of Lemma 1
We begin by fixing and analyzing the Lyapunov function during the inter-event time . We aim to find a tight upper bound on . First, we write in integral form as
Applying Jensen’s gives us the inequality
The main idea is then to bound on . By adopting the notation , we can rewrite the Lyapunov function as
By (9), we have that for , and therefore, for . Whence, reads as
We can write the first two time derivatives of from (24) as follows,
We now use Assumption 2-(i) to find an upper bound on as follows,
We use Assumption (2)-(ii) to bound . The first term in can be bounded as follows,
The second and third term in can also be bounded as follows,
Putting the last two bounds together, we obtain
where we have used the definition of in (1). To bound , we use Taylor’s theorem to expand as
for some . Using the bound from (A-A), we can bound the last result as follows,
The proof is complete.
A-B Proof of Lemma 2
We follow the same logic as the proof of Lemma 1 to find a tight upper bound for by bounding . Notice that if we do not know the bounds in Assumption 2-(i), the bound in (A-A) still holds, whereas the bounds for in (A-A) and in (A-A) break. To find legitimate bounds for the latter functions, we use Taylor’s theorem to express as follows,
for some . By (A-A) we know that for . Hence, we can bound as
We use Taylor’s theorem one more time to express as
for some . Therefore, we can bound as follows,