SelfTriggered TimeVarying Convex Optimization
Abstract
In this paper, we propose a selftriggered algorithm to solve a class of convex optimization problems with timevarying objective functions. It is known that the trajectory of the optimal solution can be asymptotically tracked by a continuoustime state update law. Unfortunately, implementing this requires continuous evaluation of the gradient and the inverse Hessian of the objective function which is not amenable to digital implementation. Alternatively, we draw inspiration from selftriggered control to propose a strategy that autonomously adapts the times at which it makes computations about the objective function, yielding a piecewise affine state update law. The algorithm does so by predicting the temporal evolution of the gradient using known upper bounds on higher order derivatives of the objective function. Our proposed method guarantees convergence to arbitrarily small neighborhood of the optimal trajectory in finite time and without incurring Zeno behavior. We illustrate our framework with numerical simulations.
Timevarying optimization, selftriggered control, adaptive step size
I Introduction
In this paper, we address a class of timevarying optimization problems where the goal is to asymptotically track a unique, timevarying optimal trajectory given by
(1) 
Problems of this form are generally referred to as timevarying optimization or parametric programming in the literature, and often arise in dynamical systems that involve an objective function or a set of constraints that have a dependence on time or a dynamic parameter, in general. Particular examples include fast model predictive control using online convex optimization [1], real time convex optimization in signal processing [2], distributed optimization of timevarying functions [3], timevarying pose estimation [4], traffic engineering in computer networks [5], neural network learning [6, 7], and dynamic density coverage for mobile robots [8].
From an optimization perspective, a general framework for solving problem (1) is to sample the objective function at particular times of interest, and solve the corresponding sequence of stationary optimization problems by standard iterative algorithms such as gradient or Newton’s methods. However, these algorithms clearly ignore the dynamic aspect of the problem which means they yield solutions with a final steadystate error whose magnitude is related to the timevarying aspects of the problem [9].
From a dynamical systems perspective, one could perform time sensitivity analysis of the optimal solution to propose a continuoustime dynamical system whose state is asymptotically driven to the optimal solution [7, 10]. The resulting dynamics is a combination of standard descent methods and a prediction term which tracks the drift in the optimal solution. For errorfree tracking, however, we need to solve the dynamics continuously, implying that we need continuous access to the objective function and all of its derivatives that appear in the continuoustime dynamics. A natural solution to this is to implement the continuoustime dynamics periodically. In a recent work [11], the authors proposed a periodic sampling strategy in which the objective function is periodically sampled with a constant period , and a single step of prediction along with multiple iterations of standard gradient or Newton’s algorithm are combined to achieve an asymptotic error bound that depends on and the number of descent steps taken between the sampling times.
Instead, we are interested in utilizing selftriggered control strategies [12, 13, 14, 15] to adaptively determine when samples of the objective function are needed without sacrificing the convergence; see [16] for a survey. From a dynamical systems perspective, this strategy plays a similar role as step size selection in stationary optimization, where a proper continuoustime dynamics for instance) is discretized aperiodically using a backtracking line search method [17]. In timevarying optimization, however, the line search method is no longer applicable as time and space become entangled. In this context, we can view our selftriggered sampling strategy as a way of adaptively choosing a proper step size in both time and space together. There are similar works that propose eventtriggered broadcasting strategies to solve static distributed optimization problem [18, 19, 20, 21], but to the knowledge of the authors, no work has been reported on an aperiodic discretization of continuous timevarying optimization problems.
Statement of contributions: In this work we are interested in developing a realtime algorithm that can asymptotically track the timevarying solution to a timevarying optimization problem. Our starting point is the availability of a continuoustime dynamics such that the solutions to this satisfy as . Then, we are interested in a realtime implementation such that is to be updated at discrete instants of time and is held constant between updates. In contrast to standard methods that consider periodic samples, our contribution is the development of a selftriggered control strategy that autonomously determines how often should be updated. Intuitively, the selftriggered strategy determines how long the current control input can be applied without negatively affecting the convergence. Our algorithm guarantees that the state can asymptotically track an arbitrarily small neighborhood around while ensuring Zeno behavior is avoided. Simulations illustrate our results.
Notation Let , , and be the set of real, nonnegative, and strictly positive real numbers. and denote nonnegative and positive integers, respectively. is the space of dimensional vectors and is the space of by symmetric matrices. The onenorm and twonorm of is denoted by and , respectively. The gradient of the function with respect to is denoted by . The partial derivatives of with respect to and are denoted by and , respectively. Higher order derivatives are also defined similarly.
Ii Preliminaries and Problem Statement
Let be a decision variable, a time index, and a realvalued convex function taking values . We interpret as a timevarying objective and consider the corresponding timevarying optimization problem in which we want to find the argument that minimizes the objective at time , Consider the function , and define the following optimization problem,
(2) 
We impose the following assumption on .
Assumption 1
The objective function is uniformly strongly convex in , i.e., satisfies for some , and for all .
By virtue of Assumption 1, is unique for each [17]. The optimal trajectory is then implicitly characterized by for all . Using the chain rule to differentiate this identity with respect to time yields
(3) 
Since the lefthand side of the above equation is identically zero for all , it follows that the optimal point obeys the dynamics
(4) 
The above dynamics suggests that the optimizer needs to follow the minimizer with the same dynamics, in addition to taking a descent direction in order to decrease the suboptimality. Choosing Newton’s method as a descent direction yields the following continuoustime dynamics,
(5) 
where the vector field is given by,
(6) 
Here is arbitrary. An appropriate Lyapunov function for (5) is
(7) 
which is zero along the optimal path, i.e., . It can be verified that under the continuoustime dynamics (5), the Lyapunov function evaluated at satisfies the ODE
(8) 
Solving the latter ODE yields the closed form solution , where is the initial point, and is the initial time assumed to be zero. This implies that exponential convergence of to requires continuous evaluation of the gradient and the inverse Hessian of the objective function, according to (5) and (6), which is computationally expensive and is not amenable to digital implementation. Instead, we can use a simple Euler method to discretize (5). More precisely, suppose we use a sequence of periodic sampling times with period , i.e., for any to arrive at the following piecewise affine state update law,
(9) 
where is the estimate of obtained from the ideal dynamics (5). Now if the vector field is uniformly Lipschitz in , i.e., for all , we have the inequality for some , and that the initial condition satisfies , the discretization error at time would satisfy the bound [23]
(10) 
The above inequality implies that as , i.e., we can control only the order of magnitude of the discretization error by . Moreover, this upper bound is very crude and is of no use for quantifying the suboptimality of [23]. Motivated by this observation, we are interested in a sampling strategy that autonomously adapts the times at which it makes computations about the objective function in order to control the suboptimality. We formalize the problem next.
Problem 1
Given the dynamics (9), find a strategy that determines the least frequent sequence of sampling times such that:

for each , is determined without having access to the objective function for ,

converges to any neighborhood of the optimal trajectory after a finite number of samples, and remains there forever, and

for some and all .
The first property guarantees that the proposed method is completely online. The second property enables the optimizer to arbitrarily bound the discretization error. The last property ensures Zeno behavior is avoided. In order to develop the main results, we make the following Assumption about the objective function.
Assumption 2
The objective function is thrice continuously differentiable. Furthermore,

The secondorder derivatives are bounded by known positive constants, i.e.,

The thirdorder derivatives are bounded by known positive constants, i.e.,
where .
The first bound in Assumption 2(i) and the last bound in Assumption 2(ii) are equivalent to uniform Lipschitz continuity of the gradient and the Hessian in , respectively. Both assumptions are standard assumptions in secondorder methods [17]. All other bounds are related to the timevarying aspect of the objective function and bound the rate at which the gradient and Hessian functions vary with time. Notice that except for the bound , all the other bounds are required for to be uniformly Lipschitz [11].
Iii SelfTriggered Strategy
In this section, we design a selftriggered sampling strategy that meets the desired specifications defined in Problem 1. Consider the discrete implementation of the ideal dynamics (5) at a sequence of times that is to be determined,
(11) 
Recalling the Lyapunov function (7), the instantaneous derivatives of at the discrete sampling times are precisely
(12) 
In other words, the property (8) that holds at all times in the continuoustime framework is now only preserved at discrete sampling times. This means in general there is no guarantee that remains negative between sampling times , as the optimizer is no longer updating its dynamics during this time interval. Given these observations, we need to predict, without having access to the objective function or its derivatives for , the earliest time after at which the Lyapunov function could possibly increase, and update the state dynamics at that time, denoted by . Consequently, we desire a tight upper bound on so that we are taking samples as conservatively as possible. Mathematically speaking, for each , we can characterize the upper bound as follows,
(13) 
where is the class of all strongly convex objective functions such that

,

,

,

satisfies Assumption 2.
In words, is the set of all possible objective functions that agree with and it’s first and secondorder derivatives at , and satisfy the bounds in Assumption 2. Intuitively, the set formalizes, in a functional way, the fact that we find without having access to the objective function for . The above definition implies that . In particular, we have that by (13) and (12). Once is characterized at time as a function of , the next sampling time is set as the first time instant at which crosses zero, i.e.,
(14) 
where is the inverse of the map . This choice ensures that for . With this policy, the evaluated Lyapunov function becomes a piecewise continuously differentiable monotonically decreasing function of with discontinuous derivatives at the sampling times. We can view as a triggering function which triggers the optimizer to sample when the event occurs for some . This concept is illustrated in Figure 1. In the next Subsection, we characterize in closedform.
Iiia Triggering Function
Lemma 1 (Secondorder selftriggered strategy)
Proof.
See Appendix AA. ∎
A careful investigation into the Proof of Lemma 1 reveals the fact that can still be characterized without having access to the bounds in Assumption(i), i.e., we assume known bounds on the thirdorder derivatives only. By virtue of this relaxation, our strategy prescribes less conservative sampling times. We will see in the next lemma that the triggering function, in this case, is a thirdorder polynomial.
Lemma 2 (Thirdorder selftriggered strategy)
Proof.
See Appendix AB. ∎
Remark 1
It can be observed from (1) and (2) that is fully characterized at time without having access to the objective function for . In this context, the selftriggered strategy is online, implying the property (i) in Problem 1. Moreover, has a unique root on the interval when , implying that the sampling time is welldefined and the step size satisfies for all .
IiiB Asymptotic Convergence
The triggering functions developed in the previous lemmas have the following properties by construction:

is convex in and strictly increasing on .

on .

.

.
We establish in the next theorem that the above properties are enough to secure asymptotic monotone convergence of the Lyapunov function to zero.
Theorem 1
Proof.
See Appendix AC. ∎
Remark 2 (Role of )
In the proof of Theorem 1, we showed that the Lyapunov function at the sampling times satisfies the inequality
Combining this inequality with the trivial inequality lets us conclude that for all , the step sizes are bounded as . Therefore, increasing will reduce the step sizes such that the effective step size is bounded by one. This observation is consistent with backtracking line search method in stationary optimization in which the step sizes are bounded by one.
We have the following corollary as an immediate consequence of Theorem 1.
Corollary 1
Next, we discuss the implementation issues of the selftriggered strategy proposed above.
IiiC Implementation
It can be seen from Theorem 1 and the expression of in (1) or (2) that as , , and therefore , i.e., the step sizes vanish asymptotically. This might cause Zeno behavior, i.e., the possibility for infinitely many samples over a finite interval of time. To avoid this possibility, we need to modify the algorithm to ensure that the step sizes are lower bounded by a positive constant all the time; a stronger property than no Zeno behavior. For this purpose, we implement the algorithm in two phases: In the first phase, we use the sampling strategy developed in Subsection IIIA until the state reaches within a prespecified neighborhood around . In the second phase, we switch the triggering strategy so as to merely maintain in that neighborhood forever. More specifically, for the sequence of sampling times and any , define
In words, is the first sampling time at which the Lyapunov function is below the threshold . By Corollary 1, is finite. Now for , we propose another selftriggered sampling strategy such that the Lyapunov function satisfies for all . Recalling the inequality , we can obtain an upper bound for as follows,
(18) 
The righthand side is a polynomial in which can be fully characterized at . Now for , we set the next sampling time as the first time instant after at which the upper bound function in the righthand side crosses , i.e., we select according to the following rule,
(19) 
where the new triggering function is defined as
(20) 
This policy guarantees that for all . As a result, by virtue of strong convexity [17], i.e., the inequality , and recalling (7), the following bound
(21) 
will hold for all .
Remark 3
According to (19), the sampling time is obtained by solving the equation for . By (20), it can be verified that for . In the limiting case , there are two solutions to ; the first one is which implies zero step size and is ignored; the other one is strictly greater that , and is selected as the next sampling time.
We summarize the proposed implementation in Table 1, where we use the notation and .
Secondorder selftriggered strategy
Given:
Thirdorder selftriggered strategy
Given:
We close this section by the following theorem which accomplishes the main goals defined in Problem 1.
Theorem 2
Let be the sequence of sampling times generated by Algorithm 1. Then, for any , there exists a nonnegative integer such that: (i) for all ; and (ii) for all and some .
Iv Simulation
In this section, we perform numerical experiments to illustrate our results. For simplicity in our exposition, we consider the following convex problem in onedimensional space
where , , , , and . For these numerical values, we have that , , and . We solve this problem for the time interval via Algorithm 1 using the triggering function (2), and setting and . The total number of updates are , with the step sizes having a mean value of and standard deviation . For comparison, we also solve the optimization problem by a more standard periodic implementation. We plot all the solutions in Figure 2 along with the of the total number of samples required in each execution. It can be observed that small sampling periods, e.g., , yield a convergence performance similar to the selftriggered strategy, but uses a far higher number of updates. On the other hand, larger sampling periods, e.g., , result in comparable number of samples as the selftriggering strategy at the expense of slower convergence. It should also be noted that we do not know a priori what sampling period yields good convergence results with a reasonable number of requires samples; however, the selftriggered strategy is capable of automatically tuning the step sizes to yield good performance while utilizing a much smaller number of samples.
Effect of : Next, we study the effect of the design parameter on the number of samples and the convergence performance of the selftriggered strategy. More specifically, we run Algorithm 1 with all the parameters as before, and with different values of . Figure 3 shows the resulting trajectories for various values of . It is observed that does not change neither the transient convergence phase, but rather affects the steady state tracking phase. Moreover, the number of samples are almost unaffected by changing .
Effect of : Finally, we study the performance of the selftriggered strategy as changes. Intuitively, higher values of puts more weight on the descent part of the dynamics () than the tracking part (), according to (5). Hence, we expect that we get more rapid convergence to the neighborhood of the optimal trajectory by increasing . Figure 4 illustrates the resulting trajectories for different values of . As expected, as we increase , the trajectory converges faster to the optimal trajectory. The number of samples, however, are not affected by . This observation is in agreement with Remark 2, where we showed that the effective step sizes are between zero and one, independent of . In the limiting case , the step sizes get arbitrarily small which is not desirable.
V Conclusion
In this paper, we proposed a realtime selftriggered strategy to aperiodically implement a continuoustime dynamics that solves continuously timevarying convex optimization problems. The sampling times are autonomously chosen by the algorithm to ensure asymptotic convergence to the optimal solution while keeping the number of updates at the minimum. We illustrated the effectiveness of the proposed method with numerical simulations. In future work, we will consider the case where the term in (6) is not available, and needs to be estimated with backward difference in time.
Appendix A Appendix
Aa Proof of Lemma 1
We begin by fixing and analyzing the Lyapunov function during the interevent time . We aim to find a tight upper bound on . First, we write in integral form as
(22) 
Applying Jensen’s gives us the inequality
(23) 
The main idea is then to bound on . By adopting the notation , we can rewrite the Lyapunov function as
(24) 
By (9), we have that for , and therefore, for . Whence, reads as
(25) 
We can write the first two time derivatives of from (24) as follows,
(26) 
In order to bound , we proceed to bound , , and , using the known upper bounds granted by Assumption 2. We use chain rule to derive from (25) as follows,
(27) 
We now use Assumption 2(i) to find an upper bound on as follows,
(28) 
where we have used the definition of in (1). We apply the chain rule again on (27) to get
(29) 
We use Assumption (2)(ii) to bound . The first term in can be bounded as follows,
The second and third term in can also be bounded as follows,
Putting the last two bounds together, we obtain
(30) 
where we have used the definition of in (1). To bound , we use Taylor’s theorem to expand as
for some . Using the bound from (AA), we can bound the last result as follows,
(31) 
where by (24), . We substitute all the upper bounds in (AA), (AA), and (AA) back in to write
(32)  
Plugging the bound (32) back in (23), evaluating the integral, and using the definition of in (1) would let us to conclude that
(33) 
The proof is complete.
AB Proof of Lemma 2
We follow the same logic as the proof of Lemma 1 to find a tight upper bound for by bounding . Notice that if we do not know the bounds in Assumption 2(i), the bound in (AA) still holds, whereas the bounds for in (AA) and in (AA) break. To find legitimate bounds for the latter functions, we use Taylor’s theorem to express as follows,
(34) 
for some . By (AA) we know that for . Hence, we can bound as
We use Taylor’s theorem one more time to express as
(35) 
for some . Therefore, we can bound as follows,