# Optimal Stochastic Strongly Convex Optimization with a Logarithmic Number of Projections

## Abstract

We consider stochastic strongly convex optimization with a complex inequality constraint. This complex inequality constraint may lead to computationally expensive projections in algorithmic iterations of the stochastic gradient descent (SGD) methods. To reduce the computation costs pertaining to the projections, we propose an Epoch-Projection Stochastic Gradient Descent (Epro-SGD) method. The proposed Epro-SGD method consists of a sequence of epochs; it applies SGD to an augmented objective function at each iteration within the epoch, and then performs a projection at the end of each epoch. Given a strongly convex optimization and for a total number of iterations, Epro-SGD requires only projections, and meanwhile attains an optimal convergence rate of , both in expectation and with a high probability. To exploit the structure of the optimization problem, we propose a proximal variant of Epro-SGD, namely Epro-ORDA, based on the optimal regularized dual averaging method. We apply the proposed methods on real-world applications; the empirical results demonstrate the effectiveness of our methods.

## 1Introduction

Recent years have witnessed an increased interest in adopting the stochastic (sub)gradient (SGD) methods [1] for solving large-scale machine learning problems. In each of the algorithmic iterations, SGD reduces the computation cost by sampling one (or a small number of) example for computing a stochastic (sub)gradient. Thus the computation cost in SGD is independent of the size of the data available for training; this property makes SGD appealing for large-scale optimization. However, when the optimization problems involve a complex domain (for example a positive definite constraint or a polyhedron one), the projection operation in each iteration of SGD, which is used to ensure the feasibility of the intermediate solutions, may become the computational bottleneck.

In this paper we consider to solve the following constrained optimization problem

where is -strongly convex [23] and is convex. We assume a stochastic access model for , in which the only access to is via a stochastic gradient oracle; in other words, given arbitrary , this stochastic gradient oracle produces a random vector , whose expectation is a subgradient of at the point , i.e., , where denotes the subdifferential set of at . On the other hand we have the full access to the (sub)gradient of .

The standard SGD method [5] solves Eq. (Equation 1) by iterating the updates in Eq. (Equation 2) with an appropriate step size , e.g., ), as below

and then returning as the final solution for a total number of iterations . Note that is a projection operator defined as

If the involved constraint function is complex (e.g., a polyhedral or a positive definite constraint), computing the associated projection may be computationally expensive; for example, a projection onto a positive definite cone over requires a full singular value decomposition (SVD) operation with time complexity of .

In this paper, we propose an epoch-based SGD method, called Epro-SGD, which requires only a logarithmic number of projections (onto the feasible set), and meanwhile achieves an optimal convergence rate for stochastic strongly convex optimization. Specifically, the proposed Epro-SGD method consists of a sequence of epochs; within each of the epochs, the standard SGD is applied to optimize a composite objective function augmented by the complex constraint function, hence avoiding the expensive projections steps; at the end of every epoch, a projection operation is performed to ensure the feasibility of the intermediate solution. Our analysis shows that given a strongly convex optimization and for a total number of iterations, Epro-SGD requires only projections, and meanwhile achieves an optimal rate of convergence at , both in expectation and with a high probability.

To exploit the structure (for example the sparisty) of the optimization problem, we propose a proximal variant of the Epro-SGD method, namely Epro-ORDA, which utilizes an existing optimal dual averaging method to solve the involved proximal mapping. Our analysis shows that Epro-ORDA similarly requires only a logarithmic number of projections while enjoys an optimal rate of convergence.

For illustration we apply the proposed Epro-SGD methods on two real-world applications, i.e., the constrained Lasso formulation and the large margin nearest neighbor (LMNN) classification. Our experimental results demonstrate the efficiency of the proposed methods, in comparison to the existing methods.

## 2Related Work

The present work is inspired from the break-through work in [20], which proposed two novel one-projection-based stochastic gradient descent (OneProj) methods for stochastic convex optimizations. Specifically the first OneProj method was developed for general convex optimization; it introduces a regularized Lagrangian function as

then applies SGD to the convex-concave problem , and finally performs only one projection at the end of all iterations, where is a bounded ball subsuming as a subset.

The second OneProj method was developed for strongly convex optimization. The proposed method introduced an augmented objective function

where is a parameter dependent on the total number of iterations , and is a problem specific parameter [20]. OneProj applies SGD to the augmented objective function, specifically using a stochastic subgradient of and a subgradient of , and then performs a projection step after all iterations. For a total number iterations, the OneProj method achieves a rate of convergence at , which is suboptimal for stochastic strongly convex optimization.

Several recent works [15] propose optimal methods with optimal rates of convergence at for stochastic strongly convex optimization. In particular, the Epoch-SGD method [15] consists of a sequence of epochs, each of which has a geometrically decreasing step size and a geometrically increasing iteration number. This method however needs to project the intermediate solutions onto a feasible set at every algorithmic iteration; when the involved constraint is complex, the involved projection is usually computationally expensive. This limitation restricts the practical applications on large scale data analysis. Therefore we are motivated to develop an optimal stochastic algorithm for strongly convex optimization but with a constant number of projections.

Another closely related work is the logT-SGD [33] for stochastic strongly convex and smooth optimization. LogT-SGD achieves an optimal rate of convergence, while requires to perform projections, where is the ratio of the smoothness parameter to the strong convexity parameter. There are several key differences between our proposed Epro-SGD method and logT-SGD: (i) logT-SGD and its analysis rely on both the smoothness and the strong convexity of the objective function; in contrast, Epro-SGD only assumes that the objective function is strongly convex; (ii) the number of the required projections in logT-SGD is , where the conditional number can be very large in real applications; in contrast, Epro-SGD requires at most projections.

Besides reducing the number of projections in SGD, another line of research is based on the conditional gradient algorithms [7]; this type of algorithms mostly build upon the Frank-Wolfe technique [11], which eschews the projection in favor of a linear optimization step; however in general, they require the smoothness assumption in the objective function. On the other hand, [12] extended Frank-Wolfe techniques to stochastic or online setting for general and strongly convex optimizations. Specifically [16] presents an online/stochastic Frank-Wolfe (OFW) algorithm with a convergence rate for general convex optimization problems, which is slower than the optimal rate . [12] presents an algorithm for online strongly convex optimization with an regret bound, implying an convergence rate for stochastic stronlgy convex optimization. This algorithm requires the problem domain to be a polytope, instead of a convex inequality constraint used in this paper; it also hinges on an efficient local linear optimization oracle that amounts to approximately solving a linear optimization problem over an intersection of a ball and and the feasible domain; furthermore the convergence result only holds in expectation and is sub-optimal.

## 3Epoch-Projection Sgd Algorithm

In this section, we present an epoch-projection SGD method, called Epro-SGD, for solving Eq. (Equation 1) and discuss its convergence result. Based on a stochastic dual averaging algorithm, we then present a proximal variant of the proposed Epro-SGD method.

### 3.1Setup and Background

Denote the optimal solution to Eq. (Equation 1) by and its domain set by . Since is -strongly convex [23] and is convex, the optimization problem in Eq. (Equation 1) is strongly convex. Note that the strong convexity in implies that for any . Our analysis is based on the following assumptions:

The stochastic subgradient is uniformly bounded by , i.e., .

The subgradient is uniformly bounded by , i.e., for any .

There exists a positive value such that

**Remarks** Assumptions A1 and A2 respectively impose an upper bound on the stochastic subgradient of the objective function and the constraint function . Assumption A3 ensures that the projection of a point onto a feasible domain does not deviate too much from this intermediate point. Note that Assumption A1 is previously used in [15]; a condition similar to Assumption A3 is used in [20], which however simply assumes that , without considering possible non-differentiability in .

A key consequence of Assumption A3 is presented in the following lemma.

If , we have ; the inequality in Eq. ( ?) trivially holds. If , we can verify that , and there exists and such that (using duality theory). It follows that (), and thus is the same direction as . It follows that

where the last inequality uses Assumption A3. This completes the proof of this lemma.

The result in Lemma ? is closely related to *the polyhedral error bound condition* [13]; this condition shows that the distance of a point to the optimal set of a convex optimization problem whose epigraph is a polyhedron is bounded by the distance of the objective value at this point to the optimal objective value scaled by a constant. For illustration, we consider the optimization problem

with an optimal set as . If , is the closest point in the optimal set to . Therefore, by *the polyhedral error bound condition* of a polyhedral convex optimization, if is a polyhedral function, there exists a such that

Below we present three examples in which Assumption A3 or Lemma ? is satisfied. Example : an affine function with . Example : the norm constraint where . Example : the maximum of a finite number of affine functions satisfying Lemma ? as well as the polyhedral error bound condition [31].

### 3.2Main Algorithm

To solve Eq. (Equation 1) (using Epro-SGD), we introduce an augmented objective function by incorporating the constraint function as

It is worth noting that the augmented function in Eq. (Equation 4) does not have any iteration-dependent parameter, for example the parameter in Eq. (Equation 3). is a prescribed parameter satisfying , as illustrated in Lemma ?.

The details of our proposed Epro-SGD algorithm is presented in Algorithm ?. Similar to Epoch-SGD [15], Epro-SGD consists of a sequence of epochs, each of which has a geometrically decreasing step size and a geometrically increasing iteration number (Line in Algorithm ?). The updates in every intra-epoch (Line - ) are standard SGD steps applied to the augmented objective function with . Epro-SGD is different from Epoch-SGD in that the former computes a projection only at the end of each epoch, while the latter computes a projection at each iteration. Consequently, when the projection step is computationally expensive (e.g., projecting onto a positive definite constraint), Epro-SGD may require much less computation time than Epoch-SGD.

In Lemma ?, we present an important convergence analysis for the intra-epoch steps of Algorithm ?, which are key building blocks for deriving the main results in Theorem ?.

Let and denote by the expectation conditioned on the randomness until round . It is easy to verify that , and . For any , we have

where . Furthermore by the convexity of , we have

Noting that , taking expectation over randomness and summation over , we have

Let . Since , we have

It follows that

If , we have . Following from and , we can verify that and also holds.

Next we show that holds when . From Lemma ?, we have

Moreover it follows from and that the following inequality holds

Substituting Eqs. (Equation 8) and (Equation 9) into Eq. (Equation 7), we have

By some rearrangement, we have . Furthermore we have

where the second inequality follows from , and for any . This completes the proof of the lemma.

We present a main convergence result of the Epro-SGD algorithm in the following theorem.

From the updating rule , we can easily verify Eq. ( ?). Since , the inequality trivially holds.

Let . It follows that and . Next we show the inequality

holds by induction. Note that Eq. (Equation 10) implies , due to . Let . It follows from Lemma ?, , and , the inequality in Eq. (Equation 10) holds when . Assuming that Eq. (Equation 10) holds for , we show that Eq. (Equation 10) holds for .

For a random variable measurable with respect to the randomness up to epoch . Let denote the expectation conditioned on all the randomness up to epoch . Following Lemma ?, we have

Since by the strong convexity in , we have

which completes the proof of this theorem.

Remark

We compare the obtained main results in Theorem ? with several existing works. Firstly Eq. ( ?) implies that Epro-SGD achieves an optimal bound , matching the lower bound for a strongly convex problem [15]. Secondly in contrast to the OneProj method [20] with a convergence rate , Epro-SGD uses no more than projections to obtain an convergence rate. Epro-SGD thus has better control over the solution for not deviating (too much) from the feasibility domain in the intermediate iterations. Thirdly compared to Epoch-SGD with its convergence rate bounded by , the convergence rate bound of Epro-SGD is only worse by a factor of constant . Particularly consider a positive definite constraint with , , and , we have and the bound of Epro-SGD is only worse by a factor of than Epoch-SGD. Finally compared to the logT-SGD algorithm [33] which requires projections ( is the conditional number), the number of projections in Epro-SGD is independent of the conditional number.

The main results in Lemma ? and Theorem ? are expected convergence bounds. In Theorem ? (proof provided in Appendix) we show that Epro-SGD also enjoys a high probability bound under a boundedness assumption, i.e., for all . Note that the existing Epoch-SGD method [15] uses two different methods to derive its high probability bounds. Specifically the first method relies on an efficient function evaluator to select the best solutions among multiple trials of run; while the second one modifies the updating rule by projecting the solution onto the intersection of the domain and a center-shifted bounded ball with decaying radius. These two methods however may lead to additional computation steps, if being adopted for deriving high probability bounds for Epro-SGD.

Remark

The assumption can be satisfied practically, if we estimate the value of such that , and then project the intermediate solutions onto at every iteration. Note that Epoch-SGD [15] requires a total number of projections, and its high probability bound of Epoch-SGD is denoted by with a probability at lest , where .

### 3.3A Proximal Variant

We propose a proximal extension of Epro-SGD, by exploiting the structure of the objective function. Let the objective function in Eq. (Equation 1) be a sum of two components

where is a relatively simple function, for example a squared -norm or -norm, such that the involved proximal mapping

is easy to compute. The optimization problem in Eq. (Equation 1) can be rewritten as

Denote by the optimal solution to Eq. (Equation 11). We similarly introduce an augmented objective function as

The update of the proximal SGD method for solving (Equation 11) [9] is given by

If is a sparse regularizer, the proximal SGD can guarantee the sparsity in the intermediate solutions and usually yields better convergence than the standard SGD. However, given a complex constraint, solving the proximal mapping may be computational expensive. Therefore, we consider a proximal variant of Epro-SGD which involves only the proximal mapping of without the constraint . An instinctive solution is to use the following update in place of step 6 in Algorithm ?:

Based on this update and using techniques in Lemma ?, we obtain a similar convergence result (proof provided in Appendix), as presented in the following lemma [8].

Different from the main result in Lemma ?, Eq. ( ?) has an additional term ; it makes the convergence analysis in Epro-SGD difficult. To overcome this difficulty, we adopt the optimal regularized dual averaging (ORDA) algorithm [6] for solving Eq. (Equation 12). The details of ORDA are presented in Algorithm ?. The main convergence results of ORDA are summarized in the following lemma (proof provided in Appendix).

We present a proximal variant of Epro-SGD, namely Epro-ORDA, in Algorithm ?, and summarize its convergence results in Theorem ?. Note that Algorithm ? and the convergence analysis in Lemma ? are independent of the strong convexity in ; the strong convexity is however used for analyzing the convergence of Epro-ORDA in Theorem ? (proof provided in Appendix).

## 4An Example of Solving Lmnn via Epro-Sgd

In this section, we discuss an application of applying the proposed Epro-SGD to solve a high dimensional distance metric learning (DML) with a large margin formulation, i.e., the large margin nearest neighbor (LMNN) classification method [30]. LMNN classification is one of the state-of-the-art methods for k-nearest neighbor classification. It learns a positive semi-definite distance metric, based on which the examples from the k-nearest neighbors always belong to the same class, while the examples from different classes are separated by a large margin.

To describe the LMNN method, we first present some notations. Let , be a set of data points, where and denote the feature representation and the class label, respectively. Let be a positive definite matrix that defines a distance metric as . To learn a distance metric that separates the examples from different classes by a large margin, one needs to extract a set of similar examples (from the same class) and dissimilar examples (from a different class), denoted by , where shares the same class label to and a different class from . To this end, for each example one can form by extracting the k nearest neighbors (defined by an Euclidean distance metric) that share the same class label to , and form by extracting a set of examples that have a different class label. Then an appropriate distance metric could be obtained from the following constrained optimization problem

where is a hinge loss and is a trade-off parameter. In Eq. (Equation 15), is used as the constraint to ensure that Assumption A3 holds. Minimizing the first term is equivalent to maximizing the margin between and . The matrix encodes certain prior knowledge about the distance metric; for example, the original LMNN work [30] defines as , where are all k-nearest neighbor pairs from the same class. Other works [19] have used a weighted summation of distances between all data pairs or intra-class covariance matrix [25]. The last term is used as a regularization term and also makes the objective function strongly convex.

For data sets of very high dimensionality, i.e., , LMNN in Eq. (Equation 15) usually produces a sub-optimal solution [25], as this formulation does not capture the sparsity structure of the features. Therefore we add a sparse regularizer and express the formulation below

where is an elmenent-wise -norm excluding the diagonal entries. Note that this sparse regularizer have been previously used in [25] for a different purpose.

Many standard optimization solvers or algorithms may not be efficient for solving Eq. (Equation 16). Firstly, the optimization problem in Eq. (Equation 16) can be formulated as a semi-definite program (SDP); however, general SDP solvers usually scale poorly with the number of triplets and is not suitable for large scale data analysis. Secondly, the gradient decent method presented in [30] requires to project intermediate solutions onto a positive definite cone; this operation invokes expensive singular value decomposition (SVD) for a large matrix and this limitation restricts the real-world applications of the gradient descent method. Thirdly, [25] employs a block coordinate descent (BCD) method to solve an L-penalized log-det optimization problem; the BCD method is not suitable for solving Eq. (Equation 16), as the loss function is not linear in the variable .

We employ the proposed Epro-SGD algorithm to solve the LMNN formulation in Eq. (Equation 16). Let and . The positive definite constraint can be rewritten into an inequality constraint as , where denotes the minimum eigen-value of the matrix . We also make the correspondences , , , and provide necessary details below in a question-answer form

How to compute the stochastic gradient of ? First sample one triplet (or a small number of triplets) and then compute if , otherwise.

How to compute the gradient of ? By the theory of matrix analysis, the subgradient of can be computed by if , and zero otherwise, where denotes the eigevector of associated with its minimum eigenvalue.

What is the solution to the following proximal gradient step?

The solution can be obtained via a soft-thresholding algorithm [2].

What are the appropriate values for , that are necessary for running the algorithm? The value of . The value of is . The value of can be set to . The value of . The value of can be estimated as if we assume . The value of is usually tuned among a set of prespecified values.

Finally, we discuss the impact of employing Epro-SGD and Epro-ORDA method on accelerating the computation for solving LMNN. Note that at each iteration to compute the gradient of , we need to compute the minimum eigen-value and its eigen-vector. For a dense matrix, it usually involves a time complexity of . However, by employing a proximal projection, we can guarantee that the intermediate solution is a element-wise sparse solution, for which the computation of the last eigen-pair can be substantially reduced to be linear to the number of non-zeros elements in .

To analyze the running time compared to the Epoch-SGD method, let us assume we are interested in an -accurate solution. In the following discussion, we take a particular choice of and suppress the dependence on constants and only consider dependence on , and . The number of iterations required by Epro-SGD or Epro-ORDA is . Taking into account the running time per iteration, the total running time of Epoch-SGD is and that of Epro-SGD/Epro-ORDA is . When is very large, the speed-up can be orders of magnitude.

## 5Experiments

In this section, we empirically demonstrate the efficiency and effectiveness of the proposed Epro-SGD algorithm. We compare the following four algorithms:

Stochastic sub-Gradient Descent method (SGD) [27]: we set the step size and SGD achieves a rate of convergence , requiring projections for a constrained convex optimization problem.

One-Projection SGD method (OneProj) [20]: we set the step size and OneProj achieves a rate of convergence , requiring only one projection for a constrained strongly convex optimization problem.

-projections SGD method (logT) [33]: we set the step size and logT achieves a rate of convergence , requiring projections steps for a constrained strongly convex optimization problem.

the proposed Epro-SGD with number of projections (Epro): we set the step size and Epro archives a rate of convergence with projections steps for a constrained strongly convex optimization problem.

For illustration, we apply the competing algorithms for solving the constrained Lasso problem and the Large Margin Nearest Neighbor Classification (LMNN) in Eq. (Equation 16) respectively. We implement all algorithms using Matlab Ra and conduct all simulations on an Intel(R) Xeon(R) CPU E5-2430 (15M Cache, 2.20 GHz).

### 5.1Experiments on the Constrained Lasso Formulation

We apply the proposed Epro-SGD algorithm and the other three competing algorithms to solve the L-norm constrained least squares optimization problem

Eq. (Equation 17) is an equivalent constrained counterpart of the well studied Lasso formation [29]. They aim at achieving entry-wise sparsity in the weight vector while computing a linear predictor for regression.

We use the algebra data, a benchmark data from KDD Cup 2010 [28], for the following experiments. Specifically we use a preprocessed version of the algebra data^{1}

In the experiments, we respectively run all competing algorithms for iterations; we then record the obtained objective values and the corresponding computation time. The experimental results are presented in Figure 1. The left plot shows how the objective value is changed with respect to the algorithm iteration. Note that for Eq. (Equation 17), the number of algorithm iterations is equal to the number of stochastic gradient computation (the access to the subgradient of the objective function). From this plot, we can observe that after running iterations, Epro and SGD attain smaller objective values, compared to OneProj and LogT; we can also observe that OneProj empirically converges slightly faster than logT. The right plot shows how the objective value is changed with respect to the computation time. For this experiment, we set the maximum computation time to 3035 seconds, which is the computation time required by running Epro for iterations. We can observe that Epro attain a smaller objective value, compared to the other three competing method; meanwhile, the standard SGD and OneProj attain similar objective values, given a fixed amount of computation time.

### 5.2Experiments on the Lmnn Formulation

We apply the four competing algorithms to solve the LMNN formulation in Eq. (Equation 16). We use the Cora data [24] for the following experiments. Cora consists of scientific publications exclusively from different categories. Each publication is represented by a normalized vector of length and dimensionality . From this data, we construct neighbor pairs (NP) by randomly selecting publications of the same label; we then construct non-neighbor (NNT) by randomly selecting 3 non-neighbor publications (of a different label) for each of the NPs. Therefore, in each iteration of the SGD-type methods, we can use a NP and a NNT to construct a stochastic gradient for the optimization formulation. We set , , and in Eq. (Equation 16). We terminate the algorithms when the iteration number is larger than or the relative change of the objective values in two iterations is smaller than ; we also record the obtained objective values, the required iteration number, and the computation time. Similarly we tune the initial step size respectively for the competing algorithms; specifically, we set for SGD, and for Epro, for OneProj, and for LogT.

The experimental results are presented in Figure 2. Similarly in the left plot, we illustrate how the objective value changes with respect to the iteration number. For the LMNN formulation in Eq. (Equation 16), we can observe that Epro converges empirically much faster than all three competing algorithms; in particular, Epro converges after iterations, while the other algorithms need more iterations. In the right plot, we illustrate how the objective value changes with respect to the computation time. We can observe that Epro converges using a smaller amount of computation time. specifically, in our experiment Epro converges with the computation time as seconds; while the other three competing algorithms need much more computation time.

## 6Conclusions

We proposed an epoch-projection based SGD method, called Epro-SGD, for stochastic strongly convex optimization. The proposed Epro-SGD applies SGD on each iteration within its epochs and only performs a projection at the end of each epoch. Our analysis shows that Epro-SGD requires only a logarithmic number of projections, while achieves a guaranteed optimal rate of convergence both in expectation as well as with high probability. Additionally we proposed a variant of Epro-SGD based on an existing dual averaging method, called Epro-ORDA, which exploit structures of the optimization problems by incorporating an associated proximal mapping iteratively. For illustration, we applied the proposed Epro-SGD method for solving a large margin distance metric learning formulation and a constrained Lasso formulation respectively with a positive definite constraint. Our empirical results demonstrate the effectiveness of the proposed method.

## Acknowledgements

The authors would like to thank the anonymous reviewers for their helpful comments. T. Yang was supported in part by NSF (1463988, 1545995).

## Supplemental Materials

From Assumption A1, we have . Hence

Moreover from the strong convexity in we have

From the two inequalities above, we can easily verify that

This completes the proof.

### Proof of Theorem

The proof of Theorem ? is based on an important result, as summarized in Lemma ?.

*Proof of Theorem ?* The proof below follows from techniques used in Lemma ? and Theorem ?. Since is -strongly convex, we have

Combining the above inequality with the inequality in (Equation 5) and taking summation over all , we have

We substitute the bound in Lemma ? into the above inequality with . We consider two cases. In the first case, we assume . As a result, we have

which together with the inequality in (Equation 18) leads to the bound

In the second case, we assume

where the last step uses the fact . We thus have

Combing the results of the two cases, we have, with a probability ,

where . Following the same analysis, we have

Let . By induction, we have

Assume , by plugging the values of , we have

where we use and and . This completes the proof of this theorem.

### Proof of Lemma

To prove Lemma ?, we derive an inequality similar to Eq. (Equation 6); the rest proof of Lemma ? is similar to that of Lemma ?.

Corollary ? can be proved using techniques similar to the ones in [9] but with extra care on the stochastic gradient. As a consequence we have

### Proof of Lemma

The lemma is a corollary of results in [6] for general convex optimization. In particular, if we consider the stochastic composite optimization

where is a simple function such that its proximal mapping can be easily solved and is only accessible through a stochastic oracle that returns a stochastic subgradient . To state the convergence of ORDA for general convex problems, [6] makes the following assumptions: (i) and (ii)

When , the first inequality holds and the second inequality holds with . Applying to the augmented objective

We note that and . Follow the inequality (26) in the appendix of [6], we obtain that

by using the Euclidean distance and their notation , and noting that is the inverse of their notation . Then the second inequality is Lemma ? can be proved similarly as for Lemma ?.

### Proof of Theorem

Recall and . Let . By the values of and we have

Define . We first prove the inequality

by induction. It is true for because of Lemma ?, and . Now assume it is true for and we prove it for . For a random variable measurable with respect to the randomness up to epoch . Let denote the expectation conditioned on all the randomness up to epoch . Following Lemma ?, we have

Since by the strong convexity, we have