SECOND ORDER OPTIMALITY CONDITIONS FOR STRONG LOCAL MINIMIZERS VIA SUBGRADIENT GRAPHICAL DERIVATIVE

# Second Order Optimality Conditions for Strong Local Minimizers via Subgradient Graphical Derivative

Nguyen Huy Chieu111 Email: nghuychieu@gmail.com. ,    Le Van Hien222Department of Natural Science Teachers, Ha Tinh University, Ha Tinh, Vietnam; email: lehiendhv@gmail.com.,    Tran T. A. Nghia333Department of Mathematics and Statistics, Oakland University, Rochester, MI 48309, USA; email: nttran@oakland.edu. Research of this author was supported by the US National Science Foundation under grant DMS-1816386.,   Ha Anh Tuan444Faculty of Basic Science, Ho Chi Minh city University of Transport, Ho Chi Minh city, Vietnam; email: hatuanhuyhoang@gmail.com
###### Abstract

This paper is devoted to the study of second order optimality conditions for strong local minimizers in the frameworks of unconstrained and constrained optimization problems in finite dimensions via subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous proper function at a proximal stationary point is sufficient for the quadratic growth condition. It is also a necessary condition for the latter property when the function is either subdifferentially continuous, prox-regular, twice epi-differentiable or variationally convex. By applying our results to the -cone reducible constrained programs, we establish no-gap second order optimality conditions for (strong) local minimizers under the metric subregularity constraint qualification. These results extend the classical second order optimality conditions by surpassing the well-known Robinson’s constraint qualification. Our approach also highlights the interconnection between the strong metric subregularity of subdifferential and quadratic growth condition in optimization problems.

Key words. Quadratic growth, strong local minimizer, second order sufficient condition, subgradient graphical derivative, metric subregularity constraint qualification, conic programming.

2010 AMS subject classification. 49J53, 90C31, 90C46

## 1 Introduction

In this paper, we mainly study second order optimality conditions for strong local minimizers to nonsmooth extended-real-valued functions with applications to conic programming in finite dimensional spaces. Strong local minimizer is an important concept in optimization at which the quadratic growth condition is satisfied. In the case of unconstrained -smooth optimization problems, this property of a minimizer is fully characterized by the positive definiteness of the Hessian of the cost function at stationary points. When the problem is not smooth, several different types of second order directional derivatives were introduced to study strong minimizers [3, 6, 7, 30, 31, 33]. These structures are purely primal due to the involvement of quantities only on primal spaces. Under some constraint qualifications and special regularities, these second order structures could be computed and lead to many optimization applications; see, e.g., [7, 30].

It is natural to raise the question whether there is any non-primal second order structure that could characterize strong minimizers and if it exists, the concern is about its computability in different frameworks under weaker regularities. In 2014, Aragón Artacho and Geoffroy [2] used subgradient gradient derivative, which is a second order construction as the graphical derivative acting on the subgradient mapping, to depict strong minimizers to convex functions. Due to the involvement of both graphical derivative and subgradient, it is known as a primal-dual structure. The idea of using subgradient graphical derivative to investigate strong local minimizers indeed dates back to Eberhard and Wenczel [15]. Several related second order structures with full computation have been also used in optimization for different purposes in [5, 10, 8, 9, 17, 18, 30]. The approach [2] is based on the interconnection between the quadratic growth condition and the strong metric subregularity of the convex subdifferential of the cost function studied earlier in [1, 34]. Without convexity, Drusvyatskiy, Mordukhovich and Nghia [14] showed that the latter property of the limiting subdifferential at a local minimizer implies the quadratic growth condition. The converse implication is also true for a big class of nonconvex semi-algebraic functions [12]. It is worth noting that the strong metric subregularity of the subdifferential is a remarkable property in the study of the linear convergence of various first-order methods [5, 13, 32]. Therefore, the connection between the quadratic growth condition and the strong metric subregularity of the subdifferential is also interesting from numerical viewpoint.

The first aim of this paper is to characterize strong local minimizers of unconstrained nonsmooth optimization problems without convexity via the subgradient graphical derivative structure. We indeed show in Theorem 3.2 and  3.3 that the positive definiteness of the subgradient graphical derivative at proximal stationary points is sufficient for strong local minimizers. It also become necessary for two broad classes of subdifferentially continuous, prox-regular, twice epi-differentiable functions [30, Chapter 13] and variationally convex functions [29, 28]. Through our approach, it is revealed that the appearance of strong local minimizers/quadratic growth conditions to the cost function is equivalent to the strong metric subregularity of subgradient mapping at local minimizers in the latter two favorable settings. Since functions from the just mentioned two classes are not necessarily semi-algebraic and convex, our results complement the corresponding ones constructed in [1, 14, 12].

The second aim of this paper is to study strong local minimizers to smooth conic programming by applying our theory to establishing no-gap second order optimality conditions for (strong) local minimizers. For nonlinear programming, such a no-gap optimality condition is well-known in literature [4, 21] via the classical second order necessary and sufficient conditions at stationary points under the Mangasarian-Fromovitz constraint qualification or even under the calmness condition (see [20, Theorem 2.1] and [7, Theorem 3.70 (i)]). This result was extended in [7] to the class of -cone reducible programming, which include problems of nonlinear programming, semidefinite programming, and second order cone programming under Robinson’s constraint qualification (RCQ). Without RCQ, there are limited results about this important characterization. The approach in [7] is mainly based on some primal structures such as the second order subderivatives and second order tangent cones; see also [6]. With the primal-dual structure, we are able to establish similar results under the so-called metric subregularity constraint qualification (MSCQ) [18, 17], which is strictly weaker than RCQ. Especially, connection between the quadratic growth condition and the strong metric subregularity at local minimizers is also obtained.

The rest of the paper is organized as follows. Section 2 recalls some materials from variational analysis, which are needed for the sequel analysis. Section 3 is devoted to the study of the quadratic growth condition for extended-real-valued functions via the subgradient graphical derivative. Section 4 presents our results on no-gap second order necessary and sufficient optimality conditions for conic programs under the metric subregularity constraint qualification. Finally, section 5 consists of some concluding remarks on the obtained results as well as the perspectives of this research direction.

## 2 Preliminaries

In this section we recall some basic notions and facts from variational analysis that will be used repeatedly in the sequel; see [11, 23, 24, 30] for more details. Let be a nonempty subset of the Euclidean space and be a point in . The (Bouligand-Severi) tangent/contingent cone to the set at is known as

 TΩ(¯x):={v∈Rn|there exist tk↓0, vk→v  with  ¯x+tkvk∈Ω for all k∈N}.

The polar cone of the tangent cone is the (Fréchet) regular normal cone to at defined by

 ˆNΩ(¯x):=TΩ(¯x)∘. (2.1)

Another normal cone construction used in our work is the (Mordukhovich) limiting/basic normal cone to at defined by

 NΩ(¯x):={v∈Rn|there existxk\lx@stackrelΩ→¯x,vk∈ˆNΩ(xk) with vk→v}.

When we set and by convention. When the set is convex, the above tangent cone and normal cones reduce to the tangent cone and normal cone in the sense of classical convex analysis.

Consider the set-valued mapping with the domain and graph . Suppose that is an element of . The graphical derivative of at for is the set-valued mapping defined by

 DF(¯x|¯y)(w):={z∈Rm|(w,z)∈TgphF(¯x,¯y)} for  w∈Rn, (2.2)

which means ; see, e.g., [11, 30]. We note further that if is a single-valued mapping differentiable at then for any .

Following [11, Section 3.8], we say is metrically subregular at for with modulus if there exists a neighborhood of such that

 d(x;F−1(¯y))≤κd(¯y;F(x))for allx∈U, (2.3)

where represents the distance from a point to a set . The infimum of all such is the modulus of metric subregularity, denoted by . If additionally is an isolated point to , we say is strongly metrically subregular at for . It is known from [11, Theorem 4E.1] that is strongly metrically subregular at for if and only if

 DF(¯x|¯y)−1(0)={0}. (2.4)

Moreover, in the latter case, its modulus of (strong) metric subregularity is computed by

 subregF(¯x|¯y)=1inf{∥z∥|z∈DF(¯x|¯y)(w),∥w∥=1}. (2.5)

Assume that is an extended-real-valued lower semicontinuous (l.s.c.) proper function with . The limiting subdifferential (known also as the Mordukhovich/basic subdifferential) of at is defined by

 ∂f(¯x):={v∈Rn|(v,−1)∈Nepif(¯x,f(¯x))},

where is the epigraph of . Another subdifferential construction used in this paper is the proximal subdifferential of at defined by

 ∂pf(¯x):={v∈Rn|liminfx→¯xf(x)−f(¯x)−⟨v,x−¯x⟩∥x−¯x∥2>−∞}. (2.6)

It is well-known that

 ∂f(¯x)={v∈Rn|there % exists(xk,vk)→(¯x,v)withvk∈∂pf(xk)andf(xk)→f(¯x)}, (2.7)

which shows that .

Function is said to be prox-regular at for if there exist such that for all with we have

 f(x)≥f(u)+⟨v,x−u⟩−r2∥x−u∥2for allv∈∂f(u)∩Bε(¯v), (2.8)

where is the closed ball with center and radius ; see [30, Definition 13.27]. This clearly implies that whenever with . Moreover, is said to be subdifferentially continuous at for if whenever and , one has ; [30, Definition 13.28]. In the case is subdifferentially continuous at for , the inequality “” in the definition of prox-regularity above could be omitted.

Recall [30, Definition 13.3] that the second subderivative of at for and is given by

 d2f(¯x|v)(w)=liminf τ↘0w′⟶wΔ2τf(¯x|v)(w′), (2.9)

where

 Δ2τf(¯x|v)(w′)=f(¯x+τw′)−f(¯x)−τ⟨v,w′⟩12τ2.

Function is said to be twice epi-differentiable at for if for every and choice of there exist such that

 f(¯x+τkwk)−f(¯x)−τk⟨v,wk⟩12τ2k→d2f(¯x|v)(w);

see, e.g., [30, Definition 13.6]. We note that fully amenable functions, including the maximum of finitely many -functions, are important examples for subdifferentially continuous prox-regular and twice epi-differentiable l.s.c. proper functions [30, Corollary 13.15 & Proposition 13.32].

The main second order structure used in this paper is the subgradient graphical derivative at for , which is defined from (2.2) by

 D(∂f)(¯x|¯v)(w):={z|(w,z)∈Tgph∂f(¯x,¯v)}for allw∈Rn. (2.10)

In the case that is twice epi-differentiable, prox-regular, subdifferentially continuous at for , it is known from [30, Theorem 13.40] that

 D(∂f)(¯x|¯v)=∂hwithh=12d2f(¯x|¯v), (2.11)

which is an important formula in our study. When is twice differentiable at , it is clear that .

## 3 Second Order Optimality Conditions for Strong Local Minimizer via Subgradient Graphical Derivative

Given a function and a point , is called a strong local minimizer of with modulus if there is a number such that the following quadratic growth condition (QGC, in brief) holds

 f(x)−f(¯x)≥κ2∥x−¯x∥2for allx∈Bγ(¯x). (3.1)

We define the exact modulus for QGC of at by

 QG(f;¯x):=sup{κ>0|¯x is a strong local % minimizer of f with modulus κ}.

In this section we introduce several new sufficient and necessary conditions for the quadratic growth condition (3.1) by using the second order construction defined in (2.10). The following result taken from [14, Corollary 3.5] providing a sufficient condition for the QGC of at (3.1) via strong metric subregularity on the subgradient mapping is a significant tool in our analysis.

###### Lemma 3.1.

(strong metric subregularity of the subdifferential, [14, Corollary 3.5]). Let be a l.s.c. proper function and let be a stationary point of with . Suppose that the subgradient mapping is strongly metrically subregular at for with modulus and there are real numbers and such that

 f(x)≥f(¯x)−r2∥x−¯x∥2for % allx∈Bδ(¯x). (3.2)

Then for any , there exists a real number such that

 f(x)≥f(¯x)+α2∥x−¯x∥2% for allx∈Bη(¯x). (3.3)

When the function is convex, the QGC could be fully characterized via the positive definiteness of subgradient graphical derivative (2.10) [2, Corollary 3.7]. Without convexity, we show in the following result that such a property is sufficient for QGC.

###### Theorem 3.2.

(Sufficient condition I for strong local minimizers via subgradient graphical derivative). Let be a proper l.s.c. function with . Suppose that and that there exists some real number such that

 ⟨z,w⟩≥c∥w∥2for allz∈D(∂f)(¯x|0)(w)andw∈\rm domD(∂f)(¯x|0). (3.4)

Then is a strong local minimizer with any modulus . Moreover, we have

 QG(f;¯x)≥inf{⟨z,w⟩∥w∥2∣∣z∈D(∂f)(¯x|0)(w),w∈\rm domD(∂f)(¯x|0)} (3.5)

with the convention that .

Proof. Since , we have and there exist such that

 f(x)−f(¯x)≥−r2∥x−¯x∥2for allx∈Bγ(¯x). (3.6)

To proceed, pick any and define , it is clear that

 g(x)−g(¯x)≥s2∥x−¯x∥2for allx∈Bγ(¯x). (3.7)

Note further that and thus . Thanks to the sum rule of graphical derivative [11, Proposition 4A.2], we have

 D(∂g)(¯x|0)(w)=D(∂f)(¯x|0)(w)+swfor allw∈Rn. (3.8)

Take any with , i.e., . It follows from (3.4) that , which means

 ∥z∥⋅∥w∥≥⟨z,w⟩≥(c+s)∥w∥2. (3.9)

We obtain that , i.e., is strongly metrically subregular at for by (2.4). Moreover, by (2.5) tells us that

 subreg∂g(¯x|0)≤(c+s)−1.

Since is a local minimizer of by (3.7), it follows from Lemma 3.1 that for any there exists such that

 g(x)≥g(¯x)+12(subreg∂g(¯x|0)+ε)∥x−¯x∥2≥g(¯x)+12((c+s)−1+ε)∥x−¯x∥2,

for all Since , we obtain from the latter that

 (3.10)

By choosing sufficiently small, is a strong local minimizer of with a positive modulus being smaller than but arbitrarily close to . This verifies that is a strong local minimizer of with any modulus in and the QGC of holds at . Moreover, the inequality (3.5) follows from (3.10) when taking and the infimum on the right-hand side of (3.5). The proof is complete.

When the function is twice differentiable, the above theorem recovers the classical second order sufficient condition, which says if and there exists some such that

 ⟨∇2f(¯x)w,w⟩≥c∥w∥2for allw∈Rn, (3.11)

then is a strong local minimizer of . Condition (3.11) is known to be equivalent to the condition:

 ⟨∇2f(¯x)w,w⟩>0for allw∈Rn,w≠0. (3.12)

In the nondifferential case, it is natural to question whether condition (3.4) is equivalent to the following condition:

 ⟨z,w⟩>0for allz∈D(∂f)(¯x|0)(w),w∈\rm domD(∂f)(¯x|0)∖{0}. (3.13)

Obviously, this inequality is a consequence of (3.4). In the general case, we do not know yet whether the converse implication is also true. However, we show that (3.13) is also a sufficient condition to the QGC in the next result, which could be seen as a refinement for our Theorem 3.2 above. The equivalence between (3.4) and (3.13) will be clarified later in Theorem 3.8, Theorem 3.11, and Theorem 4.6 for several broad classes of nondifferentiable functions.

It is worth noting that both (3.4) and (3.13) imply the strong metric regularity of the subgradient mapping at for due to (2.4). This important feature allows us to use Lemma 3.1 to verify strong local minimizer in the following theorem.

###### Theorem 3.3.

(Sufficient condition II for strong local minimizers via subgradient graphical derivative). Let be a proper l.s.c. function with . Consider the following assertions:

1. is a strong local minimizer of .

2. is a local minimizer and is strongly metrically subregular at for .

3. and is positive definite in the sense of (3.13).

Then we have the implications .

Proof. The implication follows directly from Lemma 3.1. To justify , suppose that and condition (3.13) is satisfied. It follows that

 D(∂f)(¯x|0)−1(0)={0}.

By (2.4), is strongly metrically subregular at for with some . Since , we find some such that (3.6) is valid. Pick any and define as in the proof of Theorem 3.2 again. For any with , we derive from (3.8) that . It follows from (3.13) that , which means

 ∥z∥⋅∥w∥≥⟨z,w⟩≥s∥w∥2.

This together with (2.4) and (2.5) tells us that is strongly metrically subregular at for with

 subreg∂g(¯x|0)≤s−1.

Since is a local minimizer of by (3.7), it follows from Lemma 3.1 again that for any with , there exists such that

 g(x)≥g(¯x)+12(subreg∂g(¯x|0)+ε)∥x−¯x∥2≥g(¯x)+12(s−1+ε)∥x−¯x∥2,

for all Since , we derive

 f(x)≥f(¯x)+12[1s−1+ε−s]∥x−¯x∥2=f(¯x)−12sεs−1+ε∥x−¯x∥2for allx∈Bδ(¯x).

Since and is strongly metrically subregular at for with modulus , is a (strong) local minimizer of by Lemma 3.1. The proof is complete.

As far as we know, the first idea of using the subgradient graphical derivative to study the quadratic growth condition was initiated by Eberhard and Wenczel [15] in which they introduced the so-called sufficient condition of the second kind.

###### Definition 3.4.

(sufficient condition of the second kind,[15]) Let be a proper l.s.c. function with and . We say the sufficient condition of the second kind holds at when there exists such that for each with

 ∃z∈D(∂pf)(¯x|0)(w)satisfying⟨z,w⟩≥κ. (3.14)

Precisely, [15, Theorem 71(2)] claims that when the function is l.s.c., prox-bounded, and proximally stable, the sufficient condition of the second kind at with ensures the QGC of at . However, it seems to us that this result is incorrect even in the convex case. To see this, let us consider the following example.

###### Example 3.5.

Define the function by

 f(x)=⎧⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪⎩xifx>1αn+1x+βn+1ifαn+1

where , with , , and . It is easy to see that is a continuous and convex function with global optimal solution , which clearly implies that is prox-bounded and proximally stable at in the sense of [15].

Moreover, direct computation on gives us that

 ∂pf(x)=∂f(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩{1}ifx>1[αn+1,αn]ifx=αn,n=0,1,2,…{αn+1}ifαn+1

Define further , we have and

 Tgph∂pf(¯x,0)⊂TK(¯x,0)=K. (3.17)

Next we verify the “” inclusion in (3.17). Take any with and consider the following three cases:

• Case 1: clearly belongs to .

• Case 2: . Choose as , we have and thus .

• Case 3: . Fix satisfying and define for , we have .

It follows that . Similarly, we have . This together with (3.17) ensures the equality in (3.17). Note further that and . Thus the sufficient condition of the second kind (3.14) holds at with . However, both (3.4) and (3.13) are not satisfied and the quadratic growth condition (3.1) is not valid at . This tells us that [15, Theorem 71(2)] is inaccurate even in the convex case.

As discussed in the Introduction, QGC and strong local minimizer could be fully characterized via several different types of second order directional derivatives [3, 7, 31, 33, 30]. For instance, it follows from [30, Theorem 13.24] that is a strong local minimizer to a proper function if and only if (or ) and the second subderivative (2.9) of at for is positive definite in the sense that

 d2f(¯x|0)(w)>0for allw∈Rn,w≠0. (3.18)

It is clear that second subderivative (2.9) is a construction on primal space, while the subgradient graphical derivative (2.10) includes both primal and dual spaces. Connection between these two constructions could be found in (2.11) for a special class of subdifferentially continuous, prox-regular and twice epi-differentiable functions. Despite the simplicity of second subderivative and the full characterization of QGC (3.18), computing could be challenging under some strong regularity conditions. On the other hand, subgradient graphical derivative is fully computed in many broad classes of optimization problems [8, 10, 17] under milder assumptions.

Unlike (3.18), both of our conditions (3.4) and (3.13) are not generally necessary conditions for strong local minimizers, as shown in the following example.

###### Example 3.6.

Let be the function defined as follows

 f(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩x  if x∈{0}∪[1,+∞),12nif x∈[32n+2,12n), n=0,1,2,...,2x−12n+1if x∈[12n+1,32n+2), n=0,1,2,...,f(−x) if x<0.

We see that for all , which means that is a strong local minimizer of . On the other hand, since

 (∞⋃n=0(−12n,−32n+2)∪{0}∪∞⋃n=0(32n+2,12n))×{0}⊂\rm gph∂f,

it follows that Therefore, for and we have

 z∈D(∂f)(¯x|0)(w)and⟨z,w⟩=0.

This shows that (3.4) and (3.13) are not necessary conditions for strong local minimizers.

Our next aim is to present several classes of functions at which both (3.4) and (3.13) are also necessary conditions for strong local minimizers. To this end, we first need the following lemma.

###### Lemma 3.7.

Let be a proper function. Suppose that is positively homogenenous of degree in the sense that for all and . Then for any and , we have .

Proof. For any with , by (2.6) and (2.7) we find sequences , , and such that , , , and that

 h(u)−h(wk)≥⟨zk,u−wk⟩−rk2∥u−wk∥2for allu∈Bεk(wk).

By choosing with and in the above inequality, the positive homogeneneity of degree of tells us that

 (λ2−1)h(wk)=h(λwk)−h(wk)≥⟨zk,(λ−1)wk⟩−rk2(λ−1)2∥wk∥2. (3.19)

When satisfying , we get from inequality (3.19) that

 (λ+1)h(wk)≥⟨zk,wk⟩−rk2(λ−1)∥wk∥2.

Taking gives us that . Similarly, when with , we derive from (3.19) that

 (λ+1)h(wk)≤⟨zk,wk⟩−rk2(λ−1)∥wk∥2.

By letting , the latter implies . Thus we have , which clearly yields when due to the choice of at the beginning.

###### Theorem 3.8.

(Characterization of strong local minimizers for prox-regular and twice epi-differentiable functions). Let be a l.s.c. proper function with . Suppose that and is subdifferentially continuous, prox-regular, and twice epi-differentiable at for . Then the following assertions are equivalent:

1. is a strong local minimizer.

2. is a local minimizer and is strongly metrically subregular at for .

3. is positive definite in the sense of (3.13).

4. is positive definite in the sense of (3.4).

Moreover, if one of the assertions holds then

 QG(f;¯x)=inf{⟨z,w⟩∥w∥2∣∣z∈D(∂f)(¯x|0)(w)}. (3.20)

Proof. Since is subdifferentially continuous and prox-regular at for , we have . Thus, implications follow from Theorem 3.3. It remains to verify and (3.20) is valid. To this end, suppose that is a strong local minimizer with modulus  as in (3.1). We derive from (3.1) and (2.9) that

 d2f(¯x|0)(w)≥κ∥w∥2for all w∈Rn. (3.21)

Since is subdifferentially continuous, prox-regular, and twice epi-differentiable at for it follows from (2.11) that

 D(∂f)(¯x|0)=∂hwithh(⋅):=12d2f(¯x|0)(⋅) (3.22)

Note from (2.9) and (3.21) that is proper and positively homogenenous of degree . By Lemma 3.7, for any we obtain from (3.21) and (3.22) that

 ⟨z,w⟩=2h(w)=d2f(¯x|0)(w)≥κ∥w∥2, (3.23)

which clearly verifies (iv) and

 κ≤inf{⟨z,w⟩∥w∥2∣∣z∈D(∂f)(¯x|0)(w)}.

Since is an arbitrary modulus of the strong local minimizer the latter implies that

 QG(f;¯x)≤inf{⟨z,w⟩∥w∥2∣∣z∈D(∂f)(¯x|0)(w)}.

This along with (3.5) justifies (3.20) and finishes the proof.

Besides the full characterization of strong local minimizers in terms of (3.4) and (3.13) for a class of prox-regular and twice epi-differentiable functions, the above theorem also tells us the equivalence between QGC and the strong metric subregularity of subdifferential at a local minimizer for . This correlation has been also established for different classes of functions in [1, 14, 12].

The above theorem allows us to recover [15, Corollary 73].

###### Corollary 3.9.

Let a l.s.c. proper function with . Suppose that