OracleBased Robust Optimization via Online Learning
Abstract
Robust optimization is a common framework in optimization under uncertainty when the problem parameters are not known, but it is rather known that the parameters belong to some given uncertainty set. In the robust optimization framework the problem solved is a minmax problem where a solution is judged according to its performance on the worst possible realization of the parameters. In many cases, a straightforward solution of the robust optimization problem of a certain type requires solving an optimization problem of a more complicated type, and in some cases even NPhard. For example, solving a robust conic quadratic program, such as those arising in robust SVM, ellipsoidal uncertainty leads in general to a semidefinite program. In this paper we develop a method for approximately solving a robust optimization problem using tools from online convex optimization, where in every stage a standard (nonrobust) optimization program is solved. Our algorithms find an approximate robust solution using a number of calls to an oracle that solves the original (nonrobust) problem that is inversely proportional to the square of the target accuracy.
1 Introduction
The Robust Optimization (RO; see BenTal and Nemirovski (2002); BenTal et al. (2009); Bertsimas et al. (2011)) framework addresses a fundamental problem of many convex optimization problems: slight inaccuracies in data give rise to significant fluctuations in the solution. While there are different approaches to handle uncertainty in the parameters of an optimization problem, the RO approach choose a solution that performs best against the worst possible parameter. When the objective function is convex in the parameters, and concave in the uncertainty, and when the uncertainty set is convex the overall optimization problem is convex.
Despite its theoretical and empirical success, a significant hinderance of adopting RO to large scale problems is the increased computational complexity. In particular, robust counterpart of an optimization problem is often more difficult, albeit usually convex, mathematical problems. For example, the robust counterpart of conic quadratic programming with ellipsoidal uncertainty constraints becomes a semidefinite program, for which we currently have significantly slower solvers.
RO has recently gained traction as a tool for analyzing machine learning algorithms and for devising new ones. In a sequence of papers, Xu, Caramanis and Mannor show that several standard machine learning algorithms such as Lasso and norm regularized support vector machines have a RO interpretation (Xu et al., 2009, 2010). Beyond these works, robustness is a desired property for many learning algorithms. Indeed, making standard algorithms robust to outliers or to perturbation in the data has been proposed in several works; see Lanckriet et al. (2003); Bhattacharyya et al. (2004a, b); Shivaswamy et al. (2006); Trafalis and Gilbert (2007). However in these cases, the problem eventually solved is more complicated than the original problem. For example, in Trafalis and Gilbert (2007) the original problem is a standard support vector machine, but when robustifying it to input uncertainty, one has to solve a secondorder conic program (in the nonseparable case). Another example is Shivaswamy et al. (2006) where the uncertainty is a probability distribution over inputs. In that case, the original SVM becomes a secondorder conic program as well.
The following question arrises: can we (approximately) solve a robust counterpart of a given optimization problem using only an algorithm for the original optimization formulation? In this paper we answer this question on the affirmative: we give two metaalgorithms that receive as input an oracle to the original mathematical problem and approximates the robust counterpart by invoking the oracle a polynomial number of times. In both approaches, the number of iterations to obtain an approximate robust solution is a function of the approximation guarantee and the complexity of the uncertainty set, and does not directly depend on the dimension of the problem. Our methods differ on the assumptions regarding the uncertainty set and the dependence of the constraints on the uncertainty. The first method allows any concave function of the noise terms but is limited to convex uncertainty sets. The second method allows arbitrary uncertainty sets as long as a “pessimization oracle” (as termed by Mutapcic and Boyd 2009) exists— an oracle that finds the worst casenoise for a given feasible solution. Our methods are formally described as template, or metaalgorithms, and are general enough to be applied even if the robust counterpart is NPhard^{1}^{1}1Recall that we are only providing an approximate solution, and thus our algorithms formally constitute a “polynomial time approximation scheme” (PTAS)..
Our approach for achieving efficient oraclebased RO is to reduce the robust formulation to a zerosum game, which we solve by a primaldual technique based on tools from online learning. Such primaldual methods originated from the study of approximation algorithms for linear programs (Plotkin et al., 1995) and were recently proved invaluable in understanding Lagrangian relaxation methods (see e.g. Arora et al. 2012) and in sublineartime optimization techniques (Clarkson et al., 2012; Hazan et al., 2011; Garber and Hazan, 2011). We show how to apply this methodology to oraclebased RO. Along the way, we contribute some extensions to the existing online learning literature itself, notably giving a new FollowthePerturbedLeader algorithm for regret minimization that works with (additive) approximate linear oracles.
Finally, we demonstrate examples and applications of our methods to various RO formulation including linear, semidefinite and quadratic programs. The latter application builds on recently developed efficient lineartime algorithms for the trust region problem (Hazan and Koren, 2014).
Related work.
Robust optimization is by now a field of study by itself, and the reader is referred to BenTal et al. (2009); Bertsimas et al. (2011) for further information and references. The computational bottleneck associated with robust optimization was addressed in several papers. Calafiore and Campi (2004) propose to sample constraints from the uncertainty set, and obtain an “almostrobust” solution with high probability with enough samples. The main problem with their approach is that the number of samples can become large for a highdimensional problem.
For certain types of discrete robust optimization problems, Bertsimas and Sim (2003) propose solving the robust version of the problem via (the dimension) solutions of the original problem. Mutapcic and Boyd (2009) give an iterative cutting plane procedure for attaining essentially the same goal as us, and demonstrate impressive practical performance. However, the overall running time for their method can be exponential in the dimension.
Oraganization.
The rest of the paper is organized as follows. In Section 2 we present the model and set the notations for the rest of the paper. In Section 3.1 we describe the simpler of our two metaalgorithms: a meta algorithm for approximately solving RO problems that employs dual subgradient steps, under the assumption that the robust problem is convex with respect to the noise variables. In Section 3.2 we remove the latter convexity assumption and only assume that the problem of finding the worstcase noise assignment can be solved by invoking a “pessimization oracle”. This approach is more general than the subgradientbased method and we exhibit perhaps our strongest example of solving robust quadratic programs using this technique in Section 4. Section 4 also contains examples of application of our technique for robust linear and semidefinite programming. We conclude in Section 5.
2 Preliminaries
We start this section with the standard formulation of RO. We then recall some basic results from online learning.
2.1 Robust Optimization
Consider a convex mathematical program in the following general formulation:
(1) 
Here are convex functions, is a convex set in Euclidean space, and is a fixed parameter vector. The robust counterpart of this formulation is given by
(2) 
where the parameter vector is constrained to be in a set called the uncertainty set. It is without loss of generality to assume that the uncertainty set has this specific form of a cartesian product, see e.g. BenTal and Nemirovski (2002). Here we also assume that the uncertainty set is symmetric (that is, its projection onto each dimension is the same set). This assumption is only made for simplifying notations and can be relaxed easily.
The following observation is standard: we can reduce the above formulation to a feasibility problem via a binary search over the optimal value of (1), replacing the objective with the constraint with being our current guess of the optimal value (of course, assuming the range of feasible values is known apriory). For ease of notation, we rename by shifting it by , and can write the first constraint as simply . With these observations, we can reduce the robust counterpart to the feasibility problem
(3) 
We say that is an approximate solution to this problem if meets each constraint up to , that is, if it satisfies for all ().
2.2 Online Convex Optimization and Regret minimization
Our derivations below use tools from online learning, namely algorithms for minimizing regret in the general prediction framework of Online Convex Optimization (OCO). In OCO^{2}^{2}2Here we present OCO as the problem of online maximization of concave reward functions rather than online minimization of convex cost functions. While the latter is more common, both formulations are equivalent. , the online predictor iteratively produces a decision from a convex decision set . After a decision is generated, a concave reward function is revealed, and the decision maker suffers a loss of . The standard performance metric in online learning is called regret, given by
The reward function is not known to the decision maker before selecting and it is, in principal, arbitrary and even possibly chosen by an adversary. We henceforth make crucial use of this robustness against adversarial choice of reward functions: the reward functions we shall use will be chosen by a dual optimization problem, thereby directing the entire algorithm towards a correct solution. We refer the reader to (CesaBianchi and Lugosi, 2006; Hazan, 2011; ShalevShwartz, 2012) for more details on online learning and online convex optimization.
Two regretminimization algorithms that we shall use henceforth (at least in spirit) are Online Gradient Descent (Zinkevich, 2003) and Follow the Perturbed Leader (Kalai and Vempala, 2005).
Online Gradient Descent (OGD).
In OGD, the decision maker predicts according to the rule
(4) 
where is the Euclidean projection operator onto the set . Hence, the OGD algorithm takes a projected step in the direction of the gradient of the current reward function. Even thought the next reward function can be arbitrary, it can be shown that this algorithm achieves a sublinear regret.
Follow the Perturbed Leader (FPL).
The FPL algorithm works in a similar setting as OGD, but with two crucial differences:

The set does not need to be convex. This is a significant advantage of the FPL approach, which we make use of in our application to robust quadratic programming (see Section 4.2).

FPL assumes that the reward functions are linear, i.e. with .
Kalai and Vempala (2005) suggest the following method for online decision making that relies on a linear optimization procedure over the set that computes for all . FPL chooses by first drawing a perturbation uniformly at random, and computing:
(5) 
The regret of this algorithm is bounded as follows.
Lemma 2 (Kalai and Vempala 2005).
For any sequence of reward vectors , let be the sequence of decisions generated by (5) with parameter . Then
where is an upper bound on the magnitude of the rewards, is an upper bound on the norm of the reward vectors, and is an upper bound on the diameter of .
For our purposes, and in order to be able to work with an approximate optimization oracle to the original mathematical program, we need to adapt the original FPL algorithm to work with noisy oracles. This adaptation is made precise in Section 3.2.
3 OracleBased Robust Optimization
In this section we formally state and prove our first (and simpler) result: an oraclebased approximate robust optimization algorithm that is based on subgradient descent.
Throughout the section we assume the availability of an optimization oracle for the original optimization problem of the form given in Figure 1, which we denote by . Such an optimization oracle approximately solves formulation (3) for any fixed noise , in the sense that it either returns an feasible solution (that meets each constraint up to ) or correctly declares that the problem is infeasible.
Input: noise vector
Output: vector that satisfies
or “infeasible” if there does not exist a vector for which
3.1 DualSubgradient MetaAlgorithm
In this section we assume that for all :

For all , the function is concave in ;

The set is convex.
Under these assumptions, the robust formulation is in fact a convexconcave saddlepoint problem that can be solved in polynomial time using interiorpoint methods. However, recall that our goal is to solve the robust problem by invoking a solver of the original (nonrobust) optimization problem.
In the setting of this section, we shall make use of the following definitions. Let be an upper bound over the diameter of , that is . Let be a constant such that for all and .
With the above assumptions and definitions, we can now present an oraclebased robust optimization algorithm, given in Algorithm 1. The algorithm is comprised of primaldual iterations, where the dual part of the algorithm updates the noise terms according to the current primal solution, via a lowregret update. For this algorithm, we prove:
Theorem 3.
Proof.
First, suppose that the algorithm returns “infeasible”. By the definition of the oracle , this happens if for some , there does not exists such that
This implies that the robust counterpart (3) cannot be feasible, as there exists an admissible perturbation that makes the original problem infeasible.
Next, suppose that a solution is returned. The premise of the oracle implies that for all and (otherwise, the algorithm would have returned “infeasible”), whence
(6) 
On the other hand, from the regret guarantee of the Online Gradient Descent algorithm we have
(7) 
Combining (10) and (12), we conclude that for all ,
where the final inequality follows from the convexity of the functions with respect to . Hence, for every we have
implying that is an approximate robust solution. ∎
3.2 DualPerturbation MetaAlgorithm
We now give our more general and intricate oraclebased approximation algorithm for RO. In contrast to the previous simple subgradientbased method, in this section we do not need the uncertainty structure to be convex. Instead, in addition to an oracle to solve the original mathematical program, we also assume the existence of an efficient “pessimization oracle” (as termed by Mutapcic and Boyd 2009), namely an oracle that approximates the worstcase noise for any given feasible solution . Formally, assume that for all the following hold:

For all , the function is linear in , i.e. can be written as for some functions and ;

There exists a linear optimization procedure that given a vector , computes a vector such that
On the surface, the linearity assumption seems very strong. However, note that we do not assume the convexity of the set . This means that the dual subproblem (that amounts to finding the worstcase noise for a given ) is not necessarily a convex program. Nevertheless, our approach can still approximate the robust formulation as long as a procedure is available.
In the rest of the section we use the following notations. Let be an upper bound over the diameter of , that is . Let and be constants such that and for all and .
We can now present our second oraclebased metaalgorithm, described in Algorithm 2. Similarly to our dualsubgradient method, the algorithm is based on primaldual iterations. However, in the dual part we now rely on the approximate pessimization oracle for updating the noise terms. This algorithm provides the following convergence guarantee.
Theorem 4.
We begin by analyzing the dual part of the algorithm, namely, the rule by which the variables are updated. While this rule is essentially an FPLlike update, we cannot apply Lemma 2 directly for two crucial reasons. First, the update uses an approximate linear optimization procedure instead of an exact one as required by FPL. Second, the reward vectors being observed by the dual algorithm are random variables that depend on its internal randomization (i.e., on the random variables ). Nevertheless, by analyzing a noisy version of the FPL algorithm (in Section 3.3 below) we can prove the following bound.
Lemma 5.
For each , with probability at least we have that
Proof.
Fix some . Note that the distribution from which the dual algorithm draws is a deterministic function of the primal variables . Hence, we can apply Lemma 4.1 of CesaBianchi and Lugosi (2006), together with the regret bound of Theorem 6 (see Section 3.3 below), and obtain that
(8) 
where denotes the expectation conditioned on . Next, note that the random variables for form a martingale differences sequence with respect to , and
Hence, by Azuma’s inequality (see e.g., Lemma A.7 in CesaBianchi and Lugosi 2006), with probability at least ,
(9) 
Equipped with the above lemma, we can now prove Theorem 4.
Proof of Theorem 4.
First, suppose that the algorithm returns “infeasible”. By the definition of the oracle , this happens if for some , there does not exists such that
This implies that the robust counterpart (3) cannot be feasible.
Next, suppose that a solution is returned (note that must lie in the set as we assume that is convex). This ensures that for all and (otherwise, the algorithm would have returned “infeasible”), whence
(10) 
On the other hand, Lemma 5 implies that for each we have
with probability at least . Recalling that for all and applying a union bound, we obtain that with probability at least ,
(11) 
Using our choice of now gives that with probability at least ,
(12) 
Combining (10) and (12), we conclude that with probability at least , for all ,
where the final inequality follows from the convexity of the functions with respect to . Hence, with probability at least , for every we have
implying that is an approximate robust solution. ∎
3.3 Follow the Approximate Perturbed Leader
As mentioned above, in our analysis we require a noisy version of the FPL algorithm, namely a variant capable of using an approximate linear optimization procedure over the decision domain rather than an exact one. Here we analyze such a variant and prove Theorem 6 being used in the proof of Lemma 5 above.
Assume we have a procedure for approximating linear programs over a (not necessarily convex) domain , that is, for all the output of satisfies
for some constant . We analyze the following version of the FPL algorithm: at round choose by first choosing a perturbation uniformly at random, and computing:
(13) 
We show that the error introduced by the noisy optimization procedure does not harm the regret too much. Formally, we prove:
Theorem 6.
For any sequence of reward vectors , let be the sequence of decisions produced by (13) with parameter . Then
where is an upper bound on the magnitude of the rewards, is an upper bound on the norm of the reward vectors, and is an upper bound on the diameter of .
Throughout this section we use the notation as a shorthand for the sum . Following the analysis of Kalai and Vempala (2005), we first prove that being the approximate leader yields approximately zero regret.
Lemma 7.
For any sequence of vectors ,
Proof.
The proof is by induction on . For the claim is trivial. Next, assuming correctness for some value of we have
which completes the proof. ∎
Next, we bound the regret of a hypothetical algorithm that on round uses the unobserved function for predicting .
Lemma 8.
For any vectors () and it holds that
where is an upper bound on the diameter of .
Proof.
Imagine a fictitious round in which a reward vector is observed. Then, using Lemma 7 we can write
Using the guarantee of , we can bound the first term on the right hand side as
Putting things together, for we have
where the final inequality follows from Hölder’s inequality, since and . ∎
Our final lemma bounds the expected difference in quality between the prediction made by the hypothetical algorithm to the one made by the approximate FPL algorithm.
Lemma 9.
For any sequence of reward vectors and distributed uniformly in the cube we have
where and .
Proof.
Lemma 3.2 in Kalai and Vempala (2005) shows that the cubes and overlap in at least fraction. On this intersection, the random variables and are identical. Otherwise, they can differ by at most . This gives the claim. ∎
We can now prove our regret bound.
Proof of Theorem 6.
Since we are bounding the expected regret, we can simply assume that with uniformly distributed in the cube . Combining the above lemmas, we see that
The claimed regret bound now follows from our choice of . ∎
4 Examples and Applications
In this section we provide several examples for the applicability of our results. All the problems we consider are stated as feasibility problems. For concreteness, we focus on ellipsoidal uncertainty sets, being the most common model of data uncertainty.
4.1 Robust Linear Programming
A linear program (LP) in the standard form is given by
s.t. 
The robust counterpart of this optimization problem is a secondorder conic program (SOCP) that can be solved efficiently, see e.g. BenTal et al. (2009); Bertsimas et al. (2011). In many cases of interest there exist highly efficient solvers for the original LP problem, as in the important case of network flow problems where the special combinatorial structure allows for algorithms that are much more efficient than generic LP solvers. However, this combinatorial structure is lost for its corresponding robust network flow problem. Hence, solving the robust problem using an oraclebased approach might be favorable in these cases. For the same reason, our technique is relevant even in the case of polyhedral uncertainty, where the robust counterpart remains an LP but possibly without the special structure of the original formulation.
In the discussion below, we assume that the feasible domain of the LP is inscribed in the Euclidean unit ball (this can be ensured via standard scaling techniques). Notice this also implies that the feasible domain of the corresponding robust formulation is inscribed in the same ball.
A robust linear program with ellipsoidal noise is given by:
s.t.  (14) 
where is a matrix controlling the shape of the ellipsoidal uncertainty, are the nominal parameter vectors, and is the dimensional Euclidean unit ball.
DualSubgradient Algorithm.
The robust linear program (4.1) is amenable to our OGDbased metaalgorithm (Algorithm 1), as the constraints are linear with respect to the noise terms . In this case we have , so that in each iteration of the algorithm, the update of the variables takes the simple form
Specializing Theorem 3 to the case of robust LPs, we obtain the following.
Corollary 10.
Proof.
DualPerturbation Algorithm.
Since the constraints of the robust LP are linear in the uncertainties , we can also apply our FPLbased metaalgorithm to the problem (4.1). Using the notations of Section 3.2, we have . Hence, the computation of the noise variables can be done in closedform, as follows:
In this case, Theorem 4 implies:
Corollary 11.
Proof.
4.2 Robust Quadratic Programming
A quadratically constrained quadratic program (QCQP) is given by
s.t. 
with , , . As in the case of LPs, we assume that the feasible domain of the above program is inscribed in the Euclidean unit ball. The robust counterpart of this optimization problem is a semidefinite program (BenTal et al., 2009; Bertsimas et al., 2011). Current stateoftheart QP solvers can handle two to three orders of magnitude larger QPs than SDPs, motivating our results. Indeed, our approach avoids reducing the robust program into an SDP and approximates it using a QP solver.
A robust QP with ellipsoidal uncertainties is given by^{3}^{3}3For simplicity, the uncertainties we consider here are only in the matrices (and not in the vectors ). In a similar, albeit more technical way we can also analyze our algorithm with general ellipsoidal uncertainties.
s.t.  (15) 
where are fixed matrices and . Here denotes the ’th entry of the noise vector .
Notice that Algorithm 1 does not apply to formulation (4.2), as the constraints are certainly not concave with respect to the noise terms (in fact, they are convex in , as we show below). This motivates the need for our FPLbased metaalgorithm.
DualPerturbation Algorithm.
We now show that the problem (4.2) falls in the scope of Section 3.2, and the assumptions required there hold for this program. We let denote the total magnitude of the admissible noise, and assume that the Frobenius norms of the nominal matrices are upper bounded by .
The following lemma shows that the ’th constraint is in fact a convex quadratic in .
Lemma 12.
The ’th constraint in (4.2) can be written as
where , and does not depend on . The matrix is a positive semidefinite with , and .
Proof.
Define and for . We have