Verifying Integer Programming Results
Abstract
Software for mixedinteger linear programming can return incorrect results
for a number of reasons, one being the use of inexact floatingpoint arithmetic.
Even solvers that employ exact arithmetic may suffer from programming or
algorithmic errors, motivating the desire for a way to produce independently
verifiable certificates of claimed results.
Due to the complex nature of stateoftheart MILP solution algorithms, the ideal form of such a
certificate is not entirely clear.
This paper proposes such a certificate format, illustrating its
capabilities and structure through examples.
The certificate format is designed with simplicity in mind and is composed of a
list of statements that can be sequentially verified using a limited number of
simple yet powerful inference rules.
We present a supplementary verification tool for compressing and checking these
certificates independently of how they were created. We report computational
results on a selection of mixedinteger linear programming instances from the
literature. To this end, we have extended the exact rational version of the MIP
solver SCIP to produce such certificates.
Keywords: correctness, verification, proof, certificate, optimality, infeasibility, mixedinteger linear programming
Kevin K. H. Cheung, Ambros Gleixner, and Daniel E. Steffy
[6ex] Verifying Integer Programming Results
Zuse Institute Berlin
Takustr. 7
14195 Berlin
Germany
Telephone: +49 30841850
Telefax: +49 3084185125
Email: bibliothek@zib.de
URL: http://www.zib.de
ZIBReport (Print) ISSN 14380064
ZIBReport (Internet) ISSN 21927782
1 Introduction
Let denote the following mixedinteger linear programming (MILP) problem:
where is a positive integer, , is a finite set of linear constraints on with rational coefficients, and .
Methods and techniques for solving instances of have been developed over the years. Solvers can now routinely handle largescale instances. As the size of the instances that can be solved and the complexity of the solvers increase, a question emerges: How does one know if the computational results are correct?
Though rare, MILP solvers do occasionally return incorrect or dubious results (see [11]). Despite such errors, maintaining a skeptical attitude that borders on paranoia is arguably neither healthy nor practical. After all, machines do outperform humans on calculations by many orders of magnitude and many tasks in life are now entrusted to automation. Hence, the motivation for asking how to verify correctness of computational results is not necessarily because of an inherent distrust of solvers. Rather, it is the desire to seek ways to identify and reduce errors and to improve confidence in the computed results. After all, if one believes in Murphy’s Law, errors tend to arise when one can least afford to have errors. Therefore, there is a practical need to improve the robustness and correctness of solvers. Previous research on computing accurate solutions to MIPs has utilized various techniques including interval arithmetic [33], exact rational arithmetic [4, 11, 15], and safely derived cuts [10]. Nevertheless, correctness cannot be guaranteed as attempts along this line still fall short of provablycorrect code using formal methods that are found in missioncritical applications such as software for medical applications and avionics. Incidentally, a footnote in [11] states: “even with a very careful implementation and extensive testing, a certain risk of an implementation error remains”.
One way to satisfy skeptics is to build solvers that output extra information that facilitates independent checking. We shall use the word certificate to refer to such extra information for a given problem that has been solved. Ideally, the certificate should allow for checking the results using fewer resources than what are needed to solve the problem from scratch. Such a certificate could in principle be used in formal verification using a proof checker as done in the Flyspeck Project [17, 35, 37] for a formal proof of Kepler’s Conjecture, or informal verification as done by Applegate et al. [3] for the Traveling Salesman Problem and by Carr et al. [9] in their unpublished work for MILP in general. Naturally, certificates should be as simple to verify as possible if they are to be convincing.
We highlight two specific applications where solution verification is desirable. First, Achterberg [1] presented MILP formulations for circuit design verification problems. For design verification, obtaining correct results is of critical importance; yet floatingpoint based solvers have been shown to return incorrect results on some of these instances [11]. Second, Pulaj [36] has recently used integer programming models to settle some open questions related to Frankl’s conjecture (the unionclosed sets conjecture). This, and other applications in pure mathematics, are also cases where correctness of the results is of the utmost importance. Software developed in connection with our paper has been successfully used to generate and check certificates for MILPs coming from both of these applications.
For linear programming (LP), duality theory tells us that an optimal primal solution and an optimal dual solution are sufficient to facilitate effective verification of optimality. In the case of checking infeasibility, a Farkas certificate will do. Therefore, verifying LP results, at least in the case when exact rational arithmetic is used, is rather straightforward. However, the situation with MILP is drastically different. From a theoretical perspective, even though some notions of duality for MILP have been formulated (see the survey [21]), small (i.e. polynomial size) certificates for infeasibility or optimality of MILPs may not even exist. As a result, there are many forms that certificates could take: a branchandbound tree, a list of derived cutting planes, a superadditive dual function, or other possibilities for problems with special structures such as pure integer linear programming and binary programming (see, for example, [8, 14, 28, 30]). Which format would be preferred for certificate verification is not entirely clear, and in this paper we provide reasoning behind our choice.
From a software perspective, MILP result certification is also considerably more complicated than LP certification. Even though most solvers adopt the branchandcut paradigm, they typically do not make the computed branchandbound tree or generated cuts readily available, and they may also utilize many other techniques including constraint propagation, conflict analysis, or reduced cost fixing. Thus, even if a solver did print out all information used to derive its solution, a verifier capable of interpreting such information would itself be highly complex, contradicting our desire for a simple verifier. As a result, other than accepting the results of an exact solver such as [11], the best that many people can do today to “verify” the results of a solver on a MILP instance is to solve the instance by several different solvers and check if the results match or minimally check that an optimal solution is indeed feasible and has the correct objective function value as done by the solution checker in [29].
The main contribution of this paper is the development of a certificate format for the verification of mixedinteger linear programs. Compared to the previous work of Applegate et al. [3] for the Traveling Salesman Problem and the unpublished work of Carr et al. [9] for general MILP, our certificate format has a significantly simpler structure. It consists of a sequence of statements that can be verified one by one using simple inference rules, facilitating verification in a manner akin to natural deduction. The approach is similar to that for verification of unsatisfiability proofs for SAT formulas. (See for example [24, 38].) This simple certificate structure makes it easier for researchers to develop their own independent certificate verification programs, or check the code of existing verifiers, even without any expert knowledge of MILP solution algorithms.
To demonstrate the utility of the proposed certificate format, we have developed a reference checker in C++ and added the capability to produce such certificates to the exact MIP solver [11] of SCIP [18]. We used these tools to verify results reported in [11]. To the best of our knowledge, this work also represents the first software for general MILP certificate verification that has been made available to the mathematical optimization community.
Organization of the paper. Even though the proposed format for the certificate is rather straightforward, some of the details are nevertheless technical. Therefore, in this paper, we discuss the certificate format at a conceptual level. The full technical specification is found in the accompanying computer files. We begin with the necessarily ingredients starting with the simple case of LP in Section 2. In Section 3, the ideas for dealing with LP are extended to pure integer linear programming. The full conceptual description of the format of the certificate is then given in Section 4. Computational experiments are reported in Section 5, and concluding remarks are given in Section 6.
Accompanying computer files. Software, technical documentation and certificate files are available at: https://github.com/ambrosgleixner/VIPR
2 Certificates for linear programming
Throughout this paper, we assume that problems are specified and solved with exact rational arithmetic.
A typical certificate of optimality for an LP is a dualfeasible solution whose objective function value matches the optimal value. However, there is no need to specify the dual when one views the task of certification as an inference procedure. (For instance, see [25].) To facilitate discussion, we call this inference procedure linear inequality inference and the details are as follows.
Let denote the system of linear constraints:
Here, is a vector of variables, , , , , , and for some nonnegative integers , , , and .
We say that is obtained by taking a suitable linear combination of the constraints in if
for some
,
, and
with
and .
Clearly, if satisfies , then it necessarily satisfies
.
We say that the inequality
is inferred from .
Remarks.

Together, is simply a feasible solution to the linear programming dual of the linear program given by:

Hooker [25] calls a surrogate of .
Suppose that an optimal solution to exists and the optimal value is . Linear programming duality theory guarantees that can be inferred from . Therefore, linear inequality inference is sufficient to certify optimality for linear programming. Conceptually, the certificate that we propose is a listing of the constraints in followed by the inequality with the associated multipliers used in the inference as illustrated in the following example.
Example 1.
The following shows an LP problem and its associated certificate.

Here, and are constraint labels. Taking the suitable linear combination gives , thus establishing that is a lower bound for the optimal value.
We remark that this type of linear inference could also be used to derive inequalities or equality constraints. Assuming that all problem data is rational, rational multipliers are sufficient to certify infeasibility or optimality.
3 Handling ChvátalGomory cutting planes
Gomory [20] showed in theory that, for pure integer linear programming (ILP), optimality or infeasibility can be established by a pure cuttingplane approach. Such an approach can also work in practice (see [6, 39]). In addition to linear inequality inference, a rounding operation is needed.
Suppose that can be inferred from by taking a suitable linear combination of the constraints. If for for some and for , then any satisfying with for must also satisfy . We say that is obtained from by rounding.
When , the inequality is known as a ChvátalGomory cut (CGcut in short). It can then be added to the system and the process of obtaining another CGcut can be repeated.
Conceptually, a certificate for an ILP instance solved using only CGcuts can be given as a list of the original constraints followed by the derived constraints.
Example 2.
The following shows an ILP problem and its associated certificate.

Note that the derived constraints in the certificate can be processed in a sequential manner. In the next section, we see how to deal with branching without sacrificing sequential processing.
4 Branchandcut certificates
In practice, most instances of are not solved by cutting planes alone. Thus, certificates as described in the previous section are of limited utility.
We now propose a type of certificate for optimality or infeasibility established by a branchandcut procedure in which the generated cuts at any node can be derived as split cuts and branching is performed on a disjunction of the form where and is integral for all feasible .
The use of split disjunctions allows us to consider branching and cutting under one umbrella. Many of the wellknown cuts generated by MILP solvers can be derived as split cuts (see [12]) and they are effective in closing the integrality gap in practice (see [16]). Branching typically uses only simple split disjunctions (where the above is a unit vector), although some studies have considered the computational performance of branching on general disjunctions [13, 27, 34].
Recall that each branching performed splits the solution space into two subcases. At the end of a branchandbound (or branchandcut) procedure, each leaf of the branchandbound tree corresponds to one of the cases and the leaves together cover all the cases needed to be considered. Hence, if the branchandbound tree is valid, all one needs to look at are the LP results at the leaves.
Our proposal is to “flatten” the branchandbound tree into a list of statements that can be verified sequentially. Thus, our approach departs from the approach in [3] and [9] which requires explicit handling of the branchandbound tree. The price we pay is that we can no longer just examine the leaves of the tree. Instead, we process the nodes in a bottomup fashion and discharge assumptions as we move up towards the root. We illustrate the ideas with an example.
Example 3.
It is known that the following has no solution.
Note that is an extreme point of the region defined by , , and . Branching on the integer variable leads to two cases:

;

.
We consider each case in turn.
Case 1.
Note that satisfies . We branch on :
Case 1a.
Taking gives the absurdity
Case 1b.
Taking gives the absurdity
Case 2.
Taking gives Rounding gives
Taking
gives the absurdity
As all cases lead to , we conclude that there is no solution. To issue a certificate as a list of derived constraints, we need a way to specify the different cases. To this end, we allow the introduction of constraints as assumptions.
Figure 1 shows conceptual certificate for the problem. Notice how the constraints , , , and are introduced to the certificate as assumptions. Since we want to end with without additional assumptions attached, we get there by gradually undoing the casesplitting operations. We call the undoing operation unsplitting. For example, and are both the absurdity with a common assumption . Since is true for all feasible , we can infer the absurdity assuming only in addition to the original constraints. We say that is obtained by unsplitting on . Similarly, both and are the absurdity and is true for all feasible , we can therefore unsplit on on to obtain without any assumption in addition to the original constraints.
In practice, the list of assumptions associated with each derived constraint needs not be prespecified as it can be deduced on the fly by a checker. For example, when processing , we see that it uses and , both of which are assumptions. Hence, we associate with the list of assumptions .
As any linear inequality can be introduced as an assumption, branching can in fact be on general disjunctions.
We now show conceptually how our proposed certificate format accommodates split cuts for . The discussion that follows focuses on split cuts at the root node but the ideas readily extend to split cuts at other nodes.
We say that dominates only if and , or and .
Let be such that for . Let . Recall that the inequality is a split cut if it is valid for the convex hull of The split cut can then be specified in the certificate as follows: Introduce as an assumption and list with the associated reason being dominated by a suitable linear combination of and Then introduce as an assumption and list with the reason being dominated by a suitable linear combination of and . Then list with as the reason. To verify , the following conditions must be checked:

and must be and , though not necessarily in that order, with and for all and for all .

dominates and dominates
The assumption list of contains all the assumptions of and except and .
5 Computational experiments
In this section, we describe software developed to produce and check
certificates for MILP results using the certificate format
developed in this paper.
Software and resources are provided at:
https://github.com/ambrosgleixner/VIPR
We again emphasize that the format was designed with simplicity in mind;
the certificate verification program we have provided is merely a reference
and others should be able to write their own verifiers without much difficulty,
even without knowledge of how MIP solvers operate.
The supporting documentation provides a more detailed technical description of the file format along with examples. A feature of the file format is that along with each derived constraint, the largest index of any derived constraint that references it is specified, thus allowing constraints to be freed from memory when they will no longer be needed as a certificate is being read and verified, leading to reduced memory consumption. The following C++ programs are provided:

viprchk. A program that verifies MILP results provided in our specified file format. All computations are performed in exact rational arithmetic using the GMP library [19].

viprttn. A program that performs simple modifications to “tighten” certificates. For each derived constraint, it computes the largest index over constraints referencing the target constraint to reduce memory required by a checker. It also has an option to remove unnecessary derived constraints.

vipr2html. A program that converts a certificate file to a “humanreadable” HTML file.
We also created a modified version of the exact rational MIP solver described in [11] and used it to compute certificates for several MIP instances from the literature. The exact rational MIP solver is based on SCIP [18] and uses a hybrid of floatingpoint and exact rational arithmetic to efficiently compute exact solutions using a pure branchandbound algorithm. In our experiments, the rational MIP solver uses CPLEX 12.6.0.0 [26] as its underlying floatingpoint LP solver and a modified version of QSopt_ex 2.5.10 [5] as its underlying exact LP solver. The exact MIP solver supports several methods for computing valid dual bounds and our certificate printing functionality is currently supported by the Projectandshift method (for dual solutions only) and the Exact LP method (for both dual solutions and Farkas proofs), for details on these methods see [11]. Future plans are to include certificate printing functionality in all dual bounding methods and release this in subsequent versions of exact SCIP; our developmental version is currently available from the authors by request.
In the following, we report some computational results on the time and memory required to produce and verify certificates. We considered the easy and numerically difficult (referred to here as ‘hard’) test sets from [11]; these test sets consist of instances from well known libraries including [2, 7, 29, 31, 32].
Experiments were conducted on a cluster of Intel(R) Xeon(R) CPU E52660 v3 at 2.60GHz; jobs were run exclusively to ensure accurate time measurement. Table 1 reports a number of aggregate statistics on these experiments. The columns under the heading SCIP report information on tests using the exact version of SCIP, using its default dual bounding strategy. The columns under SCIP+C report on tests involving the version of exact SCIP that generates certificates as it solves instances; since certificate printing is not supported for all dual bounding methods it uses only a subset of the dual bounding methods, contributing to its slower speed. Columns under the heading VIPR report time and memory usage for certificate checking. For each of the easy and hard test sets, we report information aggregated into four categories: ‘all’ reports statistics over all instances; ‘solved’ reports over instances solved by both SCIP and SCIP+C within a 1 hour time limit and a 10gb limit on certificate file size, ‘memout’ and ‘timeout’ report on instances where one of the solvers experienced a filesize limit or timeout. All averages are reported as shifted geometric means with a shift of 10 sec. for time and 1mb for memory. The column represents the number of instances in each category; represents the number in each category that were solved to optimality (or infeasibility) by a given solver; represents the time (sec.) used to solve the instance and, when applicable, output a certificate; is the time (sec.) required by the viprttn routine to tighten the certificate file; is the time (sec.) required to for viprchk to check the certificate file – on instances in the memout and timeout rows this represents the time to verify the primal and dual bounds present in the intermediate certificate printed before the solver was halted. The final three columns list the size of the certificate (in mb), before tightening, after tightening and then after being compressed to a gzipped file. The instance nw04 is excluded from the easy test set due to a memout by the viprttn routine. Timings and memory usage for individual instances are available in a document hosted together with the accompanying software.
SCIP  SCIP+C  VIPR  

Test set  
easyall  56  53  62.0  39  180.8  25.8  28.9  214  72  22 
solved  39  39  23.2  39  48.0  9.6  13.4  77  34  10 
memout  5  4  600.6  0  1769.4  377.5  169.8  10286  513  159 
timeout  12  10  357.4  0  3600.0  83.7  97.5  1151  368  108 
hardall  50  23  725.1  14  976.6  31.2  15.1  372  38  11 
solved  13  13  22.9  13  40.8  7.1  6.3  49  15  5 
memout  10  0  3600.0  0  1833.9  275.7  53.8  10269  146  39 
timeout  27  10  1811.1  1  3255.1  20.7  11.9  286  35  9 
From this table, we can make a number of observations. First, there is a noticeable, but not prohibitive, cost to generate the certificates. The differences in between SCIP and SCIP+C are due to both the difference in dual bounding strategies, and the overhead for writing the certificate files. In some additional experiments, we observed that on the 39 instances in the easysolved category, the file I/O amounted to roughly 7% of the solution time, based on this we believe that future modifications to the code will allow us to solve and print certificates in times much closer to those in the SCIP column. Perhaps most importantly, we observe that the time to check the certificates is significantly less than the time to solve the instances. Moreover, the certificate tightening program viprttn is able to make significant reductions in the certificate size, and the resulting certificate sizes are often surprisingly manageable.
6 Conclusion
This paper presented a certificate format for verifying integer programming results. We have demonstrated the practical feasibility of generating and checking such certificates on wellknown MIP instances. We see this as the first step of many in verifying the results of integer programming solvers. We now discuss some future directions made possible by this work.
Even in the context of floatingpoint arithmetic, our certificate format could serve a number of purposes. Using methods described by [10, 33], directed rounding and interval arithmetic may allow us to compute and represent valid certificates exclusively using floatingpoint data, allowing for faster computation and smaller certificate size. Additionally, generating approximate certificates with inexact data could be used for debugging solvers, or measuring the maximum or average numerical violation over all derivations. In a more rigorous direction, one could also convert our certificates to a form that could be formally verified by a proof assistant such as HOL Light [22].
Finally, we note that there are many potential ways to further simplify or optimize the certificates, beyond what is done by our viprttn routine. One natural adjustment is to reorder the deduction steps to minimize memory requirement by a checker, but much more is possible in this direction.
Acknowledgements.
We want to thank Kati Wolter for the exact version of SCIP [11] and Daniel Espinoza for QSopt_ex [5], which provided the basis for our computational experiments. We thank Gregor Hendel for his Ipet package [23], which was a big help in analyzing the experimental results. This work has been supported by the Research Campus MODAL Mathematical Optimization and Data Analysis Laboratories funded by the Federal Ministry of Education and Research (BMBF Grant 05M14ZAM). All responsibilty for the content of this publication is assumed by the authors.
References
 [1] Achterberg, T.: Constraint Integer Programming. PhD Thesis, TU Berlin (2007)
 [2] Achterberg, T., Koch, T., Martin, A.: The mixed integer programming library: MIPLIB 2003. Oper. Res. Lett. 34(4), 361–372 (2006)
 [3] Applegate, D.L., Bixby, R.E., Chvátal, V. Cook, W., Espinoza, D.G., Goycoolea, M., Helsgaun, K. Certification of an optimal TSP tour through 85,900 cities. Oper. Res. Lett. 37, 11–15 (2009)
 [4] Applegate, D.L., Cook, W.J., Dash, S., Espinoza, D.G.: Exact solutions to linear programming problems. Oper. Res. Lett. 35(6), 693â699 (2007)
 [5] Applegate, D.L., Cook, W.J., Dash, S., Espinoza, D.G.: QSopt_ex: http://www.math.uwaterloo.ca/~bico/qsopt/ex/ Last accessed on November 13, 2016
 [6] Balas, E., Fischetti, M., Zanette, A.: A hard integer program made easy by lexicography. Math. Program., Ser. A 135, 509–514 (2012)
 [7] Bixby, R.E., Ceria, S., McZeal, C.M., Savelsbergh, M.W.: An updated mixed integer programming library: MIPLIB 3.0. Optima 58, 12–15 (1998)
 [8] Boland, N.L. and Eberhard, A.C. On the augmented Lagrangian dual for integer programming. Math. Program., Ser. A 150(2), 491–509.(2015)
 [9] Carr, R., Greenberg, H., Parekh, O., Phillips, C.: Towards certificates for integer programming computations. Presentation, 2011 DOE Applied Mathematics PI meeting, October 2011. Slides www.csm.ornl.gov/workshops/applmath11/documents/talks/Phillips_talk.pdf Last accessed on November 13, 2016
 [10] Cook, W., Dash, S., Fukasawa, R., and Goycoolea, M.: Numerically safe Gomory mixedinteger cuts. INFORMS J. Comput., 21(4), 641â649 (2009)
 [11] Cook, W., Koch, T., Steffy, D., and Wolter, K.: A hybrid branchandbound approach for exact rational mixedinteger programming. Math. Program. Comput., 3, 305–344 (2013)
 [12] Cornuéjols, G.: Valid inequalities for mixed integer linear programs. Math. Program. Ser. B, 112, 3–44 (2008)
 [13] Cornuéjols, G., Liberti, L., Nannicini, G.: Improved strategies for branching on general disjunctions. Math. Program. Ser. A, 130, 225–247 (2011)
 [14] De Loera, J.A., Lee, J., Malkin, P.N., Margulies, S.: Computing infeasibility certificates for combinatorial problems through Hilbertâs Nullstellensatz J. Symbolic Comp., 46(11), 1260–1283 (2011)
 [15] Dhiflaoui, M., Funke, S., Kwappik, C., Mehlhorn, K., Seel, M., SchoÌmer, E., Schulte, R., Weber, D.: Certifying and repairing solutions to large LPs, how good are LPsolvers? In: SODA 2003, 255â256. ACM/SIAM, New York (2003)
 [16] Fukasawa, R. and Goycoolea, M.: On the exact separation of mixed integer knapsack cuts. Math. Program. Ser. A, 128, 19–41 (2008)
 [17] The Flyspeck Project. https://code.google.com/archive/p/flyspeck/. Last accessed on November 13, 2016.
 [18] Gamrath, G., Fischer, T., Gally, T., Gleixner, A.M., Hendel, G., Koch, T., Maher, S.J., Miltenberger, M., Müller, B, Pfetsch, M.E., Puchert, C., Rehfeldt, D., Schenker, S., Schwarz, R., Serrano, F., Shinano, Y., Vigerske, S., Weninger, D., Winkler, M., Witt, J.T., and Witzig, J. The SCIP Optimization Suite 3.2. ZIBreport (1560) (2016)
 [19] GNU MP: The GNU Multiple Precision Arithmetic Library version 6.1.1. http://gmplib.org. Last accessed on November 16, 2016.
 [20] Gomory, R.E.: Outline of an algorithm for integer solutions to linear programs. Bull. Amer. Math. Soc. 64, 275–278 (1958)
 [21] Guzelsoy, M. and Ralphs, T.K.: Duality for mixedinteger linear programs. Int. J. Oper. Res. 4(3), 118–137 (2007)
 [22] Harrison, J.: HOL Light: A tutorial introduction. Inter. Conf. on Formal Methods in Comp.Aided Design., 265–269 (1996)
 [23] Hendel, G.: Empirical analysis of solving phases in mixed integer programming. Master’s thesis, Technische Universität Berlin (2014). http://nbnresolving.de/urn:nbn:de:0297zib54270
 [24] Heule, M.J.H., Hunt, Jr., W.A., Wetzler, N.: Verifying refutations with extended resolution. In Conference on Automated Deductio (CADE), LNAI, 7898:345–359 (2013)
 [25] Hooker, J.N.: Integrated Methods for Optimization (2nd ed.), Springer, New York (2012)
 [26] IBM ILOG. CPLEX. https://www01.ibm.com/software/commerce/optimization/cplexoptimizer/. Last accessed on November 16, 2016.
 [27] Karamanov, M. and Cornuéjols, G.: Branching on general disjunctions. Math. Program. Ser. A, 128, 403–436 (2011)
 [28] Klabjan, D.: Subadditive approaches in integer programming. Eur. J. Oper. Res, 183, 525–545 (2007)
 [29] Koch, T., Achterberg, T., Andersen, E., Bastert, O., Berthold, T., Bixby, R.E., Danna, E., Gamrath, G., Gleixner, A.M., Heinz, S., Lodi, A., Mittelmann, H., Ralphs, T., Salvagnin, D., Steffy, D.E., Wolter, K.: MIPLIB 2010 Math. Program. Comp., 3(2), 103–163 (2011)
 [30] Lasserre, J.B.: Generating functions and duality for integer programs. Discrete Optim. 1(2), 167–187 (2004)
 [31] Lehigh University COR@L mixed integer programming collection. http://coral.ie.lehigh.edu/wiki/doku.php/info:datasets:mip. Last accessed on November 18, 2016.
 [32] Mittelmann, H.D.: Benchmarks for Optimization Software. http://plato.asu.edu/bench.html Last accessed on November 18, 2016.
 [33] Neumaier, A., Shcherbina, O.: Safe bounds in linear and mixedinteger linear programming. Math. Program. 99(2), 283â296 (2004)
 [34] Owen, J.H. and Mehrotra, S.: Experimental results on using general disjunctions in branchandbound for generalinteger linear programs. Comput. Optim. Appl. 20, 159–170 (2001)
 [35] Obua, S. and Nipkow, T.: Flyspeck II: the basic linear programs. Ann. Math. Artif. Intell 56, 245–272 (2009)
 [36] Pulaj, J.: Cutting Planes for Families Implying Frankl’s Conjecture. ZIBreport (1551), (2016)
 [37] Solovyev, A. and Hales, T.: Efficient formal verification of bounds of linear programs. Lecture Notes in Comp. Sci. 6824, 123–132 (2011)
 [38] Wetzler, N., Heule, M.J.H., Hunt, Jr., W.A.: DRATtrim: Efficient checking and trimming using expressive clausal proofs. In Theory and Applications of Satisfiability Testing (SAT), LNCS 8561, 422–429 (2014)
 [39] Zanette, A., Fischetti, M., Balas, E.: Lexicography and degeneracy: can a pure cutting plane algorithm work? Math. Program., Ser. A 130, 153–176 (2011)