A multicriteria approach to approximate solution of multiplechoice knapsack problem
Abstract
We propose a method for finding approximate solutions to multiplechoice knapsack problems. To this aim we transform the multiplechoice knapsack problem into a biobjective optimization problem whose solution set contains solutions of the original multiplechoice knapsack problem. The method relies on solving a series of suitably defined linearly scalarized biobjective problems. The novelty which makes the method attractive from the computational point of view is that we are able to solve explicitly those linearly scalarized biobjective problems with the help of the closedform formulae.
The method is computationally analyzed on a set of largescale problem instances (test problems) of two categories: uncorrelated and weakly correlated. Computational results show that after solving, in average 10 scalarized biobjective problems, the optimal value of the original knapsack problem is approximated with the accuracy comparable to the accuracies obtained by the greedy algorithm and an exact algorithm. More importantly, the respective approximate solution to the original knapsack problem (for which the approximate optimal value is attained) can be found without resorting to the dynamic programming. In the test problems, the number of multiplechoice constraints ranges up to hundreds with hundreds variables in each constraint.
Keywords:
Knapsack Multiobjective optimization Multiplechoice knapsack Linear scalarization1 Introduction
The multidimensional multiplechoice knapsack problem and the multiplechoice knapsack problem are classical generalizations of the knapsack problem and are applied to modeling many reallife problems, e.g., in project (investments) portfolio selection Nauss1978 (); Zhong2010 (), capital budgeting Pisinger2001 (), advertising Sinha1979 (), component selection in IT systems Kwong2010 (); Pyzel2012 (), computer networks management Lee1999 (), adaptive multimedia systems Khan1998 (), and other.
The multiplechoice knapsack problem is formulated as follows. Given are sets , ,…, of items, of cardinality , . Each item of each set has been assigned realvalued nonnegative ’profit’ and ’cost’ , , .
The problem consists in choosing exactly one item from each set so that the total cost does not exceed a given and the total profit is maximized.
Let , , , be defined as
Note that all form a vector of length , , and we write
In this paper, we adopt the convention that a vector is a column vector, and hence the transpose of , denoted by , is a row vector.
Problem is of the form
By using the above notations, problem can be equivalently rewritten in the vector form
where and are vectors from ,
and for any vectors , the scalar product is defined in the usual way as .
The feasible set to problem is defined by a single linear inequality constraint and the constraint , i.e.,
and finally
The optimal value of problem is equal to and the solution set is given as
Problem is hard. The approaches to solving can be: heuristics Akbar2006 (); Hifi2004 (), exact methods providing upper bounds for the optimal value of the profit together with the corresponding approximate solutions Sbihi2007 (), exact methods providing solutions Martello2000 (). There are algorithms that efficiently solve without sorting and reduction Dyer1984 (); Zemel1984 () or with sorting and reduction Dudzinski1984 (). Solving with a linear relaxation (by neglecting the constrains ) gives upper bounds on the value of optimal profit. Upper bounds can be also obtained with the help of the Lagrange relaxation. These facts and other features of are described in details in monographs Kellerer2004 (); Martello1990 ().
Exact branchandbound methods Dyer1984a () (integer programming), even those using commercial optimization software (e.g., LINGO, CPLEX) can have troubles with solving large problems. A branchandbound algorithm with a quick solution of the relaxation of reduced problems was proposed by Sinha and Zoltners Sinha1979 (). Dudziński and Walukiewicz proposed an algorithm with pseudopolynomial complexity Dudzinski1987 ().
Algorithms that use dynamic programming require integer values of data and for largescale problems require large amount of memory for backtracking (finding solutions in set ), see also the monograph Martello1990 (). The algorithm we propose does not need the data to be integer numbers.
Heuristic algorithms, based on solving linear (or continuous) relaxation of and dynamic programming Dyer1995 (); Pisinger1995 (); Pisinger2001 () are reported to be fast, but have limitations typical for dynamic programming.
The most recent approach ”reduce and solve” Chen2014 (); Gao2016 () is based on reducing the problem by proposed pseudo cuts and then solving the reduced problems by a Mixed Integer Programming solver.
In the present paper, we propose a new exact (not heuristic) method which provides approximate optimal profits together with the corresponding approximate solutions. The method is based on multiobjective optimization techniques. Namely, we start by formulating a linear biobjective problem related to the original problem . After investigating the relationships between and problems, we propose an algorithm for solving via a series of scalarized linear biobjective problems .
The main advantage of the proposed method is that the scalarized linear biobjective problems can be explicitly solved by exploiting the structure of the set . Namely, these scalarized problems can be decomposed into independent subproblems the solutions of which are given by simple closedform formulas. This feature of our method is particularly suitable for parallelization. It allows to generate solutions of scalarized problems in an efficient and fast way.
The experiments show that the method we propose generates very quickly an outcome which is an approximate solution to . Moreover, lower bound (LB) and upper bound (UB) for the optimal profit are provided.
The obtained approximate solution could serve as a good starting point for other, e.g., heuristic or exact algorithms for finding an optimal solution to the problem .
The organization of the paper is as follows. In Section 2, we provide preliminary facts on multiobjective optimization problems and we formulate a biobjective optimization problem associated with . In Section 3, we investigate the relationships between the problem and the original problem . In Section 4, we formulate scalarized problems for biobjective problem and we provide closedform formulae for solutions to problems by decomposing them into independent subproblems , . In Section 5, we present our method (together with the pseudocode) which provides a lower bound (LB) for the optimal profit together with the corresponding approximate feasible solution to for which the bound (LB) is attained. In Section 6, we report on the results of numerical experiments. The last section concludes.
2 Multiobjective optimization problems
Let , , be functions defined on and be a subset in .
The multiobjective optimization problem is defined as
where the symbol means that solutions to problem are understood in the sense of Pareto efficiency defined in Definition 2.1.
Let
Definition 2.1
A point is a Pareto efficient (Pareto maximal) solution to if
In other words, is a Pareto efficient solution to if there is no such that
(1) 
The problem where all the functions , are linear is called a linear multiobjective optimization problem.
Remark 2.1
The biobjective problem
with Pareto solutions defined as
(2) 
where
is equivalent to the problem
in the sense that Pareto efficient solution sets (as subsets of the feasible set ) coincide and Pareto elements (the images in of Pareto efficient solutions) differ in sign in the second component.
2.1 A biobjective optimization problem related to
In relation to the original multiplechoice knapsack problem , we consider the linear biobjective binary optimization problem of the form
In this problem the lefthand side of the linear inequality constraint of becomes a second criterion and the constraint set reduces to the set . There are twofold motivations of considering the biobjective problem .
First motivation comes from the fact that in the inequality
is usually seen as a budget (in general: a resource) constraint with the lefthandside to be preferably not greater than a given available budget . In the biobjective problem , this requirement is represented through the minimization of . In Theorem 3.1 of Section 3, we show that under relatively mild conditions among solutions of the biobjective problem (or the equivalent problem ) there are solutions to problem .
Second motivation is important from the algorithmic point of view and is related to the fact that in the proposed algorithm we are able to exploit efficiently the specific structure of the constraint set which contains linear equality constraints (each one referring to a different group of variables) and the binary conditions only. More precisely, the set can be represented as the Cartesian product
(3) 
of the sets , where , and
(4) 
i.e.,
and . Accordingly,
Note that due to the presence of the budget inequality constraint the feasible set of problem cannot be represented in the form analogous to (3).
According to Remark 2.1, problem can be equivalently reformulated in the form
3 The relationships between and
Starting from the multiplechoice knapsack problem of the form
in the present section we analyse relationships between problems and .
We start with a basic observation. Recall first that is solvable, i.e., the feasible set is nonempty if
On the other hand, if , is trivially solvable. Thus, in the sequel we assume that
(5) 
Let , i.e., is the maximal value of the function on the set . The following observations are essential for further considerations.

First, among the elements of which realize the maximal value , there exists at least one which is feasible for , i.e., there exists , such that , i.e.,
(6) Then, clearly, solves .

Second, none of elements which realize the maximal value is feasible for , i.e., for every , we have , i.e., any realizing the maximal value is infeasible for , i.e.
(7)
In the sequel, we concentrate on Case 2, characterized by (7). This case is related to problem . To see this let us introduce some additional notations. Let and be defined as
Let be the set of all Pareto solutions to the biobjective problem ,
(c.f. Definition 2.1). The following lemma holds.
Lemma 3.1
Assume that we are in Case 2, i.e., condition holds. There exists a Pareto solution to the biobjective optimization problem , which is feasible to problem , i.e. which amounts to .
Proof
Now we are ready to formulate the result establishing the relationship between solutions of and Pareto efficient solutions of in the case where the condition (7) holds.
Theorem 3.1
Suppose we are given problem satisfying condition (7). Let be a Pareto solution to , such that
Then solves .
Proof
Observe first that, by Lemma 3.1, there exist satisfying the constraint , i.e., condition is not dummy.
By contradiction, suppose that a feasible element , i.e., , , is not a solution to , i.e., there exists an , such that
We show that cannot satisfy condition . If , then is not a Pareto solution to , i.e., , and does not satisfy condition . Otherwise, , i.e.,
(9) 
If , then because , then, according to (9), cannot satisfy condition .
If , there exists which dominates , i.e., . Again, if , then is not a Pareto solution to , i.e. cannot satisfy condition . Otherwise, if , then either and consequently cannot satisfy condition , or , in which case and does not satisfy condition , a contradiction which completes the proof.
Theorem 3.1 says that under condition (7) any solution to satisfying condition solves problem . General relations between constrained optimization and multiobjective programming were investigated in Klamroth2007 ().
Basing ourselves on Theorem 3.1, in Section 5 we provide an algorithm for finding , a Pareto solution to , which is feasible to problem and for which the condition is either satisfied or is, in some sense, as close as possible to be satisfied. In this latter case, the algorithm provides upper and lower bounds for the optimal value of .
4 Decomposition of the scalarized biobjective problem
In the present section, we consider problem defined by (10) which is a linear scalarization of problem . In our algorithm BISSA, presented in Section 5, we obtain an approximate feasible solution to by solving a (usually very small) number of problems of the form . The main advantage of basing our algorithm on problems is that they are explicitly solvable by simple closedform expressions (17) .
For problem the following classical scalarization result holds.
Theorem 4.1
Ehrgott2005 (); Miettinen1999 () If there exist , , such that is a solution to the scalarized problem
(10) 
then is a Pareto efficient solution to problem .
Without loosing generality we can assume that . In the sequel, we consider, for , scalarized problems of the form
(11) 
Remark 4.1
According to Theorem 4.1, solutions to problems
(12) 
need not be Pareto efficient because the weights are not both positive. However, there exist Pareto efficient solutions to among solutions to these problems.
Namely, there exist and such that solutions to problems
and
are Pareto efficient solutions to problems (12), respectively. Suitable and will be determined in the next subsection.
4.1 Decomposition
Due to the highly structured form of the set and the possibility of representing in the form (3),
we can provide explicit formulae for solving problems . To this aim we decompose problems as follows.
Let . According to (3) we have
for . Consider problems , , of the form
(13) 
By solving problems , , we find their solutions . We shall show that
solves . Thus, problem (11) is decomposed into subproblems (13), the solutions of which form solutions to (11).
Note that similar decomposed problems with feasible sets and another objective functions have already been considered in Cherfi2010 () in relation to multidimensional multiplechoice knapsack problems.
Now we give a closedform formulae for solutions of . For , let
(14) 
and let be the index number for which the value is attained, i.e.,
(15) 
We show that
(16) 
is a solution to and
(17) 
is a solution to . The optimal value of is
(18) 
Namely, the following proposition holds.
Proof
Clearly, are feasible for , , because is of the form (16) and hence belongs to the set which is the constraint set of . Consequently, defined by (17) is feasible for because all the components are binary and the linear equality constraints
are satisfied.
To see that are also optimal for , , suppose by the contrary, that there exists and an element which is feasible for with the value of the objective function strictly greater than the value at , i.e.,
This, however, would mean that there exists an index such that
contrary to the definition of .
To see that is optimal for , suppose by the contrary, that there exists an element which is feasible for and the value of the objective function at is strictly greater than the value of the objective function at , i.e.,
In the same way as previously, we get the contradiction with the definition of the components of given by (17).
Let us observe that each optimization problem can be solved in time , hence problem can be solved in time , where .
Clearly, one can have more than one solution to , . In the next section, according to Theorem 3.1, from among all the solutions of we choose the one for which the value of the second criterion is greater than and as close as possible to .
Note that by using Proposition 4.1, one can easily solve problems and defined in Remark 4.1, i.e., by applying (18) we immediately get
the optimal values of and and by (17), we find their solutions and , respectively.
Proposition 4.1 and formula (17) allows to find and as defined in Remark 4.1. By (17), it is easy to find elements such that
Put
and let
where and denote the smallest nonzero decrease on of functions and from and , respectively. Note that and can easily be found.
Remark 4.2
The following formulas describe and ,
(19) 
where and , , are defined by (4), , , , are submaximal values of functions , , , .
For any , the submaximal values of a linear function on can be found by: ordering first the coefficients of the function decreasingly,
and next observing that the submaximal (i.e., smaller than maximal but as close as possible to the maximal) value of on is attained for
Basing on Remark 4.2 one can find values of and in time , , even without any sorting. It can be done for a given by finding a maximal value among all (), , except (). Therefore the computational cost of calculating and is .
We have the following fact.
Proposition 4.2
Let , , , , , be as defined above. The problems
and
where
(20) 
give Pareto efficient solutions to problem , and , respectively. Moreover,
i.e., solve problems (12), respectively.
5 Biobjective Approximate Solution Search Algorithm for solving
In this section, we propose the biobjective approximate solution search algorithm , for finding an element which is an approximate solution to . The algorithm relies on solving a series of problems defined by (11) for chosen in the way that the Pareto solutions to are feasible for and for which and