Reduction of the Pareto Set in Bicriteria Asymmetric Traveling Salesman Problem
We consider the bicriteria asymmetric traveling salesman problem (bi-ATSP). Optimal solution to a multicriteria problem is usually supposed to be the Pareto set, which is rather wide in real-world problems. We apply to the bi-ATSP the axiomatic approach of the Pareto set reduction proposed by V. Noghin. We identify series of “quanta of information” that guarantee the reduction of the Pareto set for particular cases of the bi-ATSP. An approximation of the Pareto set to the bi-ATSP is constructed by a new multi-objective genetic algorithm. The experimental evaluation carried out in this paper shows the degree of reduction of the Pareto set approximation for various “quanta of information” and various structures of the bi-ATSP instances generated randomly.
Keywords:Reduction of the Pareto set Multi-objective genetic algorithm Computational experiment
The asymmetric traveling salesman problem (ATSP) is one of the most popular problems in combinatorial optimization . Given a complete directed graph where each arc is associated with a positive weight, we search for a circuit visiting every vertex of the graph exactly once and minimizing the total weight. In this paper, we consider the bicriteria ATSP (bi-ATSP) which is a special case of the multicriteria ATSP , where an arc is associated to a couple of weights.
The best possible solution to a multicriteria optimization problem (MOP) is usually supposed to be the Pareto set [5, 21], which is rather wide in real-world problems, and difficulties arise in choosing the final variant. For that reason numerous methods introduce some mechanism to treat the MOP: utility function, rule, or binary relation, so that methods are aimed at finding an “optimal” solution with respect to this mechanism. However, some approaches do not guarantee that the obtained solution will be from the Pareto set. State-of-the-art methods are the following : multiattribute utility theory, outranking approaches, verbal decision analysis, various iterative procedures with man-machine interface, etc. In this paper, we investigate the axiomatic approach of the Pareto set reduction proposed in  which has an alternative idea. Here the author introduced an additional information about the decision maker (DM) preferences in terms of the so-called “quantum of information”. The method shows how to construct a new bound of the optimal choice, which is narrower than the Pareto set. Practical applications of the approach could be found in [11, 20].
As far as we know, the axiomatic approach of the Pareto set reduction has not been widely investigated in the case of discrete optimization problems, and an experimental evaluation has not been carried out on real-world instances. We apply this approach to the bi-ATSP in order to estimate its effectiveness, i.e. the degree of the Pareto set reduction and how it depends on the parameters of the information about DM’s preferences. We identify series of “quanta of information” that guarantee the reduction of the Pareto set for particular cases of the bi-ATSP.
Originally the reduction is constructed with respect the Pareto set of the considered problem. Due to the strongly NP-hardness of the bi-ATSP we take an approximation of the Pareto set in computational experiments. The ATSP cannot be approximated with any constant or exponential approximation factor already with a single objective function . Moreover, in , the non-approximability bounds were obtained for the multicriteria ATSP with weights 1 and 2. The results are based on the non-existence of a small size approximating set. Therefore, metaheuristics, in particular multi-objective evolutionary algorithms (MOEAs), are appropriate to approximate the Pareto set of the bi-ATSP.
Numerous MOEAs have been proposed to MOPs (see e.g. [3, 4, 14, 28, 30, 31]). There are three main classes of approaches to develop MOEAs, which are known as Pareto-dominance based (see e.g. SPEA2 , NSGA-II [3, 4], NSGA-III ), decomposition based (see e.g. MOEA/D ) and indicator based approaches (see e.g. SIBEA ). NSGA-II  has one of the best results in the literature on multi-objective genetic algorithms (MOGAs) for the MOPs with two or three objectives. In , a fast implementation of a steady-state version of NSGA-II is proposed for two dimensions.
In [9, 22], NSGA-II was adopted to the multicriteria symmetric traveling salesman problem, and the experimental evaluation was performed on symmetric instances from TSPLIB library . To the best of our knowledge, there is no adaptation of NSGA-II to the more general problem, where arc weights are non-symmetric. In this paper, we propose a new MOGA based on NSGA-II to solve the bi-ATSP using adjacency-based representation of solutions. A computational experiment is carried out on randomly generated instances. The results of the experiment show the degree of the reduction of the Pareto set approximation for various “quanta of information” and various structures of the problem instances.
2 Problem Statement
An instance of the traveling salesman problem  (TSP) is given by a complete graph , where is the set of vertices and set contains arcs (or edges) between every pair of vertices in . Each arc (or edge) is associated with a weight . The aim is to find a Hamiltonian circuit (also called a tour) of minimum weight, where the weight of a tour is the sum of its arc (or edge) weights . We denote by all possible tours of graph . If graph is undirected, we have Symmetric TSP (STSP). If is a directed graph, then we have Asymmetric TSP (ATSP).
In many situations, however, there is more than one objective function (criterion) to optimize [5, 21]. In case of the TSP, we might want to minimize the travel distance, the travel time, the expenses, the number of flight changes, etc. This gives rise to a multicriteria TSP, where Hamiltonian circuits are sought that optimize several objectives simultaneously. For the -criteria TSP, each arc (or edge) has a weight , which is a vector of length (instead of a scalar). The total weight of a tour is also a vector , where . Given this, the goal of the optimization problem could be the following: find a feasible solution which simultaneously minimizes each coordinate. Unfortunately, such an ideal solution rarely exists since objective functions are normally in conflict.
We say that one solution (tour) dominates another solution if the inequality holds. The notation means that and for all , where . This relation is also called the Pareto relation. A set of non-dominated solutions is called the set of pareto-optimal solutions [5, 21] In discrete problems, the set of pareto-optimal solutions is non-empty if the set of feasible solutions is non-empty, which is true for the multicriteria TSP. If we denote , then the Pareto set is defined as We assume that the Pareto set is specified except for a collection of equivalence classes, generated by equivalence relation iff .
In this paper, we investigate the issue of the Pareto set reduction for the bi-ATSP.
3 Pareto Set Reduction
Axiomatic approach of the Pareto set reduction is applied to both discrete and continuous problems. Due to consideration of the multicriteria ATSP we formulate the basic concepts and results of the approach in terms of notations introduced in Section 2. Further, we investigate properties of the bi-ATSP in the scope of the Pareto set reduction.
3.1 Main Approach
According to  we consider the extended multicriteria problem :
a set of all possible tours ;
a vector criterion defined on set ;
an asymmetric binary preference relation of the DM defined on set .
The notation means that the DM prefers the solution to .
Binary relation satisfies some axioms of the so-called “reasonable” choice, according which it is irreflexive, transitive, invariant with respect to a linear positive transformation and compatible with each criteria . The compatibility means that the DM is interested in decreasing value of each criterion when values of other criteria are constant. Also, if for some feasible solutions the relation holds, then tour does not belong to the optimal choice within the whole set .
In , the author established the Edgeworth–Pareto principle: under axioms of “reasonable” choice any set of selected outcomes belongs to the Pareto set . Here the set of selected outcomes is interpreted as some abstract set corresponded to the set of tours, that satisfy all hypothetic preferences of the DM. So, the optimal choice should be done within the Pareto set only if preference relation fulfills the axioms of “reasonable” choice.
In real-life multicriteria problems the Pareto set is rather wide. For this reason V. Noghin proposed a specific information on the DM’s preference relation to reduce the Pareto set staying within the set of selected outcomes [18, 19]:
We say that there exists a “quantum of information” about the DM’s preference relation if vector such that , , for all satisfies the expression . In such case we will say, that the component of criteria is more important than the component with given positive parameters , .
Thus, “quantum of information” shows that the DM is ready to compromise by increasing the criterion by amount for decreasing the criterion by amount . The quantity of relative loss is set by the so-called coefficient of relative importance , therefore .
As mentioned before the relation is invariant with respect to a linear positive transformation. Hence Definition 1 is equivalent to the existence of such vector with components , , for all that the relation holds. Further, in experimental study (Section 5) we consider “quantum of information” exactly in terms of coefficient .
In , the author established the rule of taking into account “quantum of information”. This rule consists in constructing a “new” vector criterion using the components of the “old” one and parameters of the information , . Then one should find the Pareto set of “new” multicriteria problem with the same set of feasible solutions and “new” vector criterion. The obtained set will belong to the Pareto set of the initial problem and give a narrower upper bound on the optimal choice, as a result the Pareto set will be reduced.
The following theorem states the rule of applying “quantum of information” and specifies how to evaluate “new” vector criterion upon the “old” one.
Theorem 3.1 ()
Given a “quantum of information” by Definition 1, the inclusions are valid for any set of selected outcomes . Here , and is the set of pareto-optimal solutions with respect to -dimensional vector criterion , where , for all .
3.2 Pareto Set Reduction in Bi-ATSP
Here we consider the bi-ATSP and its properties with respect to reduction of the Pareto set.
Obviously, the upper bound on the cardinality of the Pareto set is , and this bound is tight . In  authors established the maximum number of elements in the Pareto set for any multicriteria discrete problem, that in the case of the bi-ATSP gives the following upper bound: , where is the number of different values in the set , . In the case of the bi-ATSP with integer weights we get , where values and can be replaced by upper and lower bounds on the objective function , .
Now, we go to establish theoretical results estimating the degree of the Pareto set reduction. Let us consider the case when all elements of the Pareto set lay on principal diagonal of some rectangle in the criterion space.
Let , where and are arbitrary positive constants. Suppose the 1st criterion is more important than the 2nd one with coefficient of relative importance . If , then the reduction of the Pareto set consists of only one element. In the case of the reduction does not hold, i.e. .
Let in Theorem 3.2, otherwise, the 2nd criterion is more important than the 1st one with coefficient of relative importance . Then the reduction of the Pareto set has only one element if , and if .
Obviously, for any bi-ATSP instance there exists the minimum number of parallel lines with a negative slope, that all elements of the Pareto set belong to them. Thus we have
Let , where , , and are arbitrary positive constants. If criterion is more important than criterion with coefficient of relative importance and , or criterion is more important than criterion with coefficient of relative importance and , then .
Further, we identify the condition that guarantees excluding at least one element from the Pareto set.
Let the criterion is more important than the criterion with coefficient of relative importance . Suppose that there exist such tours that the following inequality holds:
then . Here , .
The difficulty in checking inequality (1) is that we should know two elements of the Pareto set. Meanwhile the tours , are pareto-optimal by definition.
4 Multi-Objective Genetic Algorithm
The genetic algorithm is a random search method that models a process of evolution of a population of individuals . Each individual is a sample solution to the optimization problem being solved. Individuals of a new population are built by means of reproduction operators (crossover and/or mutation).
4.1 NSGA-II Scheme
To construct an approximation of the Pareto set to the bi-ATSP we develop a MOGA based on Non-dominated Sorting Genetic Algorithm II (NSGA-II) . The NSGA-II is initiated by generating random solutions of the initial population. Then the population is sorted based on the non-domination relation (the Pareto relation). All individuals of the population which are not dominated by any other individual compose the first non-dominated level and are marked with the rank of 1, all individuals which are dominated by at least one individual of the rank compose the -th non-dominated level and are marked with the rank of , . To get an estimate of the density of solutions surrounding a solution in a non-dominated level of the population, two nearest solutions on each side of this solution are identified for each of the objectives. The estimation of solution is called crowding distance and it is computed as a normalized perimeter of the cuboid formed in the criterion space by the nearest neighbors.
The NSGA-II is characterized by the population management strategy known as generational model . Here the next population is constructed from the best solutions of the current population and an offspring population created from by applying selection, crossover, and mutation. The best solutions are selected using the rank and the crowding distance. Between two solutions with differing non-domination ranks, we prefer the solution with the lower rank. If both solutions belong to the same level, then we prefer the solution with the bigger crowding distance. The formal scheme of the NSGA-II is as follows:
Non-dominated Sorting Genetic Algorithm II
Step 1. Construct the initial population of size and assign . The population is sorted based on the non-domination relation. The crowding distances of individuals are calculated.
Step 2. Repeat steps 2.1-2.4 until some stopping criterion is satisfied:
2.1. Create offspring population .
Steps 2.1.1-2.1.4 are performed times:
2.1.1. Choose two parent individuals from the population.
2.1.2. Apply mutation to and and obtain individuals .
2.1.3. Create an offspring , applying a crossover to and .
2.1.4. Put individual into population .
2.2. Form a combined population . The population is sorted based on the non-domination relation. The crowding distances of individuals are calculated.
2.3. Construct population from the best individuals of population using the rank and the crowding distance to select solutions.
2.4. Set .
One iteration of the presented NSGA-II is performed in time as shown in . In our implementation of the NSGA-II four individuals of the initial population are constructed by a problem-specific heuristic presented in  for the ATSP with one criterion. The heuristic first solves the Assignment Problem, and then patches the circuits of the optimum assignment together to form a feasible tour in two ways. So, we create two solutions with each of the objectives. All other individuals of the initial population are generated randomly.
Each parent on Step 2.1.1 is chosen by -tournament selection: sample randomly individuals from the current population and select the best one by means of the rank and the crowding distance.
4.2 Recombination and Mutation Operators
The experimental results of [7, 27] for the TSP indicate that reproduction operators with the adjacency-based representation of solutions have an advantage over operators, which emphasize the order or position of the vertices in parent solutions. We suppose that a feasible solution to the bi-ATSP is encoded as a list of arcs. In the recombination operator on Step 2.1.3 we use a variant of the Directed Edge Crossover (DEC), which may be considered as a “direct descendant” of Edge Crossover  originally developed for the STSP.
The DEC operator is respectful , i.e. all arcs shared by both parents are copied into the offspring. The remaining arcs are selected so as the preference is given to those arcs that are contained in at least one of the parents. Arcs are inserted taking into account the non-violation of sub-tour elimination constraints. If the obtained offspring is equal to one of the parents, then the result of the recombination is calculated by applying the well-known shift mutation  to one of the two parents with equal probability. This approach allows us to avoid creating a clone of parents and to maintain a diverse set of solutions in the population.
The mutation is also applied to each parent on Step 2.1.2 with probability , which is a tunable parameter of the MOGA. We use a mutation operator proposed in  for the one-criteria ATSP. It performs a random jump within 3-opt neighborhood, trying to improve a parent solution in terms of one of the criteria. Each time one of two objectives is used in mutation with equal probability.
5 Computational Experiment
This section presents the results of the computational experiment on the bi-ATSP instances. Our MOGA (NSGA-II-biATSP) was programmed in C++ and tested on a computer with Intel Core i5 3470 3.20 GHz processor, 4 Gb RAM.
Various meta-heuristics and heuristics have been developed for the multicriteria STSP, such as Pareto local search algorithms, MOEAs, multi-objective ant colony optimization methods, memetic algorithms and others (see, e.g., [9, 10, 13, 15, 22]). However, we have not found in the literature any multi-objective metaheuristic proposed specifically to the multicriteria ATSP and experimentally tested on instances with non-symmetric weights of arcs.
We carried out the preliminary study to evaluate the performance of our GA on bi-ATSP instances generated randomly with . The Pareto sets were found by an exact algorithm . The generational distance  and the inverted generational distance  were involved as performance metrics. The experimental evaluation showed that the proposed MOGA yields competitive results. The values of metrics decrease not less than 7 times during iterations, and the final values are approximately on average. The number of elements in the final approximation is at least of . This indicates the convergence of the approximation obtained by NSGA-II-biATSP to the Pareto set and its diversity. Here the detailed description of the preliminary study is omitted, as the main goal of the paper is to investigate the axiomatic approach of the Pareto set reduction in the case of bi-ATSP.
Note that there exists MOOLIBRARY library , which contains instances of some discrete multicriteria problems. However, the multicriteria TSP is not presented in this library, so we generate the bi-ATSP test instances randomly and construct them from the ATSP instances of TSPLIB library, as well.
The reduction of the Pareto set approximation was tested on the following medium-size problem instances of four series with : S50[1,10][1,10], S50[1,20][1,20], S50[1,10][1,20], S50contr[1,2][1,2]. Each series consists of five problems with integer weights and of arcs randomly generated from intervals specified at the ending of the series name. In series S50contr[1,2][1,2] the criteria contradict each other with coefficient , i.e. weights are generated so that for all . We also took seven ATSP instances of series ftv from TSPLIB library : ftv33, ftv35, ftv38, ftv44, ftv47, ftv55, ftv64. The ftv collection includes instances from vehicle routing applications . These instances compose series denoted by SftvRand, and their arc weights are used for the first criterion. The arc weights for the second criterion are generated randomly from interval , were is the maximum arc weight on the first criterion. We set the population size , the tournament size , and the mutation probability . To construct an approximation of the Pareto set for each instance we run NSGA-II-bi-ATSP once and the run continued for iterations.
We compare two cases when the 1st criterion is more important than the 2nd criterion (1st-2nd case), and vice versa (2nd-1st case). The degree of the reduction of the Pareto set approximation was investigated with respect to coefficient of relative importance varying from to by step . On all instances for each value of we re-evaluate the obtained approximation in terms of “new” vector criterion upon the formulae from Theorem 3.1. Then by the complete enumeration we find the Pareto set approximation in “new” criterion space that gives us the reduction of the Pareto set approximation in the initial criterion space.
The number of elements of the Pareto set approximation and the percentage of the excluded elements from set are presented on average over series in Tables 1 and 2. Let be the difference between the maximum and minimum values of the Pareto set approximation on the -th criterion, . The value indicates the ratio between diversities of criteria of set .
For series S50[1,10][1,10], S50[1,20][1,20] when approximately of elements of the set are excluded, and when less than of elements are remained. The statement is valid for both 1st-2nd and 2nd-1st cases. Series SftvRand shows different results: in the 1st-2nd case the reduction occurs “almost uniformly”, i.e. the value of is almost proportional to the degree of the reduction, in the 2nd-1st case the condition gives approximately of the excluded elements. On series S50[1,10][1,20] in the 1st-2nd case the degree of the reduction grows slowly as tends to 1 in comparison to other series, and in the 2nd-1st case more than of elements are eliminated at .
Also, we note that on series S50[1,10][1,20] (SftvRand) for the percentage of the excluded elements in the 2nd-1st case is approximately () times greater than the percentage of the excluded elements in the 1st-2nd case. Note that for series S50[1,10][1,20] and for series SftvRand. Therefore, the ratio between diversities of values of the Pareto set approximation on components of criteria influences on the degree of the reduction in the same proportion when (each criterion has relatively the same importance).
On series S50contr[1,2][1,2], where the components of criterion contradict each other with coefficient , we do not have a reduction when , and the reduction up to one element takes place when . Thus, the results of the experiment confirm the theoretical results of Subsection 3.2. Moreover, identical character of the reduction for both 1st-2nd and 2nd-1st cases occurs only on series S50[1,10][1,10], S50[1,20][1,20], and S50contr[1,2][1,2], which have the same diversity and distribution with respect to both criteria.
Based on the results of the experiment we suppose that the degree of the reduction of the Pareto set approximation will be similar for the large-size problems with the same structure as the considered instances.
We applied to the bicriteria ATSP the axiomatic approach of the Pareto set reduction proposed by V. Noghin. For particular cases the series of “quanta of information” that guarantee the reduction of the Pareto set were identified. An approximation of the Pareto set to the bicriteria ATSP was found by a new generational multi-objective genetic algorithm. The experimental evaluation indicated the degree of reduction of the Pareto set approximation for various “quanta of information” and various problem structures.
Further research may include construction and analysis of new classes of multicriteria ATSP instances
with complex structures of the Pareto set.
It is also important to consider real-life ATSP instances with real-life decision maker
and investigate effectiveness of the axiomatic approach for them.
Moreover, developing a faster implementation of the multi-objective genetic algorithm
with steady-state replacement and local search procedures has great interest.
Acknowledgements The research was supported by RFBR grant 17-07-00371 (A. Zakharov) and by the Ministry of Science and Education of the Russian Federation under the 5-100 Excellence Programme (Yu. Kovalenko).
-  Angel, E., Bampis, E., Gourv’es, L., Monnot, J.: (Non)-approximability for the multi-criteria TSP(1,2). In: Liskiewicz, M., Reischuk, R. (eds.) FCT 2005. LNCS. vol. 3623, pp. 329–340. Springer, Heidelberg (2005). https://doi.org/10.100711537311 29
-  Ausiello, G., Crescenzi, P., Gambosi, G., Kann, V., Marchetti-Spaccamela, A., Protasi, M.: Complexity and Approximation. Springer (1999). https://doi.org/10.1007978-3-642-58412-1
-  Buzdalov, M., Yakupov, I., Stankevich, A.: Fast implementation of the steady-state NSGA-II algorithm for two dimensions based on incremental non-dominated sorting. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (GECCO-15). pp. 647–654 (2015). https://doi.org/10.11452739480.2754728
-  Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002). https://doi.org/10.11094235.996017
-  Ehrgott, M.: Multicriteria Optimization. Springer-Verlag, Berlin Heidelberg (2005). https://doi.org/10.10073-540-27659-9
-  Emelichev, V.A., Perepeliza, V.A.: Complexity of vector optimization problems on graphs. Optimization: A Journal of Mathematical Programming and Operations Research 22(6), 906–918 (1991). https://doi.org/10.108002331939108843732
-  Eremeev, A.V., Kovalenko, Y.V.: Genetic algorithm with optimal recombination for the asymmetric travelling salesman problem. In: Lirkov, I., Margenov, S. (eds.) Large-Scale Scientific Computing 2017. LNCS. vol. 10665, pp. 341–349 (2018). https://doi.org/10.1007978-3-319-73441-5 36
-  Figueira, J.L., Greco, S., Ehrgott, M.: Multiple criteria decision analysis: state of the art surveys. Springer-Verlag, New York (2005). https://doi.org/10.1007b100605
-  Garcia-Martinez, C., Cordon, O., Herrera, F.: A taxonomy and an empirical analysis of multiple objective ant colony optimization algorithms for the bi-criteria TSP. European Journal of Operational Research 180, 116–148 (2007). https://doi.org/10.1016j.ejor.2006.03.041
-  Jaszkiewicz, A., Zielniewicz, P.: Pareto memetic algorithm with path relinking for bi-objective traveling salesperson problem. European Journal of Operational Research 193, 885–890 (2009). https://doi.org/10.1016j.ejor.2007.10.054
-  Klimova, O.N.: The problem of the choice of optimal chemical composition of shipbuilding steel. J. Comput. Syst. Sci. Int. 46(6), 903–907 (2007). https://doi.org/10.1134S106423070706007X
-  Klimova, O.N., Noghin, V.D.: Using interdependent information on the relative importance of criteria in decision making. Comput. Math. Math. Phys. 46(12), 2080–2091 (2006). https://doi.org/10.1134/S0965542506120074
-  Kumar, R., Singh, P.K.: Pareto evolutionary algorithm hybridized with local search for biobjective TSP. In: Hybrid Evolutionary Algorithms. vol. 14, pp. 361–398. Springer (2007). https://doi.org/10.1007978-3-540-73297-6 14
-  Li, H., Zhang, Q.: Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II. IEEE Transactions on Evolutionary Computation 13(2), 284–302 (2009). https://doi.org/10.1109/TEVC.2008.925798
-  Lust, T., Teghem, J.: The Multiobjective Traveling Salesman Problem: A survey and a new approach. In: Advances in Multi-Objective Nature Inspired Computing. Studies in Computational Intelligence. vol. 272, pp. 119–141. Springer, Berlin (2010). https://doi.org/10.1007978-3-642-11218-8 6
-  Multiobjective optimization library: http://home.ku.edu.tr/~moolibrary/, [Online; accessed 09-Febrary-2018]
-  Noghin, V.D.: Reducing the Pareto set based on set-point information. Sci. Tech. Inf. Proc. 38(6), 435–439 (2011). https://doi.org/10.3103/S0147688211050078
-  Noghin, V.D.: Reducing the Pareto set algorithm based on an arbitrary finite set of information “quanta”. Sci. Tech. Inf. Proc. 41(5), 309–313 (2014). https://doi.org/10.3103/S0147688214050086
-  Noghin, V.D.: Reduction of the Pareto Set: An Axiomatic Approach. Springer International Publishing (2018). https://doi.org/10.1007978-3-319-67873-3
-  Noghin, V.D., Prasolov, A.V.: The quantitative analysis of trade policy: a strategy in global competitive conflict. Int. J. Business Continuity and Risk Management 2(2), 167–182 (2011). https://doi.org/10.1504IJBCRM.2011.041490
-  Podinovskiy, V.V., Noghin, V.D.: Pareto-optimal’nye resheniya mnogokriterial’nyh zadach (Pareto-optimal solutions of multicriteria problems). Fizmatlit, Moscow (2007), in Russian
-  Psychas, I.D., Delimpasi, E., Marinakis, Y.: Hybrid evolutionary algorithms for the multiobjective traveling salesman problem. Expert Systems with Applications 42(22), 8956–8970 (2015). https://doi.org/10.1016j.eswa.2015.07.051
-  Radcliffe, N.J.: The algebra of genetic algorithms. Annals of Mathemathics and Artificial Intelligence 10(4), 339–384 (1994). https://doi.org/10.1007BF01531276
-  Reeves, C.R.: Genetic algorithms for the operations researcher. INFORMS Journal on Computing 9(3), 231–250 (1997)
-  Reinelt, G.: TSPLIB – a traveling salesman problem library. ORSA Journal on Computing 3(4), 376–384 (1991). https://doi.org/10.1287ijoc.3.4.376
-  Vinogradskaya, T.M., Gaft, M.G.: Tochnaya verhn’ya otzenka chisla nepodchinennyh reshenii v mnogokriterial’nyh zadachah (The least upper estimate for the number of nondominated solutions in multi-criteria problems). Avtom. Telemekh. 9, 111–118 (1974), in Russian
-  Whitley, D., Starkweather, T., McDaniel, S., Mathias, K.: A comparison of genetic sequencing operators. In: Proceedings of the fourth International Conference on Genetic Algorithms. pp. 69–76. Morgan Kaufmann, New York (1991)
-  Yuan, Y., Xu, H., Wang, B.: An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO-14). pp. 661–668 (2014). https://doi.org/10.11452576768.2598342
-  Zakharov, A.O.: Pareto-set reducing using compound information of a closed type. Sci. Tech. Inf. Proc. 39(5), 293–302 (2012). https://doi.org/10.3103S0147688212050073
-  Zitzler, E., Brockhoff, D., Thiele, L.: The hypervolume indicator revisited: On the design of Pareto-compliant indicators via weighted integration. In: Proc. Conf. Evolut. Multi-Criter. Optim., LNCS. vol. 4403, pp. 862–876. Springer, Berlin (2007). https://doi.org/10.1007978-3-540-70928-2 64
-  Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the Strength Pareto Evolutionary Algorithm. In: Proceedings of Evolutionary Methods for Design, Optimisation and Control with Application to Industrial Problems. pp. 95–100 (2001)