Semantic-based Distance Approaches in Multi-objective Genetic Programming

Semantic-based Distance Approaches in Multi-objective Genetic Programming

Abstract

Semantics in the context of Genetic Program (GP) can be understood as the behaviour of a program given a set of inputs and has been well documented in improving performance of GP for a range of diverse problems. There have been a wide variety of different methods which have incorporated semantics into single-objective GP. The study of semantics in Multi-objective (MO) GP, however, has been limited and this paper aims at tackling this issue. More specifically, we conduct a comparison of three different forms of semantics in MOGP. One semantic-based method, (i) Semantic Similarity-based Crossover (SSC), is borrowed from single-objective GP, where the method has consistently being reported beneficial in evolutionary search. We also study two other methods, dubbed (ii) Semantic-based Distance as an additional criteriOn (SDO) and (iii) Pivot Similarity SDO. We empirically and consistently show how by naturally handling semantic distance as an additional criterion to be optimised in MOGP leads to better performance when compared to canonical methods and SSC. Both semantic distance based approaches made use of a pivot, which is a reference point from the sparsest region of the search space and it was found that individuals which were both semantically similar and dissimilar to this pivot were beneficial in promoting diversity. Moreover, we also show how the semantics successfully promoted in single-objective optimisation does not necessary lead to a better performance when adopted in MOGP.

Semantics, genetic programming, Multi-objective optimisation.

I Introduction

Genetic Programming (GP) [koza_1994_genetic] is a form of Evolutionary Algorithm (EA) that uses genetic operations that are analogous to behavioural biology and evolve programs towards finding a solution to a problem. The range of problem domains for GP are wide and this form of EA has been found to be beneficial for problems with multiple local optima and for problems with a varying degree of complexity [eiben_2015_from], making EAs ideal for highly complex problems including the automatic configuration of deep neural networks’ architectures (an in-depth recent literature review in this emerging research area can be found in [galvan2020neuroevolution]). However despite the well documented effectiveness of canonical GP, there are well-known limitations of these methods, through the study of properties of encodings [DBLP:conf/cec/LopezMOB10, Galvan-Lopez2011], and research is on going into finding and developing approaches to improve their overall performance including promoting neutrality in deceptive landscapes [10.1007/978-3-540-73482-6_9, DBLP:journals/tec/PoliL12], dynamic fitness cases [DBLP:conf/gecco/LopezVST17Poster, DBLP:conf/ae/LopezVST17], to mention a few examples.

An area of research which has proven popular in advancing the field of GP has been semantics. There have been a number of definitions for semantics in the past but broadly speaking semantics can be defined as the behavioural output of a program when executed using a set of inputs.

Semantic Similarity-based Crossover (SSC). This method was first proposed by Uy et al. [Uy2011] in the context of single-optimisation GP. This method uses a computationally expensive procedure by applying crossover between two parents multiple times using semantic diversity as a criteria in the selection process.

Semantic-based Distance as an additional criteriOn (SDO). This is a method which originates as an improvement to the crowding distance operation as proposed by Galván et al. [DBLP:conf/gecco/GalvanS19]. This method uses a reference point or pivot from the sparsely populated region of the search point and computes the semantic distance between this pivot and every individual. This distance is optimised as an additional criterion in Evolutionary Multiobjective Optimisation.

Pivot Semantic-based Distance as an additional criteriOn (PSDO). It is a method which is a variation of SDO but instead prefers solutions that are semantically similar to the pivot.

The goal of this paper are threefold: (i) to show how it is possible to naturally incorporate semantics in MOGP leading to a better performance, (ii) to demonstrate the robustness of the proposed method by using two different forms of computing semantic distance, used in SDO and PSDO, leading to similar results by these two methods, and (iii) to show how a widely successful form of semantics used in single-objective GP does not necessarily yield good results in MOGP.

The structure of this paper is as follows. Section II includes the related work. Section III outlines the background in semantics and covers multi-objective genetic programming techniques. Section IV presents the semantic-based approaches proposed in this work. Section V explains the experimental setup used and Section VI shows and discusses the results obtained by the various semantic and canonical methods used in this study. The final section offers concluding remarks.

Ii Related Work

Ii-a Semantics in Genetic Programming

Scientific studies of semantics in GP have increased dramatically over the last years given that it has been consistently reported to be beneficial in GP search, ranging from the study of geometric operators [DBLP:conf/ppsn/MoraglioKJ12], including the analysis of indirect semantics [Galvan-Lopez2016, Uy2011]. We discuss next the most relevant works to the research discussed in this paper.

Even though researchers have proposed a variety of mechanisms to use the semantics of GP programs to guide a search, it is commonly accepted that semantics refers to the output of a GP program once it is executed on a data set (also known as fitness cases in the specialised GP literature). The work conducted by McPhee et al. [McPhee:2008:SBB:1792694.1792707] paved the way for the proliferation of indirect semantics works. In their research, the authors studied the semantics of subtrees and the semantics of context (the remainder of a tree after the removal of a subtree). In their studies, the authors pointed out how a high proportion of individuals created by the widely used 90-10 crossover operator (i.e., 90%-10% internal-external node selection policy) are semantically equivalent. That is, the crossover operator does not have any useful impact on the semantic GP space, which in consequence leads to a lack of performance increase as evolution continues.

Uy et al. [Nguyen:2009:SAC:1533497.1533524] proposed four different forms of applying semantic crossover operators on real-valued problems (e.g., symbolic regression problems). To this end, the authors measured the semantic equivalence of two expressions by measuring them against a random set of points sampled from the domain. If the resulting outputs of these two expressions were close to each other, subject to a threshold value called semantic sensitivity, these expressions were regarded as semantically equivalent. In their experimental design, the authors proposed four scenarios. In their first two scenarios, Uy et al. focused their attention on the semantics of subtrees. More specifically, for Scenario I, they tried to encourage semantic diversity by repeating crossover for a number of trials if two subtrees were semantically equivalent. Scenario II explored the opposite idea of Scenario I. For the last two scenarios, the authors focused their attention on the entire trees. That is, for Scenario III Uy et al. checked if offspring and parents were semantically equivalent. If so, the parents were transmitted into the following generation and the offspring were discarded. The authors explored the opposite of this idea in Scenario IV (children semantically different from their parents). They showed, for a number of symbolic regression problems, that Scenario I produced better results compared to the other three scenarios.

The major drawback with the Uy et al. [Nguyen:2009:SAC:1533497.1533524] approach is that it can be computational expensive, since it relies on a trial mechanism that attempts to find semantically different individuals via the execution of the crossover operator multiple times. To overcome this limitation, Galván et al. [6557931] proposed a cost-effective mechanism based on the tournament selection operator to promote semantic diversity. More specifically, the tournament selection of the first parent is done as usual. That is, the fittest individual is chosen from a pool of individuals randomly picked from the population. The second parent is chosen from a pool of individuals that are semantically different from the first parent and it is also the fittest individual. If there is no individual semantically different from the first parent, then the tournament selection of the second parent is performed as usual. The proposed approach resulted in similar, and in some cases better, results compared to those reported by Uy et al. [Nguyen:2009:SAC:1533497.1533524, Uy2011] without the need of a trial and error (expensive) mechanism.

More recently, Forstenlechner et al. [Forstenlechner:2018:TES:3205455.3205592] proposed two semantic operators, Effective Semantic (ES) Crossover for Program Synthesis and ES Mutation for Program Synthesis. Program synthesis operates on a range of different data types as opposed to those that work in a single data type such as real-valued cases [Uy2010] and Boolean values [6557931]. The main elements considered by the authors were the metrics used to determine semantic similarity, named partial change, used in the first instance, and any change, used only if the first failed to be satisfied to avoid using standard crossover. In their results, the authors reported that a semantic-based GP system achieved better results in 4 out of 8 problems used in their studies.

Ii-B Multi-Objective Genetic Programming

In a multi-objective optimisation (MO) problem, one optimises with respect to multiple goals or objective functions. Thus, the task of the algorithm is to find acceptable solutions by considering all the criteria simultaneously. This can be achieved in various ways, where keeping the objectives separate is the most common. This form keeps the objectives separate and uses the notion of Pareto dominance. In this way, Evolutionary MO (EMO) [1597059, CoelloCoello1999, Deb:2001:MOU:559152] offers an elegant solution to the problem of optimising two or more conflicting objectives. The aim of EMO is to simultaneously evolve a set of the best tradeoff solutions along the objectives in a single run. EMO is one of the most active research areas in EAs thanks to its wide applicability as well as the impressive results achieved by these techniques [1597059, CoelloCoello1999, Deb:2001:MOU:559152].

MOGP has been used to classify highly unbalanced binary data [6198882, Galvan-Lopez2016]. To do so, the authors treated each objective (class) ‘separately’ using EMO approaches [Deb02afast, Zitzler01spea2:improving]. Bhowan et al. [6198882] and Galván et al. [DBLP:conf/gecco/GalvanS19, Galvan-Lopez2016, Galvan_MICAI_2016] showed, independently, how MOGP was able to achieve high accuracy in classifying binary data in conflicting learning objectives (i.e., normally a high accuracy of one class results in lower accuracy on the other).

Iii Background

Iii-a Semantics

Pawlak et al. [6808504] gave a formal definition for program semantics. Let be a program from a given programming language . The program will produce a specific output where input . The set of inputs can be understood as being mapped to the set of outputs which can be defined as .

Def 1. Semantic mapping function is a function mapping any program from to its semantic , where we can show the semantic equivalence of two programs;

(1)

This definition presents three important and intuitive properties for semantics:

  1. Every program has only one semantic attributed to it.

  2. Two or more programs may have the same semantics.

  3. Programs which produce different outputs have different semantics.

In Def. 1, we have not given a formal representation of semantics. In the following work semantics will be represented as a vector of output values which are executed by the program under consideration using an input set of data. For this representation of semantics we need to define semantics under the assumption of a finite set of fitness cases, where a fitness case is a pair comprised of a program input and its respective program output . This allows us to define the semantics of a program as follows:

Def 2. The semantics of a program is the vector of values from the output set obtained by computing on all inputs from the input set :

(2)

where is the size of the input set. Now that we have a formal definition in of semantics we can discuss the application of it in GP as elaborated in Section IV.

Iii-B Pareto dominance

In general terms a multi-objective problem seeks to find a solution that either maximizes or minimizes a number of objectives. In the case of maximization this can be represented mathematically as

(3)

where X represents the feasible solution set, represents the objective function for the feasible solution x and k 2. Typically there will not exist a unique solution that will maximize all objective functions. A candidate solution is Pareto dominant if its fitness is better or equal for all objectives and is strictly preferred by at least one in the search space. This can be formally represented by

(4)

where is the solution for objective and denotes that solution is non-dominated by solution . A candidate solution is considered Pareto optimal if is not dominated by any other candidate solution. In other words, if none of the objectives for a candidate solution can be improved without degrading at least one of the other objectives it can be considered Pareto optimal. For multi-objective problems there may exist a number of non-dominated solutions. The set of non-dominated candidate solutions for an MO problem is referred to as the first Pareto frontier when represented in objective space. In practise it is not always possible to do an exhaustive search for the true Pareto optimal set and as such this is something we seek to approximate instead. Pareto dominance relation is an integral part of MOEAs and has allowed practitioners and researchers to form important metrics in the selection process of these algorithms. Two such metrics are dominance rank and dominance count. Dominance rank is used as a fitness measure and calculates how may other solutions a candidate solution is dominated by. The lower the dominance rank the better with the lowest dominance rank of 0, i.e a solution that is not dominated by any other solution. Dominance rank can be expressed mathematically as seen in Equation 5;

(5)

where represents the cardinality of the set. This criteria is utilised in NSGA-II. Dominance count calculates how many individuals a candidate solution dominates [1]. The higher the dominance count the better. Dominance rank and Dominance count are both used in SPEA-2.

(6)

Leading on from the previous equation we can get a measure of raw fitness , by summing the fitness of all individuals such that [4];

(7)

A value of 0 corresponds to an individual which is non-dominated and a value that is high corresponds to an individual which is dominated by many other individuals, as such the raw fitness should be minimized.

Data set Classes Number of examples Imb. Features
Positive/Negative (Brief description) Total Positive Negative Ratio No. Type
Ion Good/bad (ionsphere radar signal) 351 126 (35.8%) 225 (64.2%) 1:3 34 Real
Spect Abnormal/normal (cardiac tom. scan) 267 55 (20.6%) 212 (79.4%) 1:4 22 Binary
Yeast mit/other (protein sequence) 1482 244 (16.5%) 1238 (83.5%) 1:6 8 Real
Yeast me3/other (protein sequence) 1482 163 (10.9%) 1319 (89.1%) 1:9 8 Real
Abal 9/18 (biology of abalone) 731 42 (5.75%) 689 (94.25%) 1:17 8 Real
Abal 19/other (biology of abalone) 4177 32 (0.77%) 4145 (99.23%) 1:130 8 Real
TABLE I: Binary imbalanced classification datasets used in our research

Iii-C Crowding distance

Solutions are ranked relative to each other according to a metric known as the crowding distance. The crowding distance is used to compare any pair of solutions in search space and is used in NSGA-II and SPEA2 as Pareto Dominance alone only acts as a partial order of the solutions. The crowding distance calculation is comprised of three parts;

  • Initialize the distance to zero.

  • Set the boundary solutions to . These solutions are always selected due to this constraint.

  • Calculate the average distance differences for an individual against its two nearest neighbours using the Manhattan distance, shown in Equation 8:

    (8)

where denotes the objective in question and r is the index for the current individual, where and reference its two nearest neighbours. Solutions with the highest crowding distance are considered better solutions, in other words the algorithm preferences localities along the Pareto front which are more sparsely populated with solutions than those which are more dense. In this manner the crowding distance resolves which solutions to retain when programs produce very similar fitness values.

Iv Semantic-based methods

Iv-a Semantic Similarity-based Crossover MOGP

To incorporate semantics in a MOGP paradigm, we first use the Semantic Similarity-based Crossover (SSC) originally proposed by Uy et al. [Uy2011] which, to the best of our knowledge, has been exclusively used in single-objective GP.

To use SSC in single-objective GP a semantic distance must be computed first. Using Def. 2, this distance is obtained by computing the average of the absolute difference of values for every between parent and offspring. If the distance value lies within a range, defined by one or two threshold values, then crossover is used to generate offspring. Because this condition may be hard to satisfy, the authors tried to encourage semantic diversity by repeatedly applying crossover up to 20 times. If after this, the condition is not satisfied, then crossover is executed as usual.

SSC made a notable impact in GP, showing, for the first time, how semantic diversity can be promoted in continuous search spaces, with several subsequent papers following along this line [DBLP:conf/gecco/GalvanS19, 6557931, Krawiec:GPEM2013]. We incorporate SSC in NSGA-II and SPEA2. However, the performance increase reported when adopting SSC in single-objective optimisation is not observed in MOGP, as discussed in Section VI.

Iv-B Semantic Distance as an additional criteriOn

In this method the crowding distance as discussed in Section III-C is replaced by a semantic-based crowding distance. A pivot is selected as the individual in the first Pareto front which is furthest away from all other individuals in that front. This distance is calculated using the crowding distance as discussed previously. Once we have the pivot, we can compute the semantic differences of the pivot against all the other individuals in the population. Upper and lower semantic similarity bounds, denoted as UBSS and LBSS, respectively, are used to promote semantic diversity within a range as shown in Equation 9 or via a single bound as like in Equation 10.

(9)
(10)

The semantic-based crowding distance can also be used as an additional criterion to optimise. With the majority and minority class (these benchmark problems are discussed in detail in the following section) serving as the first two objectives to optimize semantic distance is also treated a third objective.

Iv-C Pivot Similarity Semantic-based Distance as and additional criteriOn

In Section IV-B we discussed SDO which calculates the semantic distance between each individual and a pivot. To this effect Equations 9 and 10 were used to determine if the semantic difference between a pivot and the individuals fall within a predefined range. This calculation naturally preferences programs which are semantically dissimilar to the pivot. However, given that the pivot is picked from the most sparse region of the search space this individual ought to be the most diverse as such an update a proposed update in the distance calculation is given in Equations 11 and 12, where the summed values is now taken away from the number of fitness cases. This method is referred to as Pivot Similarity Semantic-based Distance as an additional criteriOn (PSDO) and uses Equations 11 and 12 in lieu of distances shown in Equations 9 and 10.

(11)
(12)

V Experimental Setup

The data sets originate from the well-known UCI Machine Learning repository [Dua:2019]. The characteristics of these data sets vary greatly, with varying number of features and imbalance ratios. A brief description of each of these data sets is given in Table I, including the nature of their feature type, their imbalance ratios and the number of features. All data sets were split 50/50 with half of the entries being attributed to the training set and the other half for the test set. The same class imbalance ratio is kept between the training and test set. All the results reported in this work are based on the latter. The function set is the list of arithmetic operators used by the GP and are assigned at the non-terminal nodes, these were selected as , where % denotes the protected division operator.

Parameter Value
Population Size 500
Generations 50
Type of Crossover 90% internal nodes, 10% leaves
Crossover Rate 0.60
Type of Mutation Subtree
Mutation Rate 0.40
Selection Tournament (size = 7)
Initialisation Method Ramped half-and-half
Initialisation Depths:
   Initial Depth 1 (Root = 0)
   Final Depth 5
Maximum Length 800
Maximum Final Depth 8
Independent Runs 50
Semantic Thresholds UBSS = {0.25, 0.5, 0.75, 1.0}
LBSS = {-, 0.001, 0.01, 0.1}
TABLE II: Summary of parameters

A common metric used in determining fitness for binary classification problems, as the ones used in this work, is to use classification accuracy; where . However with imbalanced data sets using this accuracy measure will tend to bias towards the majority class. As such it is better to treat the minority and majority as two separate objectives where the goal is to maximize the number of correctly classified cases. This can be done using the true positive rate and true negative [6198882]. Table II gives an overview of the parameters used in our work. Of note are the UBSS and LBSS settings (bottom of Table II). These allow for a possible combination of 16 settings for the semantic thresholds. Each setting is run for 50 independent runs which results in 4800 independent runs for each semantic-based method tested. In short, when we consider all of the semantic-based methods in addition to the canonical methods for both MOGP frameworks over 29,400 independent runs were conducted. All experiments were run on Kay a supercomputer with 336 nodes operated by Irish Centre for High-End Computing.

Dataset NSGA-II SPEA2
Hypervolume Hypervolume
Average PO Front Average PO Front
Ion 0.766 0.114 0.938 0.786 0.094 0.948
Spect 0.534 0.024 0.647 0.544 0.032 0.659
Yeast 0.838 0.011 0.876 0.838 0.008 0.877
Yeast 0.950 0.009 0.976 0.946 0.015 0.978
Abal 0.847 0.058 0.961 0.832 0.078 0.960
Abal 0.576 0.122 0.842 0.544 0.147 0.834
TABLE III: Average hypervolume ( std. deviation) and last run Pareto Front for NSGA-II and SPEA2 for 50 independent runs.

Vi Results and Analysis

In order to compare the approaches used in this work, we use the hypervolume [Coello:2006:EAS:1215640] of the evolved Pareto approximations as a performance measure. For bi-objective problems, as the binary highly unbalanced classifications problems used in this work are, the hypervolume of a set of points in objective space, using reference point , is easily computed as the sum of the areas of all trapezoids fitted under each point. The reference point is required for the hypervolume calculation and as we wish to maximize each of the objectives we select this point as the lowest value that either the or can take. The hypervolume was chosen because it is the one of the most widely used performance indicator in the EMO literature. We also computed the accumulated Pareto-optimal (PO) front with respect to 50 runs: the set of non-dominated solutions after merging all 50 Pareto-approximated fronts.

Tables III and VI report, for each problem defined in Table I both, the average hypervolume over 50 runs and also the hypervolume of the accumulated PO front with respect to all 50 runs. To obtain a statistically sound conclusion, a series of Wilcoxon rank-sum tests were run on the average hypervolume results. To account for the problem of multiple comparisons that arose from testing the canonical method 16 times for each data set, a Bonferroni correction was used where = 0.05. These statistically significant differences are highlighted in boldface in Table VI (this shown in the appendix). Moreover, in this table, the symbols “+”, “–”, and “=” indicate that the results of a given semantic-based approach are significantly better, worse, or equal, respectively, than those found by the canonical NSGA-II (Table III), all the above based on the Wilcoxon rank-sum test.

Let us first focus our attention on the average hypervolume and PO front results yield by NSGA-II and SPEA2 shown in Table III. Such observed differences between them are not statistically significant in any of the dataset used. We can now compare the semantic-based methods, introduced in Section IV, against their corresponding canonical EMO algorithms. Due to the significant amount of results obtained and shown in the appendix, we summarised our findings in a series of payoff matrices to show which strategies have significant better values or ‘wins’ vs. the other methods. These payoff matrices are shown in Tables IV and V for semantic-based methods using NSGA-II and SPEA2, respectively.

Ion
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 2 15
PSDO 13 3 - 14
SSC 0 0 0 -
Spect
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 8 16
PSDO 16 3 - 15
SSC 0 0 0 -
Yeast
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 2 16
PSDO 16 1 - 16
SSC 0 0 0 -
Yeast
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 2 16
PSDO 16 4 - 16
SSC 0 0 0 -
Abal
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 12 - 10 12
PSDO 6 1 - 7
SSC 0 0 0 -
Abal
CAN. SDO PSDO SSC
CAN. - 0 0 2
SDO 12 - 1 15
PSDO 16 7 - 16
SSC 0 0 0 -

TABLE IV: Payoff tables for canonical NSGA-II, NSGA-II SDO, NSGA-II PSDO and NSGA-II SSC for each of the 6 data sets

The table can be read as follows: the strategies of the row index are compared against the strategies of the column index and for each LBSS and UBSS setting which is significantly better for the column strategy counts as one ‘win’ towards the count. For example SDO vs. NSGA-II for the Ion data set is significantly better for all settings of LBSS and UBSS and as such has 16 ‘wins’ overall (top left-hand side of Table IV). The tables have been colour coded as such with solid black denoting that the strategy is the best overall in terms of the number of ‘wins’ and light grey being the worst overall. We do not show all the results for SPEA2 and the semantic-based methods, as we did for NSGA-II, shown in Table VI, due to space constraints, but the payoff table indicates how SDO and PSDO consistently yield better results compared to canonical SPEA2.

The two methods that use semantic distance as a criteria strategies, SDO and PSDO, outperformed their respective canonical counterparts with the exception of Abal which had a large number of settings that produced no significant difference in the hypervolume averages for canonical NSGA-II. Additionally for Abal NSGA-II SDO had the greatest number of wins over other strategies and results for NSGA-II PSDO were comparatively mixed.

If we consider just SDO vs. PSDO, results tend to be mixed. NSGA-II PSDO produced more wins for certain data sets like Yeast and Abal when compared with NSGA-II SDO but under performed for Spect and Abal. Typically when NSGA-II PSDO out performed against NSGA-II SDO, it was for settings that were held constant. For instance, three of the wins associated with Spect were a result of keeping UBSS constant at 1.0 with LBSS values of (0.001, 0.01, 0.1). NSGA-II PSDO performed significantly worse most often when LBSS was undefined except for Abal where the results were reversed, i.e when LBSS was undefined PSDO performed as good or significantly better but was significantly worse for 10 of the other LBSS settings. There was little or no significant difference when comparing SPEA2 SDO and SPEA2 PSDO strategies with only SPEA2 SDO vs SPEA2 PSDO in Yeast producing 1 ‘Win’. From this we can conclude that both methods, SDO and PSDO, that treat semantic distance as an additional criterion to be optimised perform similar and yield consistently better results to SSC. This latter method is based on the notion of single-objective GP adapted to MOGP, showing that the benefits obversed in SOGP are depleted in MOGP.

Ion
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 0 16
PSDO 16 0 - 16
SSC 0 0 0 -
Spect
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 14 - 0 16
PSDO 16 0 - 16
SSC 0 0 0 -
Yeast
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 1 16
PSDO 16 0 - 16
SSC 0 0 0 -
Yeast
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - 0 16
PSDO 16 0 - 16
SSC 0 0 0 -
Abal
CAN. SDO PSDO SSC
CAN. - 0 0 0
SDO 16 - O 16
PSDO 13 0 - 13
SSC 0 0 0 -
Abal
CAN. SDO PSDO SSC
SPEA2 - 0 0 0
SDO 15 - 0 16
PSDO 16 0 - 16
SSC 0 0 0 -
TABLE V: Payoff tables for canonical SPEA2, SPEA2 SDO, SPEA2 PSDO and SPEA2 SSC for each of the 6 data sets.

Vii Conclusions

The analysis and experimentation conducted for this paper clearly show the benefits of incorporating semantics in MOGP. A key finding was that regardless of which method was used in calculating semantic distance as an additional criterion to optimise, each method (SDO and PSDO) proved to be significantly better than canonical methods as well as SSC, a method continously reported to be beneficial in single-objective GP.

When the various semantic distance methods were analysed it was found these methods had a tendency to preference programs that were semantically very similar and also semantically very dissimilar relative to the pivot. The pivot represents an individual from the most sparse region of the search space and can be considered the most divergent relative to the other programs in the population, as such it makes sense that these programs would be preferenced. Additionally the more semantically diverse programs will produce programs with significantly different outputs and ought to also be preferenced.

As the semantic distance is not an objective that is in conflict with the majority and minority class, both the SDO and PSDO methods are versatile enough to satisfy both of these criteria. The results showed that both these methods performed significantly better than canonical methods as well as the semantic-based method of SSC. While it was observed that both semantically similar and semantically dissimilar individuals relative to the pivot are preferable, it has not been shown if the specific location of the pivot in search space has any bearing on which distance metric would produce more diverse solutions for a given generation. As such further analysis on the pivot and its location in objective space relative to other individuals, may be useful in determining if a strategy of switching between SDO and PSDO methods during runtime would be worthwhile. Future work would likely benefit further from semantic-based distance methods and their is room for for further exploration of how to calculate new distance metrics.

Viii acknowledgments

This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant number 18/CRT/6049. The authors wish to acknowledge the DJEI/DES/SFI/HEA Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support.

References

Appendix

Hypervolume
Average PO Front
UBSS UBSS
LBSS 0.25 0.5 0.75 1.0 0.25 0.5 0.75 1.0
NSGA-II SDO
Ion 0.860 0.033+ 0.869 0.037+ 0.869 0.033+ 0.845 0.057+ 0.948 0.958 0.962 0.950
0.001 0.817 0.087+ 0.819 0.104+ 0.857 0.057+ 0.861 0.047+ 0.942 0.957 0.954 0.958
0.01 0.825 0.084+ 0.843 0.073+ 0.861 0.045+ 0.861 0.038+ 0.946 0.956 0.957 0.944
0.1 0.846 0.070+ 0.848 0.068+ 0.844 0.075+ 0.864 0.044+ 0.950 0.956 0.953 0.960
Spect 0.591 0.027+ 0.593 0.025+ 0.594 0.023+ 0.600 0.019+ 0.684 0.679 0.689 0.694
0.001 0.562 0.021+ 0.558 0.025+ 0.561 0.019+ 0.560 0.016+ 0.668 0.653 0.660 0.644
0.01 0.564 0.025+ 0.560 0.023+ 0.566 0.024+ 0.559 0.016+ 0.672 0.650 0.669 0.643
0.1 0.563 0.022+ 0.563 0.024+ 0.567 0.018+ 0.561 0.024+ 0.664 0.658 0.655 0.658
Yeast 0.850 0.006+ 0.849 0.008+ 0.849 0.006+ 0.849 0.006+ 0.881 0.881 0.882 0.881
0.001 0.845 0.007+ 0.847 0.006+ 0.848 0.004+ 0.848 0.005+ 0.879 0.882 0.879 0.880
0.01 0.848 0.006+ 0.849 0.005+ 0.848 0.005+ 0.850 0.005+ 0.881 0.881 0.879 0.881
0.1 0.847 0.005+ 0.848 0.005+ 0.848 0.005+ 0.850 0.005+ 0.878 0.879 0.879 0.883
Yeast 0.961 0.007+ 0.961 0.007+ 0.960 0.008+ 0.962 0.007+ 0.978 0.979 0.979 0.979
0.001 0.959 0.008+ 0.958 0.007+ 0.961 0.006+ 0.961 0.006+ 0.981 0.978 0.979 0.978
0.01 0.955 0.009+ 0.959 0.007+ 0.960 0.009+ 0.961 0.007+ 0.979 0.980 0.979 0.978
0.1 0.958 0.009+ 0.960 0.007+ 0.961 0.007+ 0.962 0.006+ 0.978 0.978 0.981 0.979
Abal 0.849 0.081 0.862 0.087 0.847 0.089 0.849 0.085 0.964 0.970 0.966 0.967
0.001 0.892 0.051+ 0.905 0.036+ 0.907 0.036+ 0.906 0.034+ 0.970 0.968 0.969 0.971
0.01 0.908 0.038+ 0.900 0.056+ 0.919 0.022+ 0.919 0.026+ 0.969 0.973 0.970 0.972
0.1 0.910 0.037+ 0.911 0.046+ 0.912 0.049+ 0.916 0.031+ 0.970 0.972 0.969 0.970
Abal 0.591 0.102 0.623 0.138 0.634 0.115 0.617 0.137 0.862 0.878 0.881 0.873
0.001 0.729 0.070+ 0.722 0.063+ 0.709 0.080+ 0.735 0.074+ 0.877 0.870 0.879 0.885
0.01 0.721 0.067+ 0.725 0.075+ 0.721 0.074+ 0.723 0.066+ 0.881 0.879 0.884 0.880
0.1 0.724 0.076+ 0.739 0.065+ 0.736 0.063+ 0.756 0.065+ 0.888 0.883 0.886 0.890
Better (+) / Worse (-) 22 / 0 22 / 0 22 / 0 22 / 0
Same (=) / NSS 0 / 2 0 / 2 0 / 2 0 / 2
NSGA-II PSDO
Ion 0.794 0.100 0.811 0.084 + 0.823 0.091 0.795 0.105 0.904 0.932 0.945 0.939
0.001 0.867 0.035+ 0.874 0.029+ 0.880 0.036+ 0.873 0.045+ 0.959 0.952 0.965 0.945
0.01 0.852 0.050+ 0.867 0.051+ 0.880 0.031+ 0.867 0.050+ 0.947 0.950 0.944 0.949
0.1 0.853 0.062+ 0.869 0.048+ 0.875 0.051+ 0.872 0.049+ 0.941 0.951 0.956 0.938
Spect 0.552 0.020+ 0.546 0.022+ 0.555 0.022+ 0.554 0.017+ 0.648 0.665 0.638 0.640
0.001 0.550 0.026+ 0.562 0.025+ 0.561 0.025+ 0.592 0.026+ 0.661 0.670 0.658 0.706
0.01 0.550 0.025+ 0.563 0.026+ 0.558 0.025+ 0.583 0.020+ 0.649 0.675 0.667 0.669
0.1 0.551 0.023+ 0.560 0.024+ 0.557 0.025+ 0.593 0.020+ 0.666 0.664 0.678 0.682
Yeast 0.846 0.006+ 0.846 0.005+ 0.847 0.005+ 0.848 0.006+ 0.864 0.868 0.871 0.869
0.001 0.849 0.006+ 0.848 0.005+ 0.850 0.007+ 0.850 0.005+ 0.873 0.868 0.871 0.869
0.01 0.850 0.005+ 0.849 0.007+ 0.850 0.006+ 0.851 0.006+ 0.870 0.874 0.872 0.872
0.1 0.850 0.006+ 0.850 0.005+ 0.850 0.005+ 0.851 0.006+ 0.876 0.873 0.872 0.870
Yeast 0.957 0.007+ 0.959 0.007+ 0.957 0.009+ 0.959 0.007+ 0.973 0.978 0.976 0.978
0.001 0.960 0.010+ 0.962 0.005+ 0.964 0.005+ 0.962 0.008+ 0.976 0.976 0.978 0.977
0.01 0.962 0.006+ 0.962 0.006+ 0.962 0.005+ 0.962 0.006+ 0.977 0.975 0.974 0.975
0.1 0.964 0.006+ 0.960 0.010+ 0.963 0.005+ 0.961 0.007+ 0.976 0.976 0.977 0.975
Abal 0.890 0.051+ 0.881 0.070+ 0.885 0.046+ 0.884 0.058+ 0.959 0.966 0.961 0.952
0.001 0.861 0.079 0.848 0.073 0.877 0.078+ 0.864 0.075 0.962 0.957 0.962 0.959
0.01 0.864 0.067 0.858 0.076 0.865 0.070 0.873 0.066 0.967 0.959 0.962 0.962
0.1 0.858 0.082 0.887 0.061+ 0.864 0.074 0.860 0.075 0.963 0.968 0.962 0.955
Abal 0.704 0.083+ 0.699 0.072+ 0.706 0.069+ 0.711 0.076+ 0.826 0.859 0.874 0.858
0.001 0.725 0.070+ 0.743 0.079+ 0.745 0.060+ 0.733 0.075+ 0.859 0.871 0.854 0.877
0.01 0.741 0.086+ 0.735 0.074+ 0.724 0.070+ 0.728 0.069+ 0.873 0.867 0.870 0.873
0.1 0.743 0.061+ 0.723 0.073+ 0.719 0.088+ 0.722 0.063+ 0.877 0.847 0.846 0.851
Better (+) / Worse (-) 20 / 0 22/ 0 21 / 0 20 / 0
Same (=) / NSS 0 / 4 0 / 2 0 / 3 0 / 4
NSGA-II SSC
Ion 0.761 0.108 0.749 0.161 0.763 0.152 0.744 0.137 0.941 0.937 0.951 0.949
0.001 0.765 0.134 0.753 0.124 0.699 0.188 0.803 0.103 0.954 0.935 0.928 0.946
0.01 0.760 0.125 0.751 0.123 0.710 0.161 0.802 0.104 0.947 0.929 0.928 0.947
0.1 0.775 0.095 0.738 0.184 0.746 0.141= 0.778 0.099 0.957 0.951 0.945 0.936
Spect 0.525 0.025 0.532 0.029 0.537 0.020 0.535 0.029 0.633 0.634 0.634 0.634
0.001 0.530 0.029 0.539 0.030 0.542 0.023 0.540 0.025 0.651 0.635 0.638 0.654
0.01 0.535 0.029 0.537 0.027 0.541 0.027 0.540 0.028 0.655 0.633 0.658 0.651
0.1 0.532 0.029 0.531 0.026 0.534 0.027 0.533 0.022 0.632 0.641 0.635 0.635
Yeast 0.819 0.041 0.829 0.023 0.835 0.014 0.834 0.017 0.874 0.875 0.878 0.878
0.001 0.825 0.031 0.834 0.029 0.834 0.019 0.826 0.039 0.877 0.877 0.877 0.877
0.01 0.827 0.027 0.835 0.016 0.836 0.019 0.830 0.030 0.874 0.877 0.877 0.879
0.1 0.831 0.027 0.828 0.034 0.831 0.028 0.835 0.014 0.879 0.876 0.876 0.875
Yeast 0.950 0.013 0.948 0.010 0.945 0.032 0.947 0.009 0.978 0.977 0.978 0.977
0.001 0.946 0.013 0.944 0.028 0.947 0.013 0.950 0.011 0.976 0.976 0.977 0.979
0.01 0.947 0.014 0.944 0.024 0.946 0.015 0.949 0.012 0.978 0.978 0.978 0.978
0.1 0.948 0.014 0.948 0.012 0.946 0.009 0.947 0.016 0.978 0.978 0.977 0.977
Abal 0.844 0.084 0.839 0.083 0.834 0.070 0.824 0.099 0.963 0.967 0.962 0.962
0.001 0.851 0.062 0.812 0.086 0.845 0.077 0.844 0.079 0.964 0.961 0.959 0.967
0.01 0.850 0.076 0.833 0.091 0.829 0.096 0.836 0.090 0.972 0.957 0.959 0.963
0.1 0.869 0.064 0.838 0.083 0.844 0.075 0.834 0.084 0.963 0.965 0.965 0.962
Abal 0.521 0.121 0.532 0.103 0.529 0.128 0.511 0.118- 0.810 0.802 0.841 0.801
0.001 0.561 0.082 0.534 0.102 0.542 0.104 0.502 0.161 0.823 0.865 0.829 0.820
0.01 0.494 0.147 0.536 0.114 0.533 0.134 0.547 0.123 0.844 0.826 0.841 0.850
0.1 0.513 0.132- 0.549 0.120 0.514 0.112 0.532 0.131 0.806 0.820 0.785 0.831
Better (+) / Worse (-) 0 / 1 0 / 0 0 / 0 0 / 1
Same (=) / NSS 0 / 23 0/ 24 1 / 23 0 / 23
TABLE VI: Average hypervolume ( std. deviation) and last run Pareto Front for NSGA-II SDO, NSGA-II PSDO and NSGA-II SSC methods.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
414468
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description