# Epistasis and constraints in fitness landscapes

## Abstract

Genotypic fitness landscapes are constructed by assessing the fitness of all possible combinations of a given number of mutations. In the last years, several experimental fitness landscapes have been completely resolved. As fitness landscapes are high-dimensional, their characterization relies on simplemeasures of their structure, which can be used as statistics in empirical applications. Here we propose two new sets ofmeasuresthat explicitly capture two relevant features of fitness landscapes: epistasis and constraints. The first set contains new measures for epistasis based on the correlation of fitness effects of mutations. They have a natural interpretation, capture well the interaction between mutations, can be obtained analytically for most landscape models and can therefore be used to discriminate between different models. The second set contains measures of evolutionary constraints based on “chains” of forced mutations along fitness-increasing paths. Some of these measures are non-monotonic in the amount of epistatic interactions, but have instead a maximum for intermediate values. We further characterize the relationships of these measures to the ones that were previous proposed (e.g. number of peaks, roughness/slope, fraction of non-additive components, etc). Finally, we show how these measures can help uncovering the amount and the nature of epistatic interactions in two experimental landscapes.

(1) Evolution Paris-Seine (UMR 7138) and Atelier de Bio-Informatique, UPMC, Paris; SMILE, CIRB (UMR 7241); Collège de France, Paris, France. (2) The Pirbright Institute, Woking, United Kingdom. (3) Ecology and Evolutionary Biology, Brown University, Providence, USA. (4) Institute for Theoretical Physics, University of Cologne, Germany. (5) Center for Ecological Research, Kyoto University, Japan. (6) Kochi University of Technology, Japan. (7) Department of Biological Sciences, University of Tokyo, Japan.

## 1Introduction

Fitness landscapes have been a very successful metaphor to study evolution. Most simply, the idea of Sewall [44] to view evolution as a hill-climbing process proved to be appealing and inspired a vast amount of theoretical work in phenotypic and molecular evolution [26]. Furthermore, this metaphor contributed to the scientific exchange with other fields, especially with computer science [30] and physics [34].

In evolutionary biology, fitness landscapes have been used to study adaptation. In the classical metaphor, an evolving population is abstracted into a particle that navigates in the landscape [26]. In a strong selection weak mutation regime [14], the evolutionary paths followed by the populations are restricted to paths of increasing fitness, which can sometimes even be completely deterministic. In this perspective, it has been emphasized that many fundamental features of adaptation depend on whether the landscape is smooth or rugged. Among other things, the ruggedness and the properties of fitness landscapes have been related to speciation processes [13], to the benefits of sexual reproduction [21], and more generally, to the repeatability of the adaptation process (e.g. [17]).

Consequently, it is now clear that several aspects of evolutionary processes directly depend on the structure of the fitness landscapes in which the organisms are evolving. Furthermore, Wright’s idea of genotype-fitness landscapes moved from a metaphor to an object of experimental studies, as several fitness landscapes were experimentally resolved [37]. In that regard, characterizing the structure of experimental and model fitness landscapes is a key step in our ability to decipherevolution at the finest scale. However, because fitness landscapes are objects of (very) high dimensionality, their characterization relies on simplescalar measures (*i.e.* statistics) that are able to capture the important features of the landscapes. In this study, we propose two new sets of measures for fitness landscapes that have an immediate interpretation in biology in terms of epistasis and evolutionary constraints.

One of the most basic ingredients that characterize the structure of fitness landscapes is epistasis. Epistasis is the interaction between the effects of mutations at different loci. It is usually defined as the non-multiplicative part of the fitness effects of combined mutations, that is the non-additive part, in log-scale. In the presence of epistasis, the fitness effect of a mutation at a given locus depends on the genetic background and consequenty, a mutation at a given locus changes the distribution of fitness effects of other mutations at other loci. For the 2-loci 2-alleles case, assuming that the genotype with the smallest fitness is labeled , epistasis can be expressed (in logscale) as the departure from additivity : , where is the malthusian fitness of the genotype (e.g. [28]).

Assuming random fitness values (*i.e.* NK landscapes), [17] showed a positive correlation between the amount of epistatic interactions and the ruggedness of a landscape, defined as the density of peaks (genotypes with no fitter neighbors). As the number of loci that interact together grows, the landscape is more rugged, and more local peaks end evolutionary paths. At the maximum number of epistatic interactions, the fitnesses of each genotype are completely uncorrelated, resulting in a so-called House-of-Cards model [20], for which several measures on paths of increasing fitness were derived [18]

Experimental fitness landscapes make possible to test predicted properties of evolutionary trajectories through evolution experiments, together with sequencing [1]. Currently available fitness landscapes are based on mutations in a few loci, typically 4-10 [35]. Since the number of genotypes scales as the product of the number of alleles at all loci, testing all the combinations of mutations in an organism (or even in a protein) is beyond the reach of any reasonable future experiment. However, small landscapes have been resolved and analyzed. An exciting opportunity raised by the recent release of these experimental landscapes is to find the adequate model(s) that is (are) able to generate landscapes that share similar features with the observed ones. In that regard, characterizing the structure of small fitness landscapes is today a key step for our understanding of evolution in the presence of realistic interactions among mutations.

The high dimensionality of fitness landscapes make them almost impossible to visualize (although some attempt were proposed, see [22] or [6]). As a consequence, the analyses of fitness landscapes will mostly rely on measures that capture important features of their structures. Several measures were proposed previously and used to analyze experimental fitness landscapes (reviewed in [35]). The most natural one that was historically used to appreciate the ruggedness of a landscape is its number of peaks [40]. Intriguingly, although ruggedness is more adequately represented by both types of extrema, only little attention has been payed to the number of sinks (genotypes with only fitter neighbors). We therefore suggest that peaks and sinks are both adequate measures of the landscape ruggedness. Most models generate landscapes with the same mean number of peaks and sinks; however, small landscapes can have a different number of peaks and sinks due to random sampling. Furthermore, there is no theoretical reason for a symmetry between peaks and sinks in real fitness landscapes.

Other measures such as ratio(ratio of the roughness over additive fitness), fraction of sign epistasis, etc. (see detailed description in the Appendix) were also proposed to characterize the structure of fitness landscapes. As they all represent direct or indirect measures of epistasis, all these measures were shown to be pairwise correlated in experimental fitness landscapes [35]. In that regard, othermeasures related to evolution but somehowuncorrelated to the amount of epistasis are also needed to investigate thenature of the interactions in the landscapes.

Here, we describe two newmeasures that can be used to characterize the structure of fitness landscapes (Figure 1).

The first one, , is the single-step correlation of fitness effects for mutations between neighbor genotypes (Figure 1b). It is a direct measure of epistasis, *i.e.* it measures how much the fitness effect of a mutation is affected when a genotype experiences another mutation. As all correlation measures, ranges from to and is a very natural quantity to describe the amount of epistasis (ruggedness) in the landscape. The second one, chains, aims at quantifying the amount of constrained evolution in the landscapes. In the limit of strong selection and low mutation rates, when a hill climbing evolutionary path reaches a genotype that has a single fitter neighbor, the next step of the path is essentially deterministic. In this case, only one of the mutations available in the landscape can improve the fitness. If several genotypes with a single way out are connected together they form a chain of “obligatory” mutational events that often ends on a peak (Figure 1c), but could also end on an intermediate genotype.

The two measures target different features of the landscapes: aims at quantifying the amount of epistasis, independently of its nature, whereas discriminate between different types of epistasis. In the following, we will present bothmeasures in detail, then discuss their relations with the existingmeasures and finally quantify them on two experimental landscapes.We also present the mean value of these measures for several landscape models, like the House of Cards (HoC), the Rough Mount Fuji (RMF) and NK landscapes as well as Ising and Eggbox models. Details on previousmeasures and model landscapes used here are given in the Appendix.

## 2Epistasis as correlation of fitness effects:

### 2.1Definition

In this section, we will derive and discuss a new measure that is a natural description of the amount of epistasis in fitness landscapes. This new measure, denoted by , is simply the correlation of fitness effects of the same mutation in single-mutant neighbors (see Figure 1b and Figure 2). It measures how the effect of a focal mutation is altered by another mutation at another locus in the background, averaged across the whole landscape.

In the following, we will define it properly in mathematical terms for the bi-allelic case. We denote the (log-scaled) fitness of a genotype by . We also define , the genotype where the locus is mutated. The fitness effect of a mutation at locus , *i.e.* the log-scale selection coefficient of the mutation, is denoted by . The new measure is then defined as the correlation between two fitness differences and measured from genotypes that are one mutation away, as illustrated by Figure 2a.

Noting that the average of across all genotypes and mutations in the landscape is 0, we define as:

where indicates a generic genotype that differs from by a single mutation. For multiallelic landscapes, the same definition can be immediately generalized to any number of alleles.

Even though is originally defined in term of fitness effects of the mutations, it can be easily recomputed by only using the fitness values themselves. If we denote the correlation between fitness of genotypes apart from distance , it is possible to rewrite by the simple formula (see proof in the Appendix):

Besides its general interest, this formulation allows us to measure in the presence of missing data. Indeed, fitness correlation functions do not need fitness data for all combinations of some set of mutations in order to be estimated.

### 2.2Interpretation

To make clear that the above measure is a metric of epistasis, we rewrite the above equation as

When there is no epistasis, the fitness effects do not depend on the background and , *i.e.* perfect correlation between fitness effects. The deviation of from 1 is proportional to the square of , which is a standard measure of the amount of epistatic effect (for 2-alleles 2-loci), normalized by the average squared fitness effect. Thanks to its normalization, this measure of epistasis is not affected by the scale and the absolute level of fitness, but only by relative differences in fitness. Shifting fitnesses by a multiplicative or additive factor does not change this measure.

The measure is defined as a correlation, therefore it is bounded by , with in the case of no epistasis. The value of is related to the prevalent type of epistatic interactions (see proof in the Appendix), which are summarized in Figure 2b:

magnitude epistasis refers to pairwise interactions that do not change the signs of fitness effects. Magnitude epistasis would still result in a positive correlation between fitness effects, therefore would still be positive even if smaller than 1: ;

sign epistasis refers to pairwise interactions where the fitness effects of one mutation change sign after the other mutation. Sign epistasis would contribute with terms of both signs to the correlation, therefore resulting in values centered around 0: ;

finally, reciprocal sign epistasis refers to pairwise interactions where both fitness effects change sign. Reciprocal sign epistasis would imply a negative correlation between fitness effects, and therefore a negative value of : .

The deviation of the mean value of from 1 for simple landscape models measures epistasis as a function of the parameters of the models (see the Appendix for details on the derivations; the reader unfamiliar with the models will also find a brief presentation there). For example, in NK landscapes the epistasis grows with the parameter K describing the number of loci involved in each interaction and in fact we have the approximate equation

For the HoC model, i.e. a maximally uncorrelated landscape, we have and therefore

*i.e.* this model shows strong random epistasis.

For RMF models, which are combinations of an additive landscape and a completely uncorrelated one, the correlation of fitness effects is

where and are the mean and variance of the additive fitness effects and is the variance of the uncorrelated HoC component. Therefore, in this case, the measure of epistasis is proportional to the variance contribution of the uncorrelated component.

### 2.3Epistasis for specific mutations

The correlation of fitness effects is also a useful measure of the interaction between specific mutations. Some simple generalizations of the measure are:

, which describes the epistatic effect of a mutation in locus on other loci:

, which describes the epistatic effects of other mutations on locus :

, which is a matrix that describes the epistatic effect of locus on locus :

These measures can also be generalized easily to multiallelic landscapes by considering pairs of mutations at different loci.

The measure , and especially are useful for exploratory and illustrative purposes, since they summarize the interactions between mutations in a clear and compact way, as it can be seen in Figure 7.

It is also possible to use the more direct measure as an alternative to . The difference lies in the normalization: , therefore treats both large and small mutations in the same way while is larger for large mutations. The choice of the most appropriate measure depends on the question, i.e. if the focus is on the interactions across all mutations, or only the largest ones.

### 2.4Decay of the correlation with distance

The measure provides information on the amount of epistasis but cannot discriminate between different types or models of fitness landscapes, as it occurs for any single measure of the amount/strength of epistatic interactions. In fact, there are many landscapes with widely different structure but with the same . For example, a HoC model realization would have as would a landscape composed by an equal mixture of additive and reciprocal sign epistatic interactions (like in an EMF model).

However, a natural and interesting extension of this measure is given by the full decay of the correlation of fitness effects with distance , which correspond to the cumulative epistatic effect of mutations and can be defined as:

where . As with , can be expressed in terms of the fitness correlation functions at distance , :

The decay of with the Hamming distance is an interesting object of study in itself , since it describes how the epistatic effects of different mutations interact with each other and their cumulative effect. The mean of can be computed analytically in most models of fitness landscapes - see the first section of the Appendix - and it brings extra information on the structure of the landscape. Different models have a different behaviour (Figure 3): RMF and HoC models show an abrupt fall already at and then a flat profile, while NK models have a gradual, approximately exponential decay with rate (Supplementary Figure 1). Models based on The Ising model (based on pairwise reciprocal sign epistasis) decays linearly until , while the eggbox (maximally epistatic, anticorrelated) oscillates between and .

Note that the fact that together with equation () implies that there is a general bound on the decay of the fitness correlation functions with distance. In detail, the bound is *i.e.*, the decay of fitness correlation functions is bounded by the 1-step correlation function.

### 2.5Correlation in signs ()

In many experimental situations, fitness is not clearly measurable on an absolute scale, but it is possible to rank the genotypes in order of increasing fitness, or at least to state if a mutation is deleterious or beneficial.

The fitness landscape can be then represented as an acyclic oriented graph, *i.e.* an oriented network where links between genotypes represent single, fitness-increasing mutations. Hereafter, we will refer to this graph as the fitness graph. As an example, the fitness graphs corresponding to different types of epistasis for 2 loci are illustrated in Figure b.

In this context, it is still possible to measure epistasis via the same method by employing a modified measure which uses just the sign of the fitness effects, instead of their value. We define as the sign of . A more robust variant would be

where is a tolerance parameter (possibly depending on the genotype, and larger than the experimental errors). The measure is defined as before:

If the landscape has no neutral mutations, we can show that this measure is related to other commonly employed measures for fitness graphs. Consider all possible pairwise mutational motifs in the fitness graph and classify the type of epistasis in each motif as magnitude epistasis, sign epistasis and reciprocal sign epistasis (see Figure b). We denote the fraction of motifs in each class by , and respectively. We have the relation (see proof in the Appendix)

What is even more interesting is that both in models and in real landscapes, the results of and are often numerically close and highly correlated (see below). The only exception is represented by landscapes with weak epistatic interactions dominated by magnitude epistasis, where . This suggests that could be used in place of for landscapes where only fitness ranks are known. These measures represent therefore a bridge between fitness graph-based measures and quantitative measures based on absolute fitness.

## 3Constraints in mutation order: *chains*

Many fitness landscape measures are correlated with the amount and strength of epistasis in the landscape. However, landscapes with similar epistasis could have widely different structure of epistatic interactions, origin and evolutionary properties. One of these properties is the amount of constraints on the possible evolutionary paths.

A natural measure of evolutionary constraints is given by the abundance and structure of maximally constrained paths. Our aim is to characterize these paths in a simple and effective way. For that, we focus on genotypes with only a single beneficial mutation. All fitness-increasing paths that pass through such genotype will share this mutation. Some landscape have “chains” of consecutive mutations with this properties (see Figure 1c). We now examine the abundance and size of these chains as measures of evolutionary constraints.

We define a *chain step* as a mutation that is the only possible fitness-increasing mutation from the genotype . Chain steps can occur one after another, forming a linear path of obligatory mutational steps, that we call a . Several chain steps can lead to the same genotype, but a genotype can have at most one outgoing chain step. For this reason, chains can form tree-like structures, that we call a *chain tree*. A chain tree is formally the set of all genotypes which are forced to evolve along obligatory paths up to a common final genotype, *i.e.* a maximal groups of connected chain steps. This definition implicitly assumes a strong selection regime [14], as there is no fixation for mutations of negative fitness effect. In an additive landscape, there is a single chain tree containing only those genotypes one mutation away from the (single) peak. Note that other landscapes can contain more than one chain tree (or none). We compute also the number of *origins*, that are all genotype that are initial points of a chain (that obviously excludes intermediate steps). And finally, we compute also the maximal *depth* of all chains in the landscape, that is, the maximum number of consecutive steps. In an additive landscape, the depth of the only chain tree is 1.

### 3.1Out-degree distribution of fitness graphs

Chains are a natural choice for a measure of evolutionary constraints in the framework of fitness graphs. In particular, chain steps are strongly related to the out-degree distribution of the fitness graph corresponding to the landscape.

In fact, there is a one-to-one correspondence between chain steps and genotypes with a single fitness-increasing mutation. The number of fitness-increasing mutations from a given genotype is the out-degree of the genotype in the fitness graph. Therefore, chain steps are simply nodes of out-degree 1.

One of the most well-studied measures of fitness landscapes is the number of peaks. Since there are no fitness-increasing mutations from a peak, peaks are simply nodes with out-degree 0. Therefore, the number of peaks is actually the first bin of the out-degree distribution. Similarly, the number of sinks correspond to the number of genotypes with out-degree (the last bin of the out-degree distribution). The number of chain steps is the second bin of the out-degree distribution of the fitness graph, and is therefore a natural step further in the characterization of the distribution of out-degree.

For the small empirical landscapes currently available (), chains contain most of the local information about evolutionary constraints. For larger landscapes, nodes with out-degree 2, 3, or more would also be relevant to assess the amount of constraints. For these landscapes, the full out-degree distribution could be an interesting object of study.

In Figure , are reported the out-degree distributions for four types of epistatic interactions (none, random, pairwise incompatibilities and compensatory at higher order) that correspond to four different theoretical landscape models (Additive, HoC, Ising and Eggbox). Results show that the out-degree distribution usually differs for different types of interactions. However, it is noteworthy to mention that Ising and HoC models show the same average out-degree distributions even though the nature of their epistatic interactions are fundamentally different as well as their overall structure (see Figure and details of the models in the Appendix).

Interestingly, only the number of chain steps is contained in theout-degree distribution. Indeed neither chain depth nor the number of chain origins can be computed fromjust theout-degree distribution. Both values really depend on the structure of thechain tree itself. A star-like tree, as we expect in additive landscapes, will have several origins and only a depth of 1, whereas a deep chain with a single origin is expected under an IMF model(Ising Mount Fuji is a mixture of additive with Ising pairwise interactions, see details of the models in the Appendix). In this last case, the chain corresponds to the subsequent replacements of alleles at neighboring loci starting from one edge up to the other. Chain depth and the number of origins somehow relate to correlations in the outdegree of nodes in the fitness increasing paths.This stresses the interest of studying both the out-degree distribution and the chain trees in fitness landscapes.

### 3.2Connection between chains and the amount of epistasis

In an additive landscape, all the genotypes around the peak are origins of a chain tree that ends at the peak. This starlike chain tree of depth 1 around the peak is the only chain tree in these landscapes. In contrast, in a HoC model, there are many small and slightly deeper chain trees; their number is of the order of (see proof in the Appendix). Therefore, it would be tempting to conclude that the number of chain trees (as well as other measures of chains) are correlated to the amount of epistasis. However, this is not the case.

For example, in an RMF model with equal additive contribution, it is possible to derive exact theoretical results for the mean of some chain tree measures –most relevant, the number of chain steps. Equations are presented in appendix B. The results are illustrated in Figure , where the mean of several chains measures are reported as a fraction of additive component in the RMF model. Because the HoC component is fixed (fixed ), the higher , the more additive is the model ( is the mean additive effect of mutations) ; consequently, when , the model becomes additive whereas it converges to an HoC model when .

Furthermore, even though chain measures for other models (e.g. IMF and EMF) cannot be computed analytically, they can be retrieved from simulations (Figure ). Interestingly, several of the chain measures appear to be non-monotonic with respect to epistasis in the RMF model and even more so in the IMF model. Both the number of steps and the chain depth tend to have a maximum for intermediate values of epistasis, when the contributions of the additive component and of the interactions are comparable (Figures Figure 5 and Figure 6). Interestingly the type of interaction plays a important role in the structure of the chains, as only short chains are observed for the RMF and EMF models whereas long chains are observed for the IMF model. Furthermore, the size of the landscape also changes the dependence between epistasisand chain measures (depth and abundance) (Figure ).

### 3.3Generalized chains

We can generalize the concept of chain by including cases where there is more than one fitness-increasing mutation from a given genotype, but all paths starting with these mutations eventually lead to the same genotype.

We can define a *generalized chain step* as a pair of genotypes such that all fitness-increasing paths from pass through . Note that in this case, a step can encompass one or more genotype(s) between and , whereas there is none in the original definition. A generalized chain can then be defined as a sequence of generalized chain steps. To compute the depth, we count all genotypes chains go through.

Generalized chains can also be interpreted in terms of constraints on evolution, but they are intrinsically non-local so they are not related to the out-degree distribution of the fitness graph. Instead, the total size (number of genotypes in the chain steps) of the generalized subchain ending at a genotype corresponds to the size of the *exclusive* basin of attraction of the genotype, the set of genotypes that will inevitably evolve through . Any path of increasing fitness starting at these genotypes will go through the genotype .

This is particularly interesting for peaks, for which it captures their exclusive basins of attraction. Non exclusive basins of attraction are usually defined as genotypes from which the peak can be found [17] and their size is commonly interpreted as a measure of the evolvability in the landscape. The exclusive basin of attraction of a peak measures how many genotypes have an obligatory end point, *i.e.* how many genotypes are forced to evolve towards a single final state.

## 4Relations between measures

In this section we discuss the relations existing between the newly proposed measures and the existing ones.

We expect that the new measures of epistasis and would be correlated to other measures of epistasis. In fact, we already discussed how they are related to some of the existing measures. In particular, (i) for pairs of loci, is directly related to the common definition of epistasis as : in fact, for the whole landscape we have , while for a pair of mutations ; (ii) for the whole landscape, can be rewritten as a function of the fitness correlation functions as ; (iii) is directly related to the number of square motifs with sign and reciprocal sign epistasis as .

Furthermore, it is possible to show that is a function of the Fourier spectrum of the landscape, provided that the standard orthonormal basis is used for the Fourier series. If we denote by the normalized weight of the coefficients of order in the Fourier spectrum, *i.e.* the sum of the squared coefficients of all -loci interactions normalized by the sum of all squared coefficients, the relation is (see proof in the Appendix). Our measure of epistasis is therefore which resembles another measure ofepistasis [35], showing again the close relation with previous measures of epistasis. The main difference is the weight of higher-order interactions: the contribution of -loci interactions to grows like for large , so that the effect of interactions is stronger if they involve more loci.

We also discussed how the number of chain steps, the number of peaks and sinks correspond to three different components of the out-degree distribution of the fitness graph. On the other hand, we also suggested that there is no direct relation between the amount of epistasis and the number of chains.

To evaluate in a more systematic way the relations between these and other measures, we perform a correlation analysis similar to [35] but using models instead of experimental landscapes. We select the number of peaks, the number of sinks, the roughness/slope ratio (ratio between epistatic “noise” and additive component, see Appendix), and as measures of epistasis, plus the number of chain steps and the maximum chain depth. We compute the Spearman correlation coefficients of all pairs of measures in the RMF landscape model varying the model parameters (in particular, the ruggedness).

peaks | sinks | steps | depth | ||||
---|---|---|---|---|---|---|---|

peaks | 1 | - | - | - | - | - | - |

sinks | 0.53 | 1 | - | - | - | - | - |

0.60 | 0.62 | 1 | - | - | - | - | |

0.73 | 0.71 | 0.76 | 1 | - | - | - | |

0.77 | 0.73 | 0.71 | 0.88 | 1 | - | - | |

steps | 0.09 | 0.01 | 0.001 | 1 | - | ||

depth | 0.007 | 0.11 | 0.08 | 0.07 | 0.08 | 0.34 | 1 |

peaks | 1 | - | - | - | - | - | - |

sinks | 0.69 | 1 | - | - | - | - | - |

0.72 | 0.45 | 1 | - | - | - | - | |

0.73 | 0.46 | 0.99 | 1 | - | - | - | |

0.72 | 0.73 | 0.72 | 0.73 | 1 | - | - | |

steps | 0.25 | 0.01 | 0.09 | 0.09 | 1 | - | |

depth | 0.08 | 0.02 | 0.02 | 0.20 | 0.68 | 1 | |

peaks | 1 | - | - | - | - | - | - |

sinks | 1.00 | 1 | - | - | - | - | - |

0.74 | 0.74 | 1 | - | - | - | - | |

0.74 | 0.74 | 0.97 | 1 | - | - | - | |

0.99 | 0.99 | 0.74 | 0.74 | 1 | - | - | |

steps | 0.97 | 0.96 | 0.71 | 0.71 | 0.95 | 1 | - |

depth | 0.93 | 0.93 | 0.67 | 0.68 | 0.91 | 0.96 | 1 |

The pairwise correlations (Table 1) confirm the intuition that the measures related to epistasis are all strongly correlated, the strongest correlation being between and as expected. For RMF and IMF, the correlation between these measures and the chain measures, on the other hand, is especially low. This is not the case for the EMF model. This shows that the chain measures quantify some landscape properties that is not simply correlated to the amount of epistasis.

Interestingly, the behavior of chains with epistasis is also apparent when we compare them across different models of epistatic interactions. In Figure 6, we show what happens in fitness landscapes with an additive contribution plus different epistatic models (HoC, Ising and Eggbox). The comparison between measures of epistasis and chain shows clearly that there is no simple relation between them, as all three models of interactions have different behavior. Chain measures appear to depend strongly on the nature of the epistatic interactions, and therefore could provide useful, independent information. The strong correlations between chain measures and epistasis observed for EMF relates to the observation that the chain measures are almost a two-steps constant function for EMF with a variable component of additive model.

## 5Measures on two experimental landscapes

As an example of application of the new measures and chains, we use them to analyze two complete experimental landscapes of size . These landscapes are illustrated in Figure 7.

The first landscape is the landscape of antibiotic (cefotaxim) resistance of -lactamase mutations in an *Escherichia coli* plasmid from [41] (Figure 7 left). The 5 mutations have a very strong effect that together give a increase in antibiotic resistance and were therefore selected together. Given the huge selective advantage of the combined mutations, this landscape is single-peaked, where the peak corresponds to the five-point mutant. It also has a single sink, that interestingly does not correspond to the wild type.

The second is one of the four complete sublandscape (csI) [12] of a larger landscape () of deleterious mutations in *Aspergillus niger* from [36] (Figure 7 right). This landscape is a combination of unrelated deleterious mutations where epistatic interactions were not filtered by natural selection. This landscape has 4 peaks and 2 sinks; in fact, at present it is one of the most rugged among the completely resolved landscapes.

As the landscapes were derived in completely different settings (co-selected beneficial for -lactamase and random deleterious for ), we might not be surprised to find that these landscapes exhibit very different structures. Indeed theoretical arguments support the intuition that landscapes of co-selected mutations differ radically from landscape of random mutation [11]. The difference in ruggedness between -lactamase and landscapes is confirmed by the values of (0.85 *vs* 0.33) and (0.43 *vs* 0.89).

To further explore the landscapes, we compute the matrices to illustrate and summarize the interactions between mutations (Figure 7b). In the -lactamase landscape, there are some clear interactions between mutations (between the 2nd or the 4th and the 1st mutations, or between the 5th and the 4th) but none of these interactions is characterized by strong sign epistasis (no red cell). On the other hand, the landscape contains several examples of interactions dominated by strong sign epistasis (for example, between the 2nd or the 4th and the 1st or the 5th mutations).

Similar conclusions come from the analysis of the decay of with distance. The decay in the -lactamase landscape is immediate but decays slowly after the first mutation (Figure 7b left), resembling the behavior of RMF models.

An interesting example of the power of is represented by the landscape (Figure 7b right). This landscape shows a non-monotonic decay, with correlation bouncing up and down. This indicates a clear compensatory structure of reciprocal sign epistasis, which is not only due to pairwise compensation, but extends to distance 4, *i.e.* to the whole landscape. In fact, the behavior of suggests a mixture of an RMF landscape and an extreme case of compensatory interactions, like the Eggbox model. This surprising result does not come out in a straightforward way while looking at other measures, even when looking at the Fourier spectrum [24]. Indeed, although the coefficient of the highest order in the Fourier decomposition measures the amount of eggbox, it compares to coefficients of smaller orders that have a complex intermingling when epistasis is not purely reciprocally signed at all orders. These coefficients contribute as well to the behaviour of .

Finally, for both landscapes, the number of steps and the maximum depth of chains are higher than expected theoretically. For the -lactamase landscape, the strength of epistasis suggests that this model should be close enough to an additive model that there should be a single chain of depth 1 and five steps ending at the peak, while we observe two chains with nine steps and their maximum depth is 3. This suggests that epistasis in this landscape is not random, but is structured in a way that constrains evolution and that is not captured by any of the models presented here. On the other hand, the number of steps and the depth of the single chain in the landscape are higher but not too different from the values expected for a RMF or EMF landscape, which is consistent with the above suggestion that this landscape resembles a mixture of RMF and EMF.

## 6Discussion

In this work, we presented two new sets of landscape measures which have a simple interpretation and cover a range of potential applications. These measures among others have been implemented in MAGELLAN, a graphical tool to explore small fitness landscapes [6].

The first application is the measure of epistasis in a comparable way across landscapes. The correlation of fitness effects is a natural measure for this. This measure can be used also for pairs of mutations, to explore the strength of epistatic interactions between mutations in a compact way.

In terms of , there is a natural scale for the strength of epistatic interactions, from purely additive interactions (), through strong random interactions () to a fully compensatory landscape (). Interactions in landscapes with are dominated by strong sign and reciprocal sign epistasis between most loci, therefore we expect such landscapes to be rare and possible only for some sets of mutations, as selection tends to favor mutations with positive interactions. In fact, the two experimental fitness landscapes analyzed have positive values of . Yet, the amount/strength of epistasis in the landscape by de Visser et al. is remarkably high: means that the fitness effect of a mutation in a given genotype, is a poor predictor of the fitness effect of the same mutation in a neighbor genotype that only differs by a *single* mutation.

Correlations of fitness effects are not only useful to quantify epistasis. Their decay contains information on the nature of epistatic interactions and can reveal interesting signals. A clear example of that is the landscape studied here. The correlations for this landscape show an oscillatory behaviour instead of the expected decay for random epistasis (*i.e.* HoC like) or for incompatibilities (*i.e.* Ising like), pointing towards a strong contribution of “eggbox-like” epistasis (reciprocal sign epistasis across multiple mutations). While the presence of pairwise reciprocal sign epistasis is not strange - it is actually quite common in compensatory interactions - the fact that reciprocal sign epistasis involves the whole landscape is quite surprising. In other words, starting from a first mutation chosen to be deleterious, it is not unreasonable that the second mutation could have a compensatory effect, but the mechanism behind the deleterious effect of the *third* mutation and the compensatory effect of the *fourth* mutation is obscure. It relate to complex pathways of interactions at the molecular level.

Many measures can only be computed if one has fitnesses for all combinations of the set of mutations (or subsets of). For example, the number of peaks lose meaning in a landscape with missing data, since the definition of a fitness maximum requires the knowledge of the fitness of all its neighbors. Since the fitness correlation functions can be computed even with missing data, the correlation of fitness effects can be estimated from equation (Equation 7) even for very sparse landscapes. The sparseness of the landscape could increase the error on the estimate, however this effect could be compensated by the larger size of the landscape. Landscapes containing a larger number of mutations would be also more representative of real gene or protein landscapes.

For some landscapes, only fitness ranks or the beneficial/deleterious nature of the fitness effects can be experimentally determined. Our measure is appropriate for these landscapes. While depends not only on positive and negativeepistasis, but it is sensitive to its strength, is based essentially on the fitness graph and therefore depends only on the sign of epistasis. and are strongly correlated across fitness landscape models.Thus a mismatch between and in real landscapes could point to some peculiarnature of epistatic interactions.

Finally, the and measures could also be useful to estimate parameters oftheoretical landscape modelsfrom empirical data, thanks to the availability of approximate analytical formulae for these quantities. For example, assuming that the underlying model of a landscapeis the NK model, themeasures is an approximately unbiasedMethod-of-Moments estimator of the parameter , *i.e.* (see eq. ). A similar approach can be used for the parameters of other landscape models. The potential of thesemeasures for model inference and goodness-of-fit tests is yet to be studied.

The other novel measures that we proposed are the number of steps and depth of chains, *i.e.* mutations that are obligatory under the strong selection regime in the landscape. These measures have an immediate evolutionary interpretation, in terms of evolvability and constraints, yet they show peculiar properties. The most relevant one is that they are often non-monotonic in epistasis, as we have shown analytically. The lack of correlation between epistasis and chains shows that these new statistics can be used to obtain independent information about *nature* of the interaction, instead of their strength.

Interestingly, the number of chain steps, the chain depth (Figure ) and the total number of accessible paths [35] seem to be peaked at intermediate values of epistasis . All these measures are related both to evolvability - deep chains show that fitness-increasing paths are open - and to constraints - chains represent obligatory paths in evolution. This suggests an evolutionary interpretation of these peaks in terms of the tradeoff between evolvability (higher at low epistasis) and constraints (stronger with high epistasis). Previous measures that were shown numerically and by some analytical approximations to be non-monotonic include the total number of accessible paths [35] and the number of exceedances, *i.e*. the number of available fitness-increasing mutations after an evolutionary step [25], which are also related to evolvability and constraints. It is worth mentioning that chains and exceedance are both related to the out-degree distribution after one step of increasing fitness. This suggests that beyond the out-degree distribution, it is perhaps worth characterizing the sequences of out-degrees, e.g. the out-degree distribution along evolutionary paths.

Real landscapes tend to have longer chains than expected according to theoretical landscape models with random epistasis. This is the case for both experimental landscapes studied here. This is especially apparent and interesting in the landscape by Weinreich et al., since it implies some highly non-random structure of epistatic interactions for this set of mutations. This result has been found independently by randomization tests (Weinreich, personal communication) and cannot be seen in complex measures of epistasis like the Fourier spectrum or the decay of correlations , however our chain measures were able to capture this signal.

The notion of obligatory steps can be easily widen using the above definition of generalized chains. However, it is worth mentioning that the generalized chains say little about the local constraints in the fitness landscape. Quite on the contrary, they give information on exclusive basins of attraction for peaks. Therefore, typically, sizes of exclusive basins of attraction (generalized chains) are very informative about the *convergence in state* (similar endpoints regardless of the path). The larger the exclusive basins, the less uncertainty remains on the state, given a starting genotype. Quite on the contrary, strict chains report local information of the landscape. They measure how path are constrained but do not necessarily predict what will be the final state. An interesting perspective would be to redefine the chains on the mutations themselves and not on the genotypes (e.g. mutation at locus is always followed by a mutation at locus , independently of the genotype). Quite clearly, strict chains are informative about the *convergence in path* in the landscape (similar path regardless of the endpoints) rather than the convergence in state.

Chains are natural measures both from the evolutionary point of view and from the mathematical point of view, since they are related to the out-degree distribution of the fitness graph. For the small complete landscapes currently available, the number of peaks, chains and sinks summarize most of the information present in the out-degree distribution. However, for larger landscapes or for incomplete ones, other components of the out-degree distribution (or other properties, like its variance) could be useful as measures for a finer characterization of fitness landscapes.

We are still far from predicting evolution on real landscapes based on their measures, partly because of the incomplete knowledge of the structure of real landscapes, and partly because of the lack of measures with a natural evolutionary interpretation. In the future, we expect to witness a strong increase in the number of published empirical landscapes that will be experimentally resolved. The measures that we propose here will therefore find applications in the understanding and classification of these landscapes, as well as in studies of model landscapes. The correlation of fitness effects is a natural measure of epistasis that is comparable across landscapes, while the decay of correlations with mutation distance and the new chain measures will be useful tools to discriminate and classify these landscapes. Chains also highlight the interplay of constraints and evolvability that influence evolution on complex landscapes.

## 7Acknowledgments

We thank Joachim Krug, Ivan Szendro, Johannes Neidhart, Arjan de Visser, Nick Barton, RA Watson for useful comments and discussions. The work was funded by grant ANR-12-JSV7-0007 from Agence Nationale de la Recherche. BS acknowledges support from the Bonn-Cologne Graduate School.

## AAppendices

### a.1Common landscapemeasures

In this section we present some common landscape measures that can be applied as statistics for experimental landscape data. Notation: a genotype is a sequence of alleles of length . For biallelic landscapes, and .

Some of the most commonmeasures for fitness landscapes are:

number of peaks [40]: it is the number of genotypes such that all their neighbours have lower fitness,

*i.e.*the number of local fitness maxima.(roughness/slope) ratio [2]: the landscape is fitted to a linear model (a linear combination of s plus a constant) by least squares. The slope is the average modulus of the coefficients of the s. The roughness is the quadratic mean of the residuals of the regression. The measure of epistasis is their ratio .

fraction of epistatic interactions [43]: the fraction of all pairs of mutations from all possible genotypes that show magnitude, sign or reciprocal sign epistasis.

number of accessible paths [41]: assume that the absolute fitness maximum corresponds to the genotype . Count the number of paths of mutations starting from to such that fitness increases after each mutation. This is the number of direct accessible paths to the maximum from its antipodal genotype.

Fourier expansion and spectrum [33]: The coefficients of the Fourier expansion are uniquely defined in terms of the Fourier decomposition

where are ordered sets. The Fourier spectrum is defined by the sum of squared coefficients for interactions of loci: . Epistasis is usually measured by .

More details can be found in the review by [35].

### a.2Models of fitness landscapes

In this section we briefly illustrate some common models of fitness landscapes that will be used in this study. Most of them are illustrated in Figure 8. Please note that we only considered here models of biallelic loci. A mathematical formulation of these models is given in the next section.

#### The Additive model (a.k.a. multiplicative model)

This is a model for non-interacting mutations with independent fitness effects. The fitness is simply the product of the fitness contributions of each locus: fitness effects of different mutations are multiplied. In log-scale, this corresponds to summing the fitness effect of each mutation; for this reason this models is called “additive”. Here, the fitness effects of different mutations are randomly drawn from a Gaussian distribution with mean and variance . As there an independent contribution of each locus, the dimension of interaction is (since each locus “interacts” only with itself).

In terms of Fourier decomposition, in this landscape all coefficients of second order and higher are zero.

#### The House-of-Cards (HoC) model

This is a model for random, uncorrelated fitness landscapes [20]. The fitness of each genotype is independent on the fitnesses of other genotypes. Here, it is randomly drawn according to a Gaussian of mean 0 and variance . As this models corresponds to full interaction between the loci, the dimension of interaction is .

In terms of Fourier decomposition, the coefficients are random variables with a marginal Gaussian distribution centered in 0.

#### The Rough Mount Fuji (RMF) model

This model interpolates between additive and uncorrelated fitness landscapes by adding the two [3]. The fitness is computed as the sum of an additive contribution and a HoC contribution. Here, the model is tuned by three parameters: mean and variance for the additive part and variance for the HoC part. (In the literature, this model is often defined with constant additive fitness effects, *i.e.* ). The model converges to an additive model when and to a HoC model when . The dimension of interactions is a mixture of dimension and dimension .

The Fourier decomposition is a linear function of the landscape, so it is a combination of the additive and the HoC decompositions.

#### The NK model

This landscape model with loci interpolates between additive and uncorrelated fitness landscapes by combining uncorrelated fitness contributions (*i.e.* small HoCs) from groups of loci in an additive way [19].There are different ways to choose the groups of interacting loci and while several properties such as mean number and mean height of local optima depend only weakly on the particular choice made [40], others seem to behave quantitatively different for some interaction choices [32]. Nonetheless it has been shown that the fitness correlation function is strictly independent of the interaction choice [7] and consequently does not depend on it either, while the number of chains may still be influenced by it. The number of interacting loci is and the interpolation is controlled by the parameter : corresponds to an additive model with independent contributions from each locus, while corresponds to an HoC model. The dimension of interaction is .

#### The Ising and the IMF models

This model originates from statistical physics [23], but has an immediate interpretation in terms of pairwise allele incompatibilities. In this model, each pair of interacting loci with different alleles causes a reduction in fitness. Here, loci interact only if they are neighbors in the genotype sequence (locus interacts only with locus and ) and the first and the last locus have a single interaction (loci are arranged on a string). The fitness cost for each pair is drawn from a Gaussian with mean and variance . More general models based on allelic incompatibilities correspond to the Sherrington-Kirkpatrick model and other spin glass models in statistical physics [23]. The dimension of interaction is as interactions only occurs between pairs. We also combined incompatibility interactions (Ising model) with an independent fitness contribution (additive model) in an “Ising Mount Fuji” (IMF) model in the same way the RMF is set.

In terms of Fourier decomposition, in this landscape all coefficients of third order and higher are zero.

#### The Eggbox and the EMF models

This model represents the extreme example of reciprocal sign epistasis of highest dimension. In this model, all genotypes in the landscapes have either low or high fitness. All the neighbours of a high-fitness genotype have low fitness, and vice versa. Therefore, in this landscape, each mutation is either deleterious (from high to low fitness) or compensatory (from low to high fitness). Fitnesses are given by a Gaussian with mean and a small variance . The dimension of interactions in this landscape is . We also combine the eggbox interactions with independant contributions in an “Eggbox Mount Fuji” (EMF) landscape that is built like the RMF or the IMF.

In terms of Fourier decomposition, this landscape is dominated by the contribution of the th-order term (that is, the term of highest order).

### a.3Formulae for in model landscapes

We assume an observed fitness given by the model

that is, an additive contribution , an epistatic contribution from some fitness model, plus the effect of measurement errors . We assume these errors to be unbiased and uncorrelated: , where is the Kronecker delta, *i.e.* when and 0 otherwise.

We define the mean squared additive effect and the mean squared experimental error .

The expected value of can be computed approximately by taking the ratio of the expected values of numerator and denominator of eq. (Equation 6) rearranged as eq. (Equation 3), instead of the expected value of the ratio (the sign in all our formulae refers to this approximation). The result is

with being the analogue of restricted to the epistatic contribution. Note that if is the statistic of the epistatic contribution .

#### Additive model

In these models and the only reduction in correlation is due to experimental noise:

#### RMF and HoC models

In these models, corresponds to the HoC model, *i.e.* they are i.i.d. random variables. Denote by the variance of the distribution of fitnesses in the HoC model:

Since for a Gaussian distribution of additive fitness effects, we obtain equation (Equation 5) for and .

#### NK models

In these models, where the s are i.i.d. random variables that depend on and other indices, randomly chosen. The s have mean and variance .

The fitness correlation function is known exactly for the pure NK model [7]: . With eq. (Equation 7) it is straightforward to compute from this. The variance is given by because on average of the change in a single mutation, each of these differences having twice the variance of the fitness contribution (since the variance of the difference of two i.i.d. variables is twice their variance). The final result is

Substituting and yields equation (Equation 4).

#### Ising model

In the following, we define . Using the above notations, in these models, where the incompatibility coefficients are randomly extracted from a Gaussian distribution with mean and variance . We define .

A mutation at locus will invert contributions of the terms containing and adding to . At the edge of the genome, loci interacts with only one neighbor, so this is reduced by a factor of 2, thus .

Only mutations at and affect the value of . Choosing mutations out of and applying the hypergeometrical distribution, there are probabilities and to choose both or exactly one of them, respectively. Each relevant mutation changes the effect of mutating by . Thus (ignoring boundaries) On the boundary only one mutation can influence , resulting in a reduced contribution of and so together:

and therefore the result is

#### Eggbox model

In this model, , i.e each mutation switches the fitness from the highest value () to the lowest () or the other way. The difference in the epistatic fitness effects from two genotypes separated by an odd number of mutations is , while it is 0 from two genotypes separated by an even number of mutations, therefore

### a.4Formulae for chains in RMF

We consider a RMF model with equal additive fitness contribution for each mutation, in addition to an HoC model with distribution and cumulative distribution .

Sorting the genotypes in order of their additive fitness, there are genotypes at the th level with mutations with positive additive fitness effect and with negative one. The average number of chain steps can be obtained as the sum over all possible steps of the probability of being a chain step. It is

The number of origins and of chain trees can be obtained in a similar way using the Cayley tree/Bethe lattice approximation. In this framework, we approximate locally the hypercube by a Cayley tree, *i.e.* a tree with L branches at each node. This means that we neglect the overlapping between the next-to-nearest neighbours of the genotype considered and we assume them to be independent genotypes instead of .

The probability that a genotype is the origin of a chain is product of the probability of having out-degree 1 and that all the other neighbours of lower fitness have out-degree different than 1:

where we define the probabilities that a neighbour genotype is the starting or ending point of a chain step ending at level :

The probability that a genotype is the endpoint of a chain is the difference between the probability of being the final genotype of a chain step and the probability of being an intermediate point in a chain. The results are

These formulae could be further simplified if is the Gumbel distribution [25].

### a.5Proofs

#### Relation between and type of epistasis

The most extreme values of for different types of epistasis are obtained in the case . Denote by the log-fitness values. The function is defined as

Since is a correlation, . is a continuous function and it is also invariant under permutations of loci and alleles, so from now on we will restrict to the subspace with . Each of the three partitions of this subspace corresponding to magnitude, sign and reciprocal sign epistasis is connected, therefore the image of each one of them under is an interval.

By definition, magnitude epistasis results in since all fitness jumps have the same sign. The extreme values 0 and 1 are both realized: in landscapes with , while in landscapes with . Therefore, for magnitude epistasis, .

Reciprocal sign epistasis require that all fitness jumps change in sign after a mutation, therefore results in by definition. The extreme case is realized in the landscape , while the other extreme case is realized for the landscapes with for . Therefore, for reciprocal sign epistasis, .

Finally, sign epistasis can have both signs of . It is easy - although tedious - to show that there are no critical points of inside the space of landscapes with sign epistasis (with ), therefore the extremal values should appear on the border. There are essentially two borders: and . On the first border, we have and the upper limit is for the landscapes with . On the second border, we have that reaches the minimum value at the landscapes with (imposing the derivative with respect to to be null). Therefore, for sign epistasis, .

#### Relation () between and the fitness correlation function

Denote the fitness correlation function at distance by . We use the identity

Averaging over the all mutations and all genotypes, then dividing by , the above terms give rise to:

Summing over genotypes and mutations, the numerator of (Equation 6) becomes . The denominator of (Equation 6) can be computed in a similar way by choosing , obtaining . is their ratio

and since , we obtain the result (Equation 7).

#### Relation () between and the fractions of square motifs

We consider a landscape without ties (*i.e.* all genotypes have different fitness values). In this case, and therefore where the average is over all genotypes and pairs of mutations, or equivalently over all square motifs and over their sides. The average over the two sides of a motif is with magnitude epistasis, with sign epistasis and with reciprocal sign epistasis. To obtain the global average, we multiply these results by the fraction of motifs of each kind, *i.e.* . Since they sum to 1, we have and substituting it we obtain the relation ( ?).

#### Number of chain steps in the HoC model

The average number of chain steps is given by the number of mutations in the landscape, , multiplied by the probability that the mutation is a chain step. This is the probability that among the initial genotype and all its neighbours, the final one is the most fit and and the initial one is second in the fitness rank. Since fitness values are random and uncorrelated, the probability that a value is maximum among is and the conditional probability that another value is second is , therefore we have that the average number of steps is .

#### Relation () between and the Fourier spectrum

We define . Since the Fourier basis is orthonormal, each component of the spectrum gives an independent contribution to the numerator and denominator of . The contribution of each to the denominator of (Equation 3) is since there are nonzero mutations with fitness effect