# The Differential Geometry of Homogeneity Spaces Across Effect Scales

###### Abstract

If an effect measure is more homogeneous than others, then its value is more likely to be stable across different subgroups or subpopulations. Therefore, it is of great importance to find a more homogeneous effect measure that allows for transportability of research results. For a binary outcome, applied researchers often claim that the risk difference is more heterogeneous than the risk ratio or odds ratio, because they find, based on evidence from surveys of meta-analyses, that the null hypotheses of homogeneity are rejected more often for the risk difference than for the risk ratio and odds ratio. However, the evidence for these claims are far from satisfactory, because of different statistical powers of the homogeneity tests under different effect scales. For binary treatment, covariate and outcome, we theoretically quantify the homogeneity of different effect scales. Because when homogeneity holds the four outcome probabilities lie in a three dimensional sub-space of the four dimensional space, we can use results from differential geometry to compute the volumes of these three dimensional spaces to compare the relative homogeneity of the risk difference, risk ratio, and odds ratio. We demonstrate that the homogeneity space for the risk difference has the smallest volume, and the homogeneity space for the odds ratio has the largest volume, providing some further evidence for the previous claim that the risk difference is more heterogeneous than the risk ratio and odds ratio.

Key Words: Additive interaction; Binary outcome; Heterogeneity; Multiplicative interaction; Odds ratio; Risk difference; Risk ratio; Scale-dependence

## 1 Introduction

Practitioners are often interested in generalizing their finding across different subgroups or populations. This transportability problem (Pearl and Bareinboim 2014; Keiding and Louis 2016) relies on homogeneity of the effect measure across these subgroups or subpopulations. However, homogeneity, or equivalently the absence of interaction, is scale dependent as documented in the classical psychology literature (Krantz and Tversky 1971; Bogartz 1976; Loftus 1978) and statistics literature (Box and Cox 1964; Cox 1984; Darroch and Borkent 1994; de Gonz’alez and Cox 2007). For recent discussion, see Wagenmakers et al. (2012), Ding et al. (2016) and Keiding and Louis (2016). Therefore, if the treatment effect is more homogeneous under one scale, then it would be preferable to use this scale as a measure of the treatment effect for transportability across populations. For a binary outcome, according to Poole et al. (2015)’s review, clinical reports and general guidelines often assert that the treatment effect is more heterogeneous under the risk difference scale than under the risk ratio and odd ratio scales. These claims in the literature are based on surveys of meta-analyses showing that rejections of the null hypotheses of homogeneity, or equivalently the absence of interaction, happen more often under the risk difference scale than under the risk ratio and odds ratio scales. However, the evidence for these claims are not satisfactory, because different rejection rates of the null hypotheses of homogeneity may simply due to different statistical powers of tests. This is true even when there is no effect in one of the two subpopulations so that there is effect heterogeneity on all scales and arguably to the same degree (Poole et al. 2015).

Recognizing this gap in the literature, for binary treatment, covariate and outcome, we use results from differential geometry to quantitatively compare the relative homogeneity of the risk difference, risk ratio and odds ratio. Our comparison has three aspects. First, homogeneity, or no interaction, restricts the four outcome probabilities to be in the three dimensional space. For different effect measures, these spaces have different domains with different volumes. We find that the volume of the domain is the smallest for the risk difference, and is the largest for the odds ratio. This finding quantifies more exactly the corresponding fact that once three outcome probabilities are fixed, it is always possible to find a fourth that would lead to odds ratio homogeneity but that it is not always possible to do so for the risk ratio and risk difference scales. Second, although these three spaces of homogeneity have measure zero in the four dimensional space, they have positive volumes in the three dimensional space. Our calculation shows that the volume of the subspace is the smallest for the risk difference, and is the largest for the odds ratio. This theoretical result demonstrates that odds ratio homogeneity holds for relatively more values of the outcome probabilities than risk ratio homogeneity, which further holds for more values of the outcome probabilities than risk difference homogeneity. Third, we compare the volumes of the acceptance regions of Wald-type tests against the null hypotheses of homogeneity for the risk difference, risk ratio and odds ratio, finding that the acceptance region has the smallest volume for the risk difference, and has the largest volume for the odds ratio. This result thus partially also explains the reason why the null hypotheses of homogeneity were rejected more often for the risk difference than for the odds ratio, a phenomenon that arises repeatedly in surveys of meta-analyses (Katerndahl and Lawler 1999; Engels et al. 2000; Sterne and Egger 2001; Deeks 2002; Deeks and Altman 2003).

The remainder of the paper proceeds as follows. Section 2 introduces the notation and definitions for homogeneity and interaction, and states our evidence for the claim that the risk difference seems more heterogeneous than the risk ratio and odds ratio. For the risk difference, risk ratio and odds ratio, Section 3 compares the volumes of the domains of the spaces of homogeneity of the outcome probabilities; Section 4 compares the volumes of the spaces of homogeneity of the outcome probabilities; Section 5 compares the volumes of the acceptance regions of the Wald-type tests for the null hypotheses of homogeneity. We conclude in Section 6.

## 2 Notation, Definitions, and Conclusion

Assume that we have binary treatments or covariates and , and a binary outcome Define the outcome probability as for In some cases, and may be the gene and environment exposures, and we are interested in gene-environment interaction. In other cases, may be a binary covariate and a binary exposure or treatment, and we are interested in the differential treatment effects of on the outcome given different levels of Based on the four outcome probabilities , we introduce concepts of interaction under different effect scales (Darroch and Borkent 1994; VanderWeele 2015). For instance, we say that there is no interaction between and on the risk difference scale if

there is no interaction between and on the risk ratio scale if

there is no interaction between and on the odds ratio scale if

We say that the treatment effect of on the outcome is homogeneous across groups of , if there is no interaction between and . Clearly, the definition of homogeneity is scale dependent. It is possible that the treatment effect of on the outcome is heterogeneous under all scales; it is also possible that the treatment effect of on the outcome is homogeneous under one scale, but heterogeneous under another scale.

One important question is under which scale the treatment effect is more homogeneous. This question has profound impact in practice, because under a more homogeneous scale the treatment effect is more likely to be transportable to other subgroups or subpopulations. Below, we quantitatively compare the three effect scales in three ways, providing theoretical evidence for the claim that the risk difference is more heterogeneous than the risk ratio and odds ratio.

## 3 Domains of the Homogeneity Spaces

Assume , which lies in a four dimensional space. If we do not impose any restrictions on the ranges of the probabilities, then we can take ; if we consider only rarer outcomes, then we can take to be a smaller number, e.g., .

In the absence of interaction on the risk difference scale, is a deterministic function of and Because the probabilities are bounded between and , this deterministic relationship restricts and to be within the following region:

In the absence of interaction on the risk ratio scale, is a deterministic function of and , which have the following constraints:

In the absence of interaction on the odds ratio scale, the deterministic relationship restricts and to be within the following region:

We can see that the homogeneity spaces have different domains for different effect scales. When , there is no restrictions on the outcome probabilities on the odds ratio scale, and is simply ; however, the restrictions on the outcome probabilities exist for any values of on the risk difference and risk ratio scales.

Define the volumes of the domains and as and . We first argue, based on the volumes of these domains, that the risk difference is more heterogeneous than the risk ratio and odds ratio.

###### Theorem 1.

The three dimensional volume of the domain is

the three dimensional volume of the domain is

the three dimensional volume of the domain cannot be easily calculated explicitly, and the numerical values of are tabulated at different values of below:

We present the formula of in the Appendix, and the numerical values of are computed via both Monte Carlo and numerical integration. Note that is the reference volume of the region . From the above theorem, at , the volumes of the domains are both under the risk ratio and odds ratio scales, because corresponds to the rare outcome cases, and the risk ratio and odds ratio are very close. At , the volumes of the domains are and under the risk difference, risk ratio and odds ratio, respectively.

## 4 Volumes of the Homogeneous Spaces

In the absence of interaction on the risk difference scale, lies in the following three dimensional space:

in the absence of interaction on the risk ratio scale, lies in the following three dimensional space:

in the absence of interaction on the odds ratio scale, lies in the following three dimensional space:

In the four dimensional space , the three dimensional sub-spaces and all have measure zero. However, it is reasonable to compare the three dimensional volumes, and , of the three sub-spaces of the four dimensional space. See the Appendix for more technical discussion about the volume of a low dimensional space in a high dimensional space.

###### Theorem 2.

The three dimensional volume of the space is

the three dimensional volume of the space is

the three dimensional volume of the space has a complicated form presented in the Appendix, and is tabulated at different values of below:

Again, for , the volumes of the homogeneity spaces for the risk ratio and odds ratio are the same; for any value of , the volumes of the homogeneity spaces for the risk difference, risk ratio, and odds ratio are in an increasing order; at , the three dimensional volumes of the homogeneity spaces are and for the risk difference, risk ratio and odds ratio, respectively.

The differences of the three dimensional volumes are due to two reasons. First, the domains of the three spaces and have different volumes as shown in Theorem 1. Second, the spaces and have different “curvatures”, which result in the differences of the volumes after taking account of the differences in the domains. We summarize this result in the following Corollary.

###### Corollary 1.

The ratio is ; the ratio is

the ratio of is tabulated below:

By comparing the ratio between the volumes of the three dimensional spaces and their domains, the risk difference is still the most heterogeneous.

## 5 Heterogeneity Comparison Based on Statistical Inference

The above theoretical results discuss the properties of the effect scales ignoring issues of statistical inference. In practice, there is another level of heterogeneity due to sampling varibility. With finite samples, we will never have exact no interaction under any effect scale. For , let be the sample analogue of , which has estimated asymptotic variance with being the sample size of group . Taking this uncertainty into account, we can compare the volumes of the regions of , within which we fail to reject the following null hypotheses of homogeneity:

We consider the Wald-type tests of significance level , with and . These three acceptance regions are

Although explicit formulas of these four dimensional volumes are not straightforward to obtain, it is easy to compute their numerical values by using Monte Carlo. Assume that ’s have the same size . Table 1 shows the volumes of the regions and for different sample sizes, from which we can see that the acceptance region of the null hypothesis of homogeneity has the smallest volume for the risk difference, and has the largest volume for the odds ratio. This numerical result also partially explains why the rejection rates of the null hypotheses of homogeneity are higher for the risk difference than for the risk ratio and odds ratio, as found in the surveys of meta-analyses (Katerndahl and Lawler 1999; Engels et al. 2000; Sterne and Egger 2001; Deeks 2002; Deeks and Altman 2003; Poole et al. 2015).

## 6 Discussion

It is often believed that the risk difference is more heterogeneous than the risk ratio and odds ratio. Previously, the evidence for this belief is based on the rejection rates of the tests against the null hypotheses of homogeneity for difference effect scales, which is not adequate either empirically or theoretically. Through theoretical calculations, we provide additional evidence for the claim that the risk difference is more heterogeneous than the risk ratio and odds ratio, by showing that the homogeneity space for the risk difference has the smallest volume of domain, the smallest three dimensional volume, and the smallest volume of the acceptance region based on the Wald-type test. However, to argue that in reality the risk difference is more heterogeneous using the theoretical calculations here we would effectively have to assume that the outcome probabilities are uniformly distributed, an assumption that may not correspond to the empirical distributions of the outcome probabilities encountered in practice. For practical problems, it may be more useful to gather some prior information about the distributions of the outcome probabilities, and compute the volumes of the domain or spaces weighted by these prior distributions. However, to do this, one would also have to specify the domains of the empirical settings under consideration (e.g., disciplines, exposures, outcomes, etc). A uniform distribution of outcome probability seems the only one natural choice a priori. However, empirical evidence, perhaps collected and compared across disciplines, might given further evidence as to which effect measure is the most heterogeneous.

## Acknowledgment

Peng Ding thanks Mr. Yi Xie in the Harvard Mathematics Department for helpful discussion on differential geometry. Tyler J. VanderWeele is funded by the U.S. National Institutes of Health grant ES017876.

## References

- Bogartz (1976) Bogartz, R. S. (1976). On the meaning of statistical interactions. Journal of Experimental Child Psychology, 22:178–183.
- Box and Cox (1964) Box, G. E. and Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological), 26:211–252.
- Cox (1984) Cox, D. R. (1984). Interaction. International Statistical Review, 52:1–24.
- Darroch and Borkent (1994) Darroch, J. N. and Borkent, M. (1994). Synergism, attributable risk and interaction for two binary exposure factors. Biometrika, 81:259–270.
- de Gonz’alez and Cox (2007) de Gonz’alez, A. B. and Cox, D. R. (2007). Interpretation of interaction: A review. The Annals of Applied Statistics, 1:371–385.
- Deeks (2002) Deeks, J. J. (2002). Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Statistics in Medicine, 21:1575–1600.
- Deeks and Altman (2003) Deeks, J. J. and Altman, D. G. (2003). Effect measures for meta–analysis of trials with binary outcomes. In Systematic Reviews in Health Care: Meta-Analysis in Context, Second Edition, pages 313–335. London, England: BMJ Publishing Group.
- Ding et al. (2016) Ding, P., Feller, A., and Miratrix, L. (2016). Randomization inference for treatment effect variation. Journal of the Royal Statistical Society: Series B (Statistical Methodology), in press.
- Engels et al. (2000) Engels, E. A., Schmid, C. H., Terrin, N., Olkin, I., and Lau, J. (2000). Heterogeneity and statistical signficance in meta-analysis: an empirical study of 125 meta-analyses. Statistics in Medicine, 19:1707–1728.
- Katerndahl and Lawler (1999) Katerndahl, D. A. and Lawler, W. R. (1999). Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: a meta-meta-analysis. American Journal of Epidemiology, 149:429–441.
- Keiding and Louis (2016) Keiding, N. and Louis, T. A. (2016). Perils and potentials of self-selected entry to epidemiological studies and surveys (with discussion). Journal of the Royal Statistical Society: Series A (Statistics in Society), in press.
- Krantz and Tversky (1971) Krantz, D. H. and Tversky, A. (1971). Conjoint-measurement analysis of composition rules in psychology. Psychological Review, 78:151–169.
- Loftus (1978) Loftus, G. R. (1978). On interpretation of interactions. Memory & Cognition, 6:312–319.
- Pearl and Bareinboim (2014) Pearl, J. and Bareinboim, E. (2014). External validity: From do-calculus to transportability across populations. Statistical Science, 29:579–595.
- Poole et al. (2015) Poole, C., Shrier, I., and VanderWeele, T. J. (2015). Is the risk difference really a more heterogeneous measure? Epidemiology, 26:714–718.
- Sterne and Egger (2001) Sterne, J. A. and Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. Journal of Clinical Epidemiology, 54:1046–1055.
- VanderWeele (2015) VanderWeele, T. J. (2015). Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford: Oxford University Press.
- Wagenmakers et al. (2012) Wagenmakers, E.-J., Krypotos, A.-M., Criss, A. H., and Iverson, G. (2012). On the interpretation of removable interactions: A survey of the field 33 years after Loftus. Memory & Cognition, 40:145–160.

Appendix

## Appendix A.1 A Lemma

The following lemma is a known result in the literature, but we give an elementary proof.

###### Lemma A.1.

Assume that are linearly independent vectors in . Let be an matrix defined as . The -dimensional volume of the parallelepiped formed by is .

###### Proof of Lemma a.1..

Let be the parallelepiped formed by . We use the following intuitive definition of the volume of a parallelepiped in a -dim subspace of -dim space:

where represents the norm, and is the projection of onto the space orthogonal to the linear space spanned by This definition is closely related to the Gram–Schmidt orthogonalization, or equivalently the QR decomposition of a matrix. We have

where has orthonormal column vectors, and is an upper-triangular matrix. In the construction of the QR decomposition, the diagonal elements of are the lengths of for . Therefore,

∎

## Appendix A.2 Computing and

For ease of notation, we use and first assume . We simply denote and by and . All vectors are column vectors. The set represents a mapping from a three dimensional set to the four dimensional space, written as

First, we need to find the domain of the mapping Because , the domain of is

Second, we need to find the infinitesimal volume spanned by the following three vectors corresponding to the directions of :

(A.1) |

Let

Lemma A.1 implies that the infinitesimal volume spanned by the vectors in (A.1) is

Therefore, the three dimensional volume of is

where the last line follows from integrals of simple polynomials.

If the probabilities are bounded by general , then the domain becomes

and the volume becomes

Applying the transformations and , the above integral becomes

From the above calculation, we know that .

## Appendix A.3 Computing and

Again we use , and first assume . We simply denote and by and . The set represents a mapping from a three dimensional set to the four dimensional space, written as

First, we need to find the domain of the mapping Because , the domain of is

For convenience in later calculation, we express in the following form:

It is straightforward to show that the above two forms of are equivalent.

Second, we need to find the infinitesimal volume spanned by the following three vectors corresponding to the directions of :

(A.2) |

Let

Lemma A.1 implies that the infinitesimal volume spanned by the vectors in (A.2) is

Therefore, the three dimensional volume of is

Because

the volume becomes

Because

the volume of becomes

Because of the following two integrals

the volume is

If the probabilities are bounded by general , then the domain becomes

and the volume becomes

Applying the transformations and , the above integral becomes

The volume of the domain is

For general , similar to the discussion in Section A.2, the volume of is .

## Appendix A.4 Computing and

Again we use . Define and , with derivatives and The set represents a mapping from a three dimensional set to a four dimensional space, written as

First, we need find the domain of the mapping Because , the domain of is

For notational simplicity, define

(A.3) | |||||

(A.4) |

In the following calculation, we write and

Second, we need to find the infinitesimal volume spanned by the following three vectors corresponding to the directions of :

(A.5) |