A Appendix to §3: Positively Affiliated Distributions

\addauthor

bkblue \addauthoragred

The correlations and network structure amongst individuals in datasets today—whether explicitly articulated, or deduced from biological or behavioral connections—pose new issues around privacy guarantees, because of inferences that can be made about one individual from another’s data. This motivates quantifying privacy in networked contexts in terms of ‘inferential privacy’—which measures the change in beliefs about an individual’s data from the result of a computation—as originally proposed by Dalenius in the 1970’s. Inferential privacy is implied by differential privacy when data are independent, but can be much worse when data are correlated; indeed, simple examples, as well as a general impossibility theorem of Dwork and Naor, preclude the possibility of achieving non-trivial inferential privacy when the adversary can have arbitrary auxiliary information. In this paper, we ask how differential privacy guarantees translate to guarantees on inferential privacy in networked contexts: specifically, under what limitations on the adversary’s information about correlations, modeled as a prior distribution over datasets, can we deduce an inferential guarantee from a differential one?

We prove two main results. The first result pertains to distributions that satisfy a natural positive-affiliation condition, and gives an upper bound on the inferential privacy guarantee for any differentially private mechanism. This upper bound is matched by a simple mechanism that adds Laplace noise to the sum of the data. The second result pertains to distributions that have weak correlations, defined in terms of a suitable “influence matrix”. The result provides an upper bound for inferential privacy in terms of the differential privacy parameter and the spectral norm of this matrix.

1 Introduction

Privacy has always been a central issue in the discourse surrounding the collection and use of personal data. As the nature of data collected online grows richer, however, fundamentally new privacy issues emerge. In a thought-provoking piece entitled “Networked Rights and Networked Harms” [22], the sociologists Karen Levy and danah boyd argue that the ‘networks’ surrounding data today—whether articulated (as in explicitly declared friendships on social networks), behavioral (as in connections inferred from observed behavior), or biological (as in genetic databases)—raise conceptually new questions that current privacy law and policy cannot address. Levy and boyd present case studies to demonstrate how the current individual-centric legal frameworks for privacy do not provide a means to account for the networked contexts now surrounding personal data.

An analogous question arises on the formal front. One of computer science’s fundamental contributions to the public debate about private data—most prominently via the literature on differential privacy1 [10]—has been to provide a means to measure privacy loss, which enables evaluating the privacy implications of proposed data analyses and disclosures in quantitative terms. However, differential privacy focuses on the privacy loss to an individual by her contribution to a dataset, and therefore—by design—does not capture all of the privacy losses from inferences that could be made about one person’s data due to its correlations with other data in networked contexts. For instance, the privacy implications of a database such as 23andme for one individual depend not just on that person’s own data and the computation performed, but also on her siblings’ data.

In this paper, we look to understand the implications of such ‘networked’ data for formal privacy guarantees. How much can be learnt about a single individual from the result of a computation on correlated data, and how does this relate to the differential privacy guarantee of the computation?

Inferential privacy. A natural way of assessing whether a mechanism protects the privacy of an individual is to ask, “Is it possible that someone, after observing the mechanism’s output, will learn a lot about the individual’s private data?” In other words, what is the inferential privacy—the largest possible ratio between the posterior and prior beliefs about an individual’s data after observing the result of a computation on the database? (This quantity is identical to the differential privacy parameter of the mechanism when individuals’ data are independent; see §2 and [18].)

The inferential privacy guarantee will depend, of course, on both the nature of the correlations in the database and on the precise mechanism used to perform the computation. Instead of seeking to design algorithms that achieve a particular inferential privacy guarantee—which would necessitate choosing a particular computational objective and correlation structure—we instead seek to analyze the inferential privacy guarantees provided by differentially private algorithms. Specifically, we ask the following question: consider the class of all mechanisms providing a certain differential privacy guarantee, say . What is the worst possible inferential privacy guarantee for a mechanism in this class?

This question is pertinent to a policy-maker who can prescribe that analysts provide some degree of differential privacy to individuals while releasing their results, but cannot control how—i.e., using what specific algorithm—the analyst will provide this guarantee. In other words, rather than an algorithm designer who wants to design an inferential privacy-preserving algorithm (for a particular scenario), this question adopts the perspective of a policy-maker who can set privacy standards that analysts must obey, but is agnostic to the analysts’ computational objectives. We choose the differential privacy guarantee as our measure of privacy for many reasons: it is, at present, the only widely-agreed-upon privacy guarantee known to provide strong protections even against arbitrary side information; there is a vast toolbox of differentially private algorithms and a well-understood set of composition rules for combining them to yield new ones; finally, differential privacy is now beginning to make its way into policy and legal frameworks as a potential means for quantifying privacy loss.

Measuring privacy loss via inferential privacy formalizes Dalenius’s [4] desideratum that “access to a statistical database should not enable one to learn anything about an individual that could not be learned without access”. While it is well known2 that non-trivial inferential privacy guarantees are incompatible with non-trivial utility guarantees in the presence of arbitrary auxiliary information, our primary contribution is modeling and quantifying what degree of inferential privacy is in fact achievable under a particular form of auxiliary information, such as that resulting from a known correlation structure or a limited set of such structures. For example, as noted earlier, if the individuals’ rows in the database are conditionally independent given the adversary’s auxiliary information, then the inferential privacy guarantee for any individual collapses to her differential privacy guarantee. At the other extreme, when all individuals’ data are perfectly correlated, the inferential privacy parameter can exceed the differential privacy parameter by a factor of (the number of individuals in the database) as we will see below. What happens for correlations that lie somewhere in between these two extremes? Do product distributions belong to a broader class of distributions with benign correlations which ensure that an individual’s inferential privacy is not much worse than her differential privacy? \bkeditA key contribution of our paper (Theorem 4.2) answers this question affirmatively while linking it to a well-known sufficient condition for ‘correlation decay’ in mathematical physics.

Correlations in networked datasets and their privacy consequences. We start with a caricature example to begin exploring how one might address these questions in a formal framework. Consider a database which contains an individual Athena and her (hypothetical) identical twin Adina, who is so identical to Athena that the rows in the database corresponding to Athena and Adina are identical in (the databases corresponding to) every possible state of the world. A differential privacy guarantee of to all database participants translates to an inferential privacy guarantee of only to Athena (and her twin), since the “neighboring” database where Athena and Adina are different simply cannot exist.3

The erosion of Athena’s privacy becomes even more extreme if the database contains individuals and they are all clones of Athena; a generalization of the preceding calculation now shows that the inferential privacy parameter is . However, in reality one is unlikely to participate in a database with many identical clones of oneself. Instead, it is interesting to consider cases with non-extreme correlations. For example, suppose now that the database contains data from Zeus and all of his descendants, and that every child’s bit matches the parent’s bit with probability . The degree of privacy afforded to Zeus now depends on many aspects of the model: the strength of the correlation (), the number of individuals in the database (), and structural properties of the tree of family relationships—its branching factor and depth, for instance. Which of these parameters contribute most crucially to inferential privacy? Is Zeus more likely to be implicated by his strong correlation with a few close relatives, or by a diffuse “dragnet” of correlations with his distant offspring?

In general, of course, networked databases, and the corresponding inferential privacy guarantees, do not come with as neat or convenient a correlation structure as in this example. In full generality, we can represent the idea of networked similarity via a joint distribution on databases that gives the prior probability of each particular combination of bits. So, for example, a world where all individuals in the database are “twins” would correspond to a joint distribution which has non-zero probability only on the all-zeros and all-ones databases, whereas a world where everyone’s data is independent has multiplicative probabilities for each database.

Such a model of correlations allows capturing a rich variety of networked contexts: in addition to situations where a single database contains sensitive information about individuals whose data have known correlations, it also captures the situation—perhaps closest to reality—where there are multiple databases to which multiple individuals contribute different (but correlated) pieces of information. In this latter interpretation, an inferential privacy guarantee limits the amount that an adversary may learn about one individual’s contribution to one database, despite the correlations both across individuals and between a single individual’s contributions to different databases.4

Our results. Consider a policy-maker who \bkeditspecifies that an analyst must provide a certain differential privacy guarantee, \bkeditand wants to comprehend the inferential privacy consequences of this policy for the population whose (correlated) data is being utilized. Our two main results can be interpreted as providing guidance to such a policy maker. The first result (\creft-pa) supplies a closed-form expression for the inferential privacy guarantee as a function of the differential privacy parameter when data are positively affiliated5[24]. The second result (\creft-bm) allows understanding the behavior of the inferential privacy guarantee as a function of the degree of correlation in the population; it identifies a property of the joint distribution of data that ensures that the policy-maker can meet a given inferential privacy target via a differential privacy requirement that is a constant-factor scaling of that target.

Among all mechanisms with a given differential privacy guarantee, which ones yield the worst inferential privacy when data are correlated? Our first main result, \creft-pa, answers this question when data are positively affiliated, in addition to giving a closed-form expression for the inferential privacy guarantee. The answer takes the following form: we identify a simple property of mechanisms (\crefdef:max-biased) such that any mechanism satisfying the property achieves the worst-case guarantee. Strikingly, the form of the worst-case mechanism does not depend on the joint distribution of the data, but only on the fact that the distribution satisfies positive affiliation. We also provide one example of such a mechanism: a “noisy-sum mechanism” that simply adds Laplace noise to the sum of the data. This illustrates that the worst inferential privacy violations occur even with one of the most standard mechanisms for implementing differential privacy, rather than some contrived mechanisms.

The aforementioned results provide a sharp bound on the inferential privacy guarantee for positively affiliated distributions, but they say little about whether this bound is large or small in comparison to the differential privacy guarantee. Our second main result fills this gap: it provides an upper bound on the inferential privacy guarantee when a bounded affiliation condition is satisfied on the correlations between individuals’ rows in a database. Representing the strengths of these correlations by an influence matrix , \creft-bm asserts that if all row sums of this matrix are bounded by then every individual’s inferential privacy is bounded by , regardless of whether or not the data are positively affiliated. Thus, \creft-bm shows that in order to satisfy -inferential privacy against all distributions with -bounded affiliation, it suffices for the policy-maker to set . We complement this result with an example showing that the ratio of inferential privacy to differential privacy can indeed be as large as , as the row sums of the influence matrix approach 1. Thus, the equivalence between inferential and differential privacy, , which holds for independent distributions, degrades gracefully to as one introduces correlation into the distribution, but only up to a point: as the row sums of the influence matrix approach 1, the ratio can diverge to infinity, becoming unbounded when the row sums exceed 1.

Techniques. Our work exposes a formal connection between the analysis of inferential privacy in networked contexts and the analysis of spin systems in mathematical physics. In brief, application of a differentially private mechanism to correlated data is analogous to application of an external field to a spin system. Via this analogy, physical phenomena such as phase transitions can be seen to have consequences for data privacy: they imply that small variations in the amount of correlation between individuals’ data, or in the differential privacy parameter of a mechanism, can sometimes have gigantic consequences for inferential privacy (§A.2 elaborates on this point). Statistical physics also supplies the blueprint for \creft-bm and its proof: our bounded affiliation condition can be regarded as a multiplicative analogue of Dobrushin’s Uniqueness Condition [5, 6], and our proof of \creft-bm adapts the proof technique of the Dobrushin Comparison Theorem [6, 11, 21] from the case of additive approximation to multiplicative approximation. \bkeditSince Dobrushin’s Uniqueness Condition is known to be one of the most general conditions ensuring exponential decay of correlations in physics, our Theorem 4.2 can informally be interpreted as saying that differential privacy implies strong inferential privacy guarantees when the structure of networked correlations is such that, conditional on the adversary’s side information, the correlations between individuals’ data decay rapidly as their distance in the network increases.

Related work.
\bkedit

Our paper adopts the term inferential privacy as a convenient shorthand for a notion that occurs in many prior works, dating back to Dalenius [4], which is elsewhere sometimes called “before/after privacy” [10], “semantic privacy” [18], or “noiseless privacy” [3]. Dwork and McSherry observed that differentially private mechanisms supply inferential privacy against adversaries whose prior is a product distribution; this was stated implicitly in [7] and formalized in [18]. However, when adversaries can have arbitrary auxiliary information, inferential privacy becomes unattainable except by mechanisms that provide little or no utility; see [9, 19] for precise impossibility results along these lines. Responses to this predicament have varied: some works propose stricter notions of privacy based on simulation-based semantics, e.g. zero-knowledge privacy [13], others propose weaker notions based on restricting the set of prior distributions that the adversary may have, e.g. noiseless privacy [3], and others incorporate aspects of both responses, e.g. coupled-world privacy [2] and the Pufferfish framework [20]. Our work is similar to some of the aforementioned ones in that we incorporate restrictions on the adversary’s prior distribution, however our goal is quite different: rather than proposing a new privacy definition or a new class of mechanisms, we quantify how effectively an existing class of mechanisms (-differentially private mechanisms) achieves an existing privacy goal (inferential privacy).

Relations between differential privacy and network analysis have been studied by many authors—e.g. [17] and the references therein—but this addresses a very different way in which networks relate to privacy: the network in those works is part of the data, whereas in ours it is a description of the auxiliary information.

The exponential mechanism of McSherry and Talwar [23] can be interpreted in terms of Gibbs measures, and Huang and Kannan [16] leveraged this interpretation and applied a non-trivial fact about free-energy minimization to deduce consequences about incentive compatibility of exponential mechanisms. Aside from their work, we are not aware of other applications of statistical mechanics in differential privacy.

2 Defining Inferential Privacy

In this section we specify our notation and basic assumptions and definitions. A population of individuals is indexed by the set . Individual ’s private data is represented by the element , where is a finite set. Except in §4 we will assume throughout, for simplicity, that , i.e. each individual’s private data is a single bit. When focusing on the networked privacy guarantee for a particular individual, we denote her index by and sometimes refer to her as “Athena”.

A database is an -tuple representing the private data of each individual. As explained in \crefsec:intro, our model encodes the ‘network’ structure of the data using a probability distribution on ; we denote this distribution by . A computation performed on the database , whose outcome will be disclosed to one or more parties, is called a mechanism and denoted by . The set of possible outcomes of the computation is , and a generic outcome will be denoted by . %

Differential privacy [7, 8, 10]. For a database and an individual , we use to denote the -tuple formed by omitting from , i.e. . We define an equivalence relation by specifying that . For a mechanism and individual , the differential privacy parameter is defined by

For any vector we say that is -differentially private if the differential privacy parameter of with respect to is at most , for every individual .

Inferential privacy. We define inferential privacy as an upper bound on the (multiplicative) change in when performing a Bayesian update from the prior distribution to the posterior distribution after observing . (If has uncountably many potential outcomes, we must instead consider doing a Bayesian update after observing a positive-probability event for some set of outcomes .)

Definition 2.1.

We say that mechanism satisfies -inferential privacy (with respect to individual ) if the inequality holds for all and all such that . The inferential privacy parameter of is the smallest with this property.

Inferential versus differential privacy. A short calculation using Bayes’ Law illuminates the relation between these two privacy notions.

Thus, the inferential privacy parameter of mechanism with respect to individual is determined by:

(1)

Equivalently, if denote the conditional distributions of given that and , respectively, then is -inferentially private if

(2)
For comparison, differential privacy asserts
(3)

When individuals’ rows in the database are independent, and (3) implies (2) with by averaging over . In other words, when bits are independent, -differential privacy implies -inferential privacy. When bits are correlated, however, this implication breaks down because the databases in (2) are sampled from different distributions. The ‘twins example’ from §1 illustrates concretely why this makes a difference: if and are point-masses on and , respectively, then the inferential privacy parameter of is determined by the equation . For an -differentially-private mechanism this ratio may be as large as since the Hamming distance between and is .

3 Positively Affiliated Distributions

Suppose a designer wants to ensure that Athena receives an inferential privacy guarantee of , given a joint distribution on the data of individuals in the database. What is the largest differential privacy parameter that ensures this guarantee? The question is very challenging even in the special case of binary data (i.e., when ) because the ratio defining inferential privacy (Equation 2) involves summing exponentially many terms in the numerator and denominator. Determining the worst-case value of this ratio over all differentially private mechanisms can be shown to be equivalent to solving a linear program with exponentially many variables (the probability of the event for every potential database ) and exponentially many constraints (a differential privacy constraint for every pair of adjacent databases).

Our main result in this section answers this question when individuals’ data are binary-valued and positively affiliated [12, 24], a widely used notion of positive correlation: \creft-pa gives a closed-form formula (Equation 6) that one can invert to solve for the maximum differential privacy parameter that guarantees inferential privacy when data are positively affiliated. The theorem also characterizes the ‘extremal’ mechanisms achieving the worst-case inferential privacy guarantee in (6) as those satisfying a ‘maximally biased’ property (\crefdef:max-biased). Intuitively, if one wanted to signal as strongly as possible that Athena’s bit is 1 (resp., 0), a natural strategy—given that Athena’s bit correlates positively with everyone else’s—is to have a distinguished outcome (or set of outcomes) whose probability of being output by the mechanism increases with the number of 1’s (resp., the number of 0’s) in the database ‘as rapidly as possible’, subject to differential privacy constraints. \creft-pa establishes that this intuition is valid under the positive affiliation assumption. (Interestingly, the intuition is not valid if one merely assumes that Athena’s own bit is positively correlated with every other individual’s bit; see \crefrmk:pos-corr.) \creflem:noisy-sum provides one simple example of a maximally-biased mechanism, namely a “noisy-sum mechanism” that simply adds Laplace noise to the sum of the bits in the database. Thus, the worst-case guarantee in \creft-pa is achieved not by contrived worst-case mechanisms, but by one of the most standard mechanisms in the differential privacy literature.

We begin by defining positive affiliation, a concept that has proven extremely valuable in auction theory (the analysis of interdependent value auctions), statistical mechanics, and probabilistic combinatorics. Affiliation is a strong form of positive correlation between random variables: informally, positive affiliation means that if some individuals’ bits are equal to 1 (or more generally, if their data is ‘large’), other individuals’ bits are more likely to equal 1 as well (and similarly for 0). We formally define positive affiliation for our setting below and then state a key lemma concerning positively affiliated distributions, the FKG inequality.

Definition 3.1 (Positive affiliation).

Given any two strings , let and denote their pointwise maximum and minimum, respectively. A joint distribution on satisfies positive affiliation if

for all possible pairs of strings . Equivalently, satisfies positive affiliation if is a supermodular function of .

Lemma 3.2 (FKG inequality; Fortuin et al. [12]).

If are three real-valued functions on such that and are monotone and is supermodular, then

(4)

In order to state the main result of this section, \creft-pa, we must define a property that characterizes the mechanisms whose inferential privacy parameter meets the worst-case bound stated in the theorem. We defer the task of describing a mechanism that satisfies the definition (or even proving that such a mechanism exists) until \creflem:noisy-sum below.

Definition 3.3.

For , a mechanism mapping to outcome set is called maximally -biased, with respect to a vector of differential privacy parameters , if there exists a set of outcomes such that for all . In this case, we call a distinguished outcome set for .

Theorem 3.4.

Suppose the joint distribution satisfies positive affiliation. Then for any and any vector of differential privacy parameters, , the maximum of the ratio

(5)

over all -differentially private mechanisms and outcome sets , is attained when is maximally -biased, with distinguished outcome set . Therefore, the inferential privacy guarantee to individual in the presence of correlation structure and differential privacy parameters is given by the formula

(6)
Proof.

Suppose and consider any -differentially private mechanism and outcome set . Letting , we have the identity

(7)

When is maximially 0-biased, with distinguished outcome set , the right side of (7) is equal to . Thus, the case of the theorem is equivalent to the assertion that

(8)

After cross-multiplying and simplifying, this becomes

(9)

If we add to both sides, we find that (9) is equivalent to

(10)

To prove (10) we will apply the FKG inequality. Set and note that is the sum of —a supermodular function—and , a linear function. Hence is supermodular. Now define and . The differential privacy constraint for implies that is monotonically non-decreasing; observe that is monotonically non-decreasing as well. The FKG inequality implies

(11)

Substituting the definitions of into (11) we readily see that it is equivalent to (10), which completes the proof. ∎

Finally, as promised at the start of this section, we show that a noisy-sum mechanism that adds Laplace noise to the sum of the bits in the database is maximally -biased for every . Together with \creft-pa, this shows that any inferential privacy guarantee that can be proven for the noisy-sum mechanism automatically extends to a guarantee for all differentially private mechanisms, when data are positively affiliated.

Lemma 3.5.

Suppose that all individuals have the same differential privacy parameter, i.e. that for some . Consider the noisy-sum mechanism that samples a random from the Laplace distribution with scale parameter and outputs the sum . For all the mechanism is maximally -biased.

Proof.

For any , let . When and , the definition of a maximally -biased mechanism requires the existence of an outcome set such that . For the set , the event coincides with the event . Since is a Laplace random variable with scale parameter , this event has probability proportional to , as desired. When the proof of the lemma proceeds identically, using the set . ∎

Remark 3.6.

Intuitively, one might expect \creft-pa to hold whenever the joint distribution is such that each pair of bits is positively correlated, a weaker property than positive affiliation which requires each pair of bits to be positively correlated even after conditioning on any possible tuple of values for the remaining bits. In Appendix A.1 we present an example illustrating that the theorem’s conclusion can be violated (in fact, quite drastically violated) when one only assumes pairwise positive correlation. The basic reason is that when bits are pairwise positively correlated, it may still be the case that one individual’s bit correlates much more strongly with a non-monotone function of the others’ bits than with any monotone function.

Remark 3.7.

The quantities appearing in \creft-pa have precise analogues in the physics of spin systems, and this analogy sheds light on inferential privacy. Appendix A.2 delves into this connection in detail; in this remark we merely sketch a dictionary for translating between inferential privacy and statistical mechanics and discuss some consequences of this translation.

In brief, an adversary’s prior distribution on corresponds to the Gibbs measure of a two-spin system with Hamiltonian . Under this correspondence, positively affiliated distributions correspond to ferromagnetic spin systems. The adversary’s posterior distribution after applying a maximally 0-biased (resp., maximally 1-biased) mechanism is equivalent to the Gibbs measure of the spin system after applying the external field (resp., ). The worst-case inferential privacy guarantee for Athena in \creft-pa is therefore equivalent (up to a bijective transformation) to the magnetization at Athena’s site when the external field is applied to the spin system.

One of the interesting implications of this correspondence concerns phase transitions. Statistical-mechanical systems such as magnets are known to undergo sharp transitions in their physical properties as one varies thermodynamic quantities such as temperature and external field strength. Translating these results from physics to the world of privacy using the dictionary outlined above, one discovers that inferential privacy guarantees can undergo surprisingly sharp variations as one varies a mechanism’s differential privacy parameter or an adversary’s belief about the strength of correlations between individuals’ bits in a database. \creft-ising in the appendix formalizes these observations about phase transitions in inferential privacy.

4 Bounded Affiliation Distributions

In this section we present a general upper bound for inferential privacy that applies under a condition that we call bounded affiliation. Roughly speaking, bounded affiliation requires that correlations between individuals are sufficiently weak, in the sense that the combined influence of all other individuals on any particular one is sufficiently small. A very similar criterion in the statistical mechanics literature, Dobrushin’s uniqueness condition [5, 6], is identical to ours except that it defines “influence” in terms of additive approximation and we define it multiplicatively (Definition 4.1). Dobrushin showed that this condition implies uniqueness of the Gibbs measure for a specified collection of conditional distributions. Its implications for correlation decay [14, 11, 21] and mixing times of Markov chains [1, 25, 15] were subsequently explored. Indeed, our proof of network differential privacy under the assumption of bounded affiliation draws heavily upon the methods of Dobrushin [6], Gross [14], and Künsch [21] on decay of correlations under Dobrushin’s uniqueness condition.

Throughout this section (and its corresponding appendix) we assume that each individual’s private data belongs to a finite set rather than restricting to . This assumption does not add any complication to the theorem statements and proofs, while giving our results much greater generality. We now define the notion of influence that is relevant to our results on distributions with bounded affiliation.

Definition 4.1.

If are jointly distributed random variables, the multiplicative influence of on , denoted by , is defined by the equation

In other words, the influence of on is one-half of the (individual) differential privacy parameter of with respect to , when one regards as a randomized function of the database . When one adopts the convention that . The multiplicative influence matrix is the matrix .

Theorem 4.2.

Suppose that the joint distribution has a multiplicative influence matrix whose spectral norm is strictly less than 1. Let denote the matrix inverse of . Then for any mechanism with individual privacy parameters , the inferential privacy guarantee satisfies

(12)

If the matrix of multiplicative influences satisfies for some , then for all .

Proof sketch..

Let be any set of potential outcomes of the mechanism such that . Let denote the conditional distribution on databases , given that , and let denote the unconditional distribution , respectively. For and for any function , let denote the expected value of under distribution . Also define the Lipschitz constants The heart of the proof lies in showing that if takes values in then

(13)

This is done by studying the set of all vectors that satisfy for all , and showing that this set is non-empty and is preserved by an affine transformation that is a contracting mapping of (when the spectral norm of is less than 1) with fixed point . To derive (12) from (13), use the definition of to choose two distinct values in such that where are the indicator functions of and , respectively. Unfortunately so direct application of (13) is not useful; instead, we define a suitable averaging operator to smooth out and , thereby improving their Lipschitz constants and enabling application of (13). A separate argument is then used to bound the error introduced by smoothing and using , which completes the proof of (12). Under the hypothesis that , the relation is easily derived from (12) by applying the formula . The full proof is presented in Appendix B. ∎

The bound in the theorem is tight up to a constant factor. This is shown in §A.2 by considering an adversary whose prior is the Ising model of a complete -ary tree at inverse temperature The entries of the influence matrix satisfy if , 0 otherwise. Thus, the row sum is maximized when is an internal node, with degree , in which case the row sum is as . In §A.2 we apply \creft-pa to show that the inferential privacy guarantee for the Ising model on a tree satisfies , matching the upper bound in Theorem 4.2 up to a constant factor.

5 Conclusion

A number of immediate questions are prompted by our results, such as incorporating -privacy into our analysis of inferential guarantees (for product distributions this was achieved in [18]) and extending the analysis in §3 to non-binary databases where an individual’s data cannot be summarized by a single bit. A key challenge here is to find an analogue of positive affiliation for databases whose rows cannot naturally be interpreted as elements of a lattice. More excitingly, however, the scenario of datasets with networked correlations raises several broad directions for future work.

\bkedit

Designing for inferential privacy: Our work takes differentially private algorithms as a primitive and analyzes what inferential privacy is achievable with given differential privacy guarantees. This allows leveraging the vast body of work on, and adoption of, differentially private algorithms, while remaining agnostic to the data analyst’s objective or utility function. However if one instead assumes a particular measure of utility, one can directly investigate the design of inferential-privacy preserving algorithms to obtain stronger guarantees: given some joint distribution(s) and utility objectives, what is the best inferential privacy achievable, and what algorithms achieve it?

Inferential privacy and network structure: An intriguing set of questions arises from returning to the original network structures that led to the model of correlated joint distributions. Note that our results in Theorem 3.4 give the inferential privacy guarantee for a particular individual: how do inferential privacy guarantees depend on the position of an individual in the network (for instance, imagine the central individual in a large star graph versus the leaf nodes), and how does the relation between the correlations and the network structure play in?

Acknowledgements The authors gratefully acknowledge helpful discussions with Christian Borgs, danah boyd, Jennifer Chayes, Cynthia Dwork, Kobbi Nissim, Adam Smith, Omer Tamuz, and Salil Vadhan.

The authors acknowledge the support of NSF awards AF-1512964 and III-1513692, and ONR award N00014-15-1-2335. Robert Kleinberg gratefully acknowledges the support of Microsoft Research New England, where he was employed while most of this research took place.

Appendix A Appendix to §3: Positively Affiliated Distributions

This appendix contains material accompanying §3 that was omitted from that section for space reasons.

a.1 Pairwise positive correlation

A weaker condition than positive affiliation is pairwise positive correlation. This property of a joint distribution on databases requires that for each pair of indices , the (unconditional) marginal distribution of the bits satisfies

If the inequality is strict for every then we say is pairwise strictly positively correlated.

Recall Theorem 3.4, which establishes that when a joint distribution satisfies positive affiliation then the worst-case inferential privacy guarantee is attained by any maximally -biased distribution. The intuition supporting the theorem statement might seem to suggest that the same conclusion holds whenever satisfies pairwise positive correlation. In this section we show that this is not the case: if satisfies pairwise positive correlation (or even strict pairwise positive correlation) there may be a mechanism whose inferential privacy guarantee is much worse than that of any maximally -biased mechanism.

Our construction applies when is of the form for two positive integers . For a database we will denote one of its entries by and the others by for . The joint distribution is uniform over the solution set of the system of congruences

(14)

Thus, to sample from one draws the bits and for independently from the uniform distribution on , then one sets for all so as to satisfy (14).

The distribution is pairwise independent, hence it is pairwise positively correlated. (The calculation of privacy parameters is much easier in the pairwise-independent case. At the end of this section we apply a simple continuity argument to modify the example to one with pairwise strict positive correlation without significantly changing the privacy parameters.)

Let us first calculate the inferential privacy for a mechanism that calculates the number of odd integers in the sequence

(15)

and adds Laplace noise (with scale parameter ) to the result. This is -differentially private since changing a single bit of changes the parity of only one element of the sequence. However, when is sampled from the number of odd integers in the sequence (15) is either 0 if or if . Hence

implying that the inferential privacy parameter of is at least .

Now let us calculate the inferential privacy parameter of a maximally 0-biased mechanism , with outcome such that , where denotes the sum of the bits in . Let (resp. ) denote the set of bit-strings in having even (resp. odd) sum, and let denote the Cartesian powers of these sets. The conditional distribution of given is the uniform distribution on , and the conditional distribution of given is the uniform distribution on . For and , let denote the -tuple . We have

(16)

Similarly,

(17)

(The extra factor of on the right side comes from the fact that , which inflates the exponent in the expression by .) To evaluate the expressions on the right sides of (16)-(17), it is useful to let and . Then we find that

Substituting these expressions into (16)-(17) we may conclude that

(18)

The inferential privacy parameter of is therefore given by

Comparing the inferential privacy parameters of and , they are and , respectively, so the inferential privacy parameter of exceeds that of the maximally 0-biased mechanism, , by an unbounded factor as .

Under the distribution we have analyzed thus far, the bits of are pairwise independent. However, we may take a convex combination of with any distribution in which all pairs of bits are strictly positively correlated—for example, a distribution that assigns equal probability to the two databases and and zero probability to all others. In this way we obtain a distribution which satisfies pairwise strict positive correlation and may can be made arbitrarily close to by varying the mixture parameter of the convex combination. Since the inferential privacy parameter of a mechanism with respect to a given prior distribution is a continuous function of that distribution, it follows that the inferential privacy parameters of and can remain arbitrarily close to the values calculated above while imposing a requirement that the prior on satisfies pairwise strict positive correlation.

a.2 Connection to Ferromagnetic Spin Systems

The quantities appearing in \creft-pa have precise analogues in the physics of spin systems, and this analogy sheds light on inferential privacy. In statistical mechanics, a two-spin system composed of sites has a state space and an energy function or Hamiltonian, . The Gibbs measure of the spin system is a probability distribution assigning to each state a probability proportional to where is a parameter called the inverse temperature. Application of an external field to the spin system is modeled by subtracting a linear function from the Hamiltonian, so that it becomes . The probability of state under the Gibbs measure then becomes

where is the partition function

Databases are in one-to-one correspondence with states under the mapping and its inverse mapping . Any joint distribution has a corresponding Hamiltonian whose Gibbs distribution (at ) equals . The positive affiliation condition is equivalent to requiring that is submodular, a property which is expressed by saying that the spin system is ferromagnetic.

For a maximally -biased mechanism with distinguished outcome set , the probabilities satisfy so

Application of the mechanism is thus analogous to application of the external field at inverse temperature 1. (The additive constant in the Hamiltonian is irrelevant, since the Gibbs measure is unchanged by an additive shift in the Hamiltonian.) Similarly, applying a maximally 1-biased mechanism is analogous to applying the external field at inverse temperature 1.

Let denote the prior probability ratio for Athena’s bit. For the networked privacy guarantee in Theorem 3.4, when the maximum on the right side of (6) is achieved by a maximally 0-biased mechanism, we have

where the operator denotes the expectation under the Gibbs measure corresponding to external field . A similar calculation in the case that a maximally 1-biased mechanism maximizes the right side of (6) yields the relation . Combining these two cases, we arrive at:

(19)

We will refer to as the magnetization at site , by analogy with the usual definition of magnetization in statistical mechanics as the average . Equation (19) thus shows that the inferential privacy guarantee for a positively affiliated distribution is completely determined by the magnetization at site when an external field of strength is applied.

Ising models and phase transitions

Let us now apply this circle of ideas to analyze the “Zeus’s family tree” example from 1. Represent Zeus and his progeny as the nodes of a rooted tree, and suppose that the joint distribution of the individuals’ bits is defined by the following sampling rule: sample the bits in top-down order (from root to leaves), setting the root’s bit to 0 or 1 with equal probability and each other node’s bit equal to the parent’s value with probability and the opposite value otherwise. This leads to a probability distribution in which the probability of any is proportional to where denotes the number of tree edges whose endpoints receive the same label, and is the number of edges whose endpoints receive opposite labels. Letting so that , and associating to via as before, we find that up to an additive constant, where denotes the edge set of the tree. Hence, the joint distribution of Zeus’s family tree is equivalent to the Gibbs measure of the Hamiltonian . Models whose Hamiltonian takes this form (for any graph, not just trees) are known as Ising models (with interaction strength ) and are among the most widely studied in mathematical physics.

Ising models are known to undergo phase transitions as one varies the inverse temperature or external field. For example, in an infinite two-dimensional lattice or -regular tree, there is a phenomenon known as spontaneous magnetization where the magnetization does not converge to zero as the external field converges to zero from above, but this phenomenon only occurs if the inverse temperature is above a critical value, , that is equal to for the two-dimensional lattice and to for the -regular tree. This phenomenon of phase transitions has consequences for inferential privacy, as articulated in \creft-ising below. To state the theorem it is useful to make the following definition.

Definition A.1.

Let be a family of joint distributions on , with each distribution being supported on for a specific value . For a differential privacy parameter , let denote the supremum, over all joint distributions , of the inferential privacy guarantee corresponding to differential privacy parameter . We say that is differentially enforceable with respect to if there exists such that .

In other words, to say that is differentially enforceable means that a regulator can ensure -inferential privacy for the individuals participating in a datasest by mandating that an analyst must satisfy -differential privacy when releasing the results of an analysis performed on the dataset.

Theorem A.2.

For a family of graphs and a given , let be the family of Ising models with interaction strength and zero external field on graphs in . Then

  1. (Sensitivity to strength of correlations.) If is the set of trees of maximum degree and for some , then every is differentially enforceable, and in fact