Audit Games This work was partially supported by the U.S. Army Research Office contract “Perpetually Available and Secure Information Systems” (DAAD19-02-1-0389) to Carnegie Mellon CyLab, the NSF Science and Technology Center TRUST, the NSF CyberTrust grant “Privacy, Compliance and Information Risk in Complex Organizational Processes,” the AFOSR MURI “Collaborative Policies and Assured Information Sharing,”, HHS Grant no. HHS 90TR0003/01 and NSF CCF-1215883. Jeremiah Blocki was also partially supported by a NSF Graduate Fellowship. Arunesh Sinha was also partially supported by the CMU CIT Bertucci Fellowship. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity.

Audit Games

Abstract

Effective enforcement of laws and policies requires expending resources to prevent and detect offenders, as well as appropriate punishment schemes to deter violators. In particular, enforcement of privacy laws and policies in modern organizations that hold large volumes of personal information (e.g., hospitals, banks, and Web services providers) relies heavily on internal audit mechanisms. We study economic considerations in the design of these mechanisms, focusing in particular on effective resource allocation and appropriate punishment schemes. We present an audit game model that is a natural generalization of a standard security game model for resource allocation with an additional punishment parameter. Computing the Stackelberg equilibrium for this game is challenging because it involves solving an optimization problem with non-convex quadratic constraints. We present an additive FPTAS that efficiently computes a solution that is arbitrarily close to the optimal solution.

1 Introduction

In a seminal paper, Gary Becker \shortciteBecker68 presented a compelling economic treatment of crime and punishment. He demonstrated that effective law enforcement involves optimal resource allocation to prevent and detect violations, coupled with appropriate punishments for offenders. He described how to optimize resource allocation by balancing the societal cost of crime and the cost incurred by prevention, detection and punishment schemes. While Becker focused on crime and punishment in society, similar economic considerations guide enforcement of a wide range of policies. In this paper, we study effective enforcement mechanisms for this broader set of policies. Our study differs from Becker’s in two significant ways—our model accounts for strategic interaction between the enforcer (or defender) and the adversary; and we design efficient algorithms for computing the optimal resource allocation for prevention or detection measures as well as punishments. At the same time, our model is significantly less nuanced than Becker’s, thus enabling the algorithmic development and raising interesting questions for further work.

A motivating application for our work is auditing, which typically involves detection and punishment of policy violators. In particular, enforcement of privacy laws and policies in modern organizations that hold large volumes of personal information (e.g., hospitals, banks, and Web services providers like Google and Facebook) relies heavily on internal audit mechanisms. Audits are also common in the financial sector (e.g., to identify fraudulent transactions), in internal revenue services (e.g., to detect tax evasion), and in traditional law enforcement (e.g., to catch speed limit violators).

The audit process is an interaction between two agents: a defender (auditor) and an adversary (auditee). As an example, consider a hospital (defender) auditing its employee (adversary) to detect privacy violations committed by the employee when accessing personal health records of patients. While privacy violations are costly for the hospital as they result in reputation loss and require expensive measures (such as privacy breach notifications), audit inspections also cost money (e.g., the cost of the human auditor involved in the investigation). Moreover, the number and type of privacy violations depend on the actions of the rational auditee—employees commit violations that benefit them.

1.1 Our Model

We model the audit process as a game between a defender (e.g, a hospital) and an adversary (e.g., an employee). The defender audits a given set of targets (e.g., health record accesses) and the adversary chooses a target to attack. The defender’s action space in the audit game includes two components. First, the allocation of its inspection resources to targets; this component also exists in a standard model of security games [\citeauthoryearTambe2011]. Second, we introduce a continuous punishment rate parameter that the defender employs to deter the adversary from committing violations. However, punishments are not free and the defender incurs a cost for choosing a high punishment level. For instance, a negative work environment in a hospital with high fines for violations can lead to a loss of productivity (see [\citeauthoryearBecker1968] for a similar account of the cost of punishment). The adversary’s utility includes the benefit from committing violations and the loss from being punished if caught by the defender. Our model is parametric in the utility functions. Thus, depending on the application, we can instantiate the model to either allocate resources for detecting violations or preventing them. This generality implies that our model can be used to study all the applications previously described in the security games literature [\citeauthoryearTambe2011].

To analyze the audit game, we use the Stackelberg equilibrium solution concept [\citeauthoryearvon Stackelberg1934] in which the defender commits to a strategy, and the adversary plays an optimal response to that strategy. This concept captures situations in which the adversary learns the defender’s audit strategy through surveillance or the defender publishes its audit algorithm. In addition to yielding a better payoff for the defender than any Nash equilibrium, the Stackelberg equilibrium makes the choice for the adversary simple, which leads to a more predictable outcome of the game. Furthermore, this equilibrium concept respects the computer security principle of avoiding “security through obscurity”— audit mechanisms like cryptographic algorithms should provide security despite being publicly known.

1.2 Our Results

Our approach to computing the Stackelberg equilibrium is based on the multiple LPs technique of Conitzer and Sandholm \shortciteConitzerS06. However, due to the effect of the punishment rate on the adversary’s utility, the optimization problem in audit games has quadratic and non-convex constraints. The non-convexity does not allow us to use any convex optimization methods, and in general polynomial time solutions for a broad class of non-convex optimization problems are not known [\citeauthoryearNeumaier2004].

However, we demonstrate that we can efficiently obtain an additive approximation to our problem. Specifically, we present an additive fully polynomial time approximation scheme (FPTAS) to solve the audit game optimization problem. Our algorithm provides a -bit precise output in time polynomial in . Also, if the solution is rational, our algorithm provides an exact solution in polynomial time. In general, the exact solution may be irrational and may not be representable in a finite amount of time.

1.3 Related Work

Our audit game model is closely related to security games [\citeauthoryearTambe2011]. There are many papers (see, e.g., [\citeauthoryearKorzhyk et al.2010, \citeauthoryearPita et al.2011, \citeauthoryearPita et al.2008]) on security games, and as our model adds the additional continuous punishment parameter, all the variations presented in these papers can be studied in the context of audit games (see Section 4). However, the audit game solution is technically more challenging as it involves non-convex constraints.

An extensive body of work on auditing focuses on analyzing logs for detecting and explaining violations using techniques based on logic [\citeauthoryearVaughan et al.2008, \citeauthoryearGarg et al.2011] and machine learning [\citeauthoryearZheng et al.2006, \citeauthoryearBodik et al.2010]. In contrast, very few papers study economic considerations in auditing strategic adversaries. Our work is inspired in part by the model proposed in one such paper [\citeauthoryearBlocki et al.2012], which also takes the point of view of commitment and Stackelberg equilibria to study auditing. However, the emphasis in that work is on developing a detailed model and using it to predict observed audit practices in industry and the effect of public policy interventions on auditing practices. They do not present efficient algorithms for computing the optimal audit strategy. In contrast, we work with a more general and simpler model and present an efficient algorithm for computing an approximately optimal audit strategy. Furthermore, since our model is related to the security game model, it opens up the possibility to leverage existing algorithms for that model and apply the results to the interesting applications explored with security games.

\citeauthor

ZhaoJ08 \shortciteZhaoJ08 model a specific audit strategy—“break the glass”—-where agents are permitted to violate an access control policy at their discretion (e.g., in an emergency situation in a hospital), but these actions are audited. They manually analyze specific utility functions and obtain closed-form solutions for the audit strategy that results in a Stackelberg equilibrium. In contrast, our results apply to any utility function and we present an efficient algorithm for computing the audit strategy.

2 The Audit Game Model

The audit game features two players: the defender (), and the adversary (). The defender wants to audit targets , but has limited resources which allow for auditing only one of the  targets. Thus, a pure action of the defender is to choose which target to audit. A randomized strategy is a vector of probabilities of each target being audited. The adversary attacks one target such that given the defender’s strategy the adversary’s choice of violation is the best response.

Let the utility of the defender be when audited target was found to be attacked, and when unaudited target was found to be attacked. The attacks (violation) on unaudited targets are discovered by an external source (e.g. government, investigative journalists,…). Similarly, define the utility of the attacker as when the attacked target is audited, and when attacked target is not audited, excluding any punishment imposed by the defender. Attacks discovered externally are costly for the defender, thus, . Similarly, attacks not discovered by internal audits are more beneficial to the attacker, and .

The model presented so far is identical to security games with singleton and homogeneous schedules, and a single resource [\citeauthoryearKorzhyk et al.2010]. The additional component in audit games is punishment. The defender chooses a punishment “rate” such that if auditing detects an attack, the attacker is fined an amount . However, punishment is not free—the defender incurs a cost for punishing, e.g., for creating a fearful environment. For ease of exposition, we model this cost as a linear function , where ; however, our results directly extend to any cost function polynomial in . Assuming is also without loss of generality as utilities can be scaled to be comparable to . We do assume the punishment rate is fixed and deterministic; this is only natural as it must correspond to a consistent policy.

We can now define the full utility functions. Given probabilities of each target being audited, the utility of the defender when target is attacked is

The defender pays a fixed cost regardless of the outcome. In the same scenario, the utility of the attacker when target is attacked is

The attacker suffers the punishment only when attacking an audited target.

Equilibrium. The Stackelberg equilibrium solution involves a commitment by the defender to a strategy (with a possibly randomized allocation of the resource), followed by the best response of the adversary. The mathematical problem involves solving multiple optimization problems, one each for the case when attacking is in fact the best response of the adversary. Thus, assuming is the best response of the adversary, the optimization problem in audit games is

The first constraint verifies that attacking is indeed a best response. The auditor then solves the problems (which correspond to the cases where the best response is , respectively), and chooses the best solution among all these solutions to obtain the final strategy to be used for auditing. This is a generalization of the multiple LPs approach of Conitzer and Sandholm \shortciteConitzerS06.

Inputs. The inputs to the above problem are specified in bit precision. Thus, the total length of all inputs is . Also, the model can be made more flexible by including a dummy target for which all associated costs are zero (including punishment); such a target models the possibility that the adversary does not attack any target (no violation). Our result stays the same with such a dummy target, but, an additional edge case needs to be handled—we discuss this case in a remark at the end of Section 3.2.

3 Computing an Audit Strategy

Because the indices of the set of targets can be arbitrarily permuted, without loss of generality we focus on one optimization problem () from the multiple optimization problems presented in Section 2. The problem has quadratic and non-convex constraints. The non-convexity can be readily checked by writing the constraints in matrix form, with a symmetric matrix for the quadratic terms; this quadratic-term matrix is indefinite.

However, for a fixed , the induced problem is a linear programming problem. It is therefore tempting to attempt a binary search over values of . This naïve approach does not work, because the solution may not be single-peaked in the values of , and hence choosing the right starting point for the binary search is a difficult problem. Another naïve approach is to discretize the interval into steps of , solve the resultant LP for the many discrete values of , and then choose the best solution. As an LP can be solved in polynomial time, the running time of this approach is polynomial in , but the approximation factor is at least (due to the in the objective). Since can be as large as , getting an -approximation requires to be , which makes the running time exponential in . Thus, this scheme cannot yield an FPTAS.

3.1 High-Level Overview

Fortunately, the problem has another property that allows for efficient methods. Let us rewrite in a more compact form. Let , and . and are always positive, and reduces to:

The main observation that allows for polynomial time approximation is that, at the optimal solution point, the quadratic constraints can be partitioned into a) those that are tight, and b) those in which the probability variables are zero (Lemma 1). Each quadratic constraint corresponding to can be characterized by the curve . The quadratic constraints are thus parallel hyperbolic curves on the plane; see Figure 1 for an illustration. The optimal values partition the constraints (equivalently, the curves): the constraints lying below the optimal value are tight, and in the constraints above the optimal value the probability variable is zero (Lemma 2). The partitioning allows a linear number of iterations in the search for the solution, with each iteration assuming that the optimal solution lies between adjacent curves and then solving the sub-problem with equality quadratic constraints.

Next, we reduce the problem with equality quadratic constraints to a problem with two variables, exploiting the nature of the constraints themselves, along with the fact that the objective has only two variables. The two-variable problem can be further reduced to a single-variable objective using an equality constraint, and elementary calculus then reduces the problem to finding the roots of a polynomial. Finally, we use known results to find approximate values of irrational roots.

3.2 Algorithm and Main Result

The main result of our paper is the following theorem:

Theorem 1.

Problem can be approximated to an additive in time using the splitting circle method [\citeauthoryearSchönhage1982] for approximating roots.

  • The technique of Lenstra et al. missing\shortciteLenstraLenstraLovasz1982 can be used to exactly compute rational roots. Employing it in conjunction with the splitting circle method yields a time bound . Also, this technique finds an exact optimal solution if the solution is rational.

Before presenting our algorithm we state two results about the optimization problem that motivate the algorithm and are also used in the correctness analysis. The proof of the first lemma is omitted due to lack of space.

Lemma 1.

Let be the optimal solution. Assume and . Then, at , for all , either or , i.e., the quadratic constraint is tight.

Lemma 2.

Assume and . Let . If for some , then . If for some , then . If for some , then and .

Proof.

The quadratic constraint for is . By Lemma 1, either or the constraint is tight. If , then, since and , the constraint cannot be tight. Hence, . If , then, or else with the constraint is not satisfied. Hence the constraint is tight. The last case with is trivial. ∎

Figure 1: The quadratic constraints are partitioned into those below that are tight (dashed curves), and those above where (dotted curves).

From Lemma 2, if lies in the region between the adjacent hyperbolas given by and (and and ), then and and for the quadratic constraint with , and for the quadratic constraint with , and the constraint is tight.

These insights lead to Algorithm 1. After handling the case of and separately, the algorithm sorts the ’s to get in ascending order. Then, it iterates over the sorted ’s until a non-negative is reached, assuming the corresponding ’s to be zero and the other quadratic constraints to be equalities, and using the subroutine to solve the induced sub-problem. For ease of exposition we assume ’s to be distinct, but the extension to repeated ’s is quite natural and does not require any new results. The sub-problem for the iteration is given by the problem :

The best (maximum) solution from all the sub-problems (including and ) is chosen as the final answer.

Lemma 3.

Assuming produces an -additive approximate objective value, Algorithm 1 finds an -additive approximate objective of optimization problem .

The proof is omitted due to space constraints.

, where is defined after Lemma 7 Sort ’s in ascending order to get , with corresponding variables and quadratic constraints Solve the LP problem for the two cases when and respectively. Let the solution be and respectively. for  to  do
       if  then
             for . Set constraints to be equalities.
      else
            
      
return
Algorithm 1

solves a two-variable problem instead of . The problem is defined as follows:

The following result justifies solving instead of .

Lemma 4.

and are equivalent for all .

Proof.

Since the objectives of both problems are identical, we prove that the feasible regions for the variables in the objective are identical. Assume is feasible in . The first two constraints are the same in and . Divide each equality quadratic constraint corresponding to non-zero by . Add all such constraints to get:

Then, since we get

The last two constraints are the same in and .

Next, assume is feasible in . Choose

Since , we have , and since for (’s are distinct) we have . Also,

which by the third constraint of is . This implies . Thus, is feasible in .

Define Define Find polynomials such that for  to  do
       if  then
            
      else
             if  then
                   ;
            else
                   if  then
                         ;
                  else
                        
                  
            
      
for for return
Algorithm 2

The equality constraint in , which forms a curve , allows substituting with a function of the form . Then, the steps in involve taking the derivative of the objective and finding those roots of the derivative that ensure that and satisfy all the constraints. The points with zero derivative are however local maxima only. To find the global maxima, other values of of interest are where the curve intersects the closed boundary of the region defined by the constraints. Only the closed boundaries are of interest, as maxima (rather suprema) attained on open boundaries are limit points that are not contained in the constraint region. However, such points are covered in the other optimization problems, as shown below.

The limit point on the open boundary is given by the roots of . This point is the same as the point considered on the closed boundary in problem given by roots of , since when . Also, the other cases when and are covered by the LP solved at the beginning of Algorithm 1.

The closed boundary in are obtained from the constraint , and . The value of the intersection of and is given by the roots of . The value of the intersection of and is given by roots of . The value of the intersection of and is simply . Additionally, as checked in , all these intersection points must lie with the constraint regions defined in .

The optimal is then the value among all the points of interest stated above that yields the maximum value for . Algorithm 2 describes , which employs a root finding subroutine . Algorithm 2 also takes care of approximate results returned by the . As a result of the approximation in the value of , the computed and can lie outside the constraint region when the actual and are very near the boundary of the region. Thus, we check for containment in the constraint region for points and accept the point if the check passes.

Remark (dummy target):

As discussed in Section 2, we allow for a dummy target with all costs zero. Let this target be . For not representing , there is an extra quadratic constraint given by , but, as and are the constraint is just . When represents , then the quadratic constraint is , and the objective is independent of as . We first claim that at any optimal solution. The proof is provided in Lemma 9 in Appendix. Thus, Lemma 1 and 2 continue to hold for with the additional restriction that .

Thus, when does not represent , Algorithm 1 runs with the the additional check in the if condition inside the loop. Algorithm 2 stays the same, except the additional constraint that . The other lemmas and the final results stay the same. When represents , then needs to be the smallest possible, and the problem can be solved analytically.

3.3 Analysis

Before analyzing the algorithm’s approximation guarantee we need a few results that we state below.

Lemma 5.

The maximum bit precision of any coefficient of the polynomials given as input to is .

Proof.

The maximum bit precision will be obtained in . Consider the worst case when . Then, is of degree and of degree . Therefore, the bit precision of and is upper bounded by , where comes from multiplying -bit numbers and arises from the maximum number of terms summed in forming any coefficient. Thus, using the fact that the upper bound is approximately . We conclude that the bit precision of is upper bounded by . ∎

We can now use Cauchy’s result on bounds on root of polynomials to obtain a lower bound for . Cauchy’s bound states that given a polynomial , any root satisfies

Using Lemma 5 it can be concluded that any root returned by satisfies .

Let . The following lemma (whose proof is omitted due to lack of space) bounds the additive approximation error.

Lemma 6.

Assume is known with an additive accuracy of , and . Then the error in the computed is at most , where and

Moreover, is of order .

We are finally ready to establish the approximation guarantee of our algorithm.

Lemma 7.

Algorithm 1 solves problem with additive approximation term if

Also, as is of order .

Proof.

The computed value of can be at most far from the actual value. The additional factor of arises due to the boundary check in . Then using Lemma 6, the maximum total additive approximation is . For this to be less than , . The other term in the above arises from the condition (this represents ) in Lemma 6. ∎

Observe that the upper bound on is only in terms of and . Thus, we can express as a function of and .

We still need to analyze the running time of the algorithm. First, we briefly discuss the known algorithms that we use and their corresponding running-time guarantees. Linear programming can be done in polynomial time using Karmakar’s algorithm [\citeauthoryearKarmarkar1984] with a time bound of , where is the length of all inputs.

The splitting circle scheme to find roots of a polynomial combines many varied techniques. The core of the algorithm yields linear polynomials ( can be complex) such that the norm of the difference of the actual polynomial and the product is less than , i.e., . The norm considered is the sum of absolute values of the coefficient. The running time of the algorithm is in a pointer based Turing machine. By choosing and choosing the real part of those complex roots that have imaginary value less than , it is possible to obtain approximations to the real roots of the polynomial with bit precision in time . The above method may yield real values that lie near complex roots. However, such values will be eliminated in taking the maximum of the objective over all real roots, if they do not lie near a real root.

LLL [\citeauthoryearLenstra et al.1982] is a method for finding a short basis of a given lattice. This is used to design polynomial time algorithms for factoring polynomials with rational coefficients into irreducible polynomials over rationals. The complexity of this well-known algorithm is , when the polynomial is specified as in the field of integers and is the Euclidean norm of coefficients. For rational coefficients specified in bits, converting to integers yields . LLL can be used before the splitting circle method to find all rational roots and then the irrational ones can be approximated. With these properties, we can state the following lemma.

Lemma 8.

The running time of Algorithm 1 with input approximation parameter and inputs of bit precision is bounded by . Using LLL yields the running time

Proof.

The length of all inputs is , where is the bit precision of each constant. The linear programs can be computed in time . The loop in Algorithm 1 runs less than times and calls . In , the computation happens in calls to and evaluation of the polynomial for each root found. is called three times with a polynomial of degree less than and coefficient bit precision less than . Thus, the total number of roots found is less than and the precision of roots is bits. By Horner’s method [\citeauthoryearHorner1819], polynomial evaluation can be done in the following simple manner: given a polynomial to be evaluated at computing the following values yields the answer as , , , …, . From Lemma 7 we get , thus, is approximately bits, and each computation involves multiplying two numbers with less than bits each. We assume a pointer-based machine, thus multiplication is linear in number of bits. Hence the total time required for polynomial evaluation is . The total time spent in all polynomial evaluation is . The splitting circle method takes time . Using Lemma 7 we get as the running time of . Thus, the total time is

When using LLL, the time in in dominated by LLL. The time for LLL is given by , which is . Thus, the overall the time is bounded by , which using Lemma 7 is . ∎

4 Discussion

We have introduced a novel model of audit games that we believe to be compelling and realistic. Modulo the punishment parameter our setting reduces to the simplest model of security games. However, the security game framework is in general much more expressive. The model [\citeauthoryearKiekintveld et al.2009] includes a defender that controls multiple security resources, where each resource can be assigned to one of several schedules, which are subsets of targets. For example, a security camera pointed in a specific direction monitors all targets in its field of view. As audit games are also applicable in the context of prevention, the notion of schedules is also relevant for audit games. Other extensions of security games are relevant to both prevention and detection scenarios, including an adversary that attacks multiple targets [\citeauthoryearKorzhyk et al.2011], and a defender with a budget [\citeauthoryearBhattacharya et al.2011]. Each such extension raises difficult algorithmic questions.

Ultimately, we view our work as a first step toward a computationally feasible model of audit games. We envision a vigorous interaction between AI researchers and security and privacy researchers, which can quickly lead to deployed applications, especially given the encouraging precedent set by the deployment of security games algorithms [\citeauthoryearTambe2011].

Appendix A Missing proofs

Proof of Lemma 1.

We prove the contrapositive. Assume there exists a such that and the quadratic constraint is not tight. Thus, there exists an such that

We show that it is possible to increase to by a small amount such that all constraints are satisfied, which leads to a higher objective value, proving that is not optimal. Remember that all ’s are , and .

We do two cases: (1) assume . Then, first, note that can be increased by or less and and can be decreased by to still satisfy the constraint, as long as

It is always possible to choose such . Second, note that for those ’s for which we get , and by assumption , thus, . Let be such that , i.e., can be increased by or less and the constraint will still be satisfied. Third, for those ’s for which , can be increased by or less, which must be accompanied with increase in in order to satisfy the quadratic constraint.

Choose feasible ’s (which fixes the choice of also) such that . Then choose an increase in : such that

Increase