Efficiency Guarantees from Data
Analysis of efficiency of outcomes in game theoretic settings has been a main item of study at the intersection of economics and computer science. The notion of the price of anarchy takes a worst-case stance to efficiency analysis, considering instance independent guarantees of efficiency. We propose a data-dependent analog of the price of anarchy that refines this worst-case assuming access to samples of strategic behavior. We focus on auction settings, where the latter is non-trivial due to the private information held by participants. Our approach to bounding the efficiency from data is robust to statistical errors and mis-specification. Unlike traditional econometrics, which seek to learn the private information of players from observed behavior and then analyze properties of the outcome, we directly quantify the inefficiency without going through the private information. We apply our approach to datasets from a sponsored search auction system and find empirical results that are a significant improvement over bounds from worst-case analysis.
Efficiency Guarantees from Data
Darrell Hoy University of Maryland firstname.lastname@example.org Denis Nekipelov University of Virginia email@example.com Vasilis Syrgkanis Microsoft Research firstname.lastname@example.org
A major field at the intersection of economics and computer science is the analysis of the efficiency of systems under strategic behavior. The seminal work of K99 (); Roughgarden2002 () triggered a line of work on quantifying the inefficiency of computer systems, ranging from network routing, resource allocation and more recently auction marketplaces RST16 (). However, the notion of the price of anarchy suffers from the pessimism of worst-case analysis. Many systems can be inefficient in the worst-case over parameters of the model, but might perform very well for the parameters that arise in practice.
Due to the large availability of datasets in modern economic systems, we propose a data-dependent analog of the price of anarchy, which assumes access to a sample of strategic behavior from the system. We focus our analysis on auction systems where the latter approach is more interesting due to the private information held by the participants of the system, i.e. their private value for the item at sale. Since efficiency is a function of these private parameters, quantifying the inefficiency of the system from samples of strategic behavior is non-trivial. The problem of estimation of the inefficiency becomes an econometric problem where we want to estimate a function of hidden variables from observed strategic behavior. The latter is feasible under the assumption that the observed behavior is the outcome of an equilibrium of the strategic setting, which connects observed behavior to unobserved private information.
Traditional econometric approaches to auctions GPV (); Paarsch2006 (), address such questions by attempting to exactly pin-point the private parameters from the observed behavior and subsequently measuring the quantities of interest, such as the efficiency of the allocation. The latter approach is problematic in complex auction systems for two main reasons: (i) it leads to statistical inefficiency, (ii) it requires strong conditions on the connection between observed behavior and private information. Even for a single-item first-price auction, uniform estimation of the private value of a player from samples of observed bids, can only be achieved at -rates GPV (). Moreover, uniquely identifying the private information from the observed behavior, requires a one-to-one mapping between the two quantities. The latter requires strong assumptions on the distribution of private parameters and can only be applied to simple auction rules.
Our approach bridges the gap between worst-case price of anarchy analysis and statistically and modeling-wise brittle econometric analysis. We provide a data-dependent analog of recent techniques for quantifying the worst-case inefficiency in auctions ST13 (); HHT14 (); RST16 (), that do not require characterization of the equilibrium structure and which directly quantify the inefficiency through best-response arguments, without the need to pin-point the private information. Our approach makes minimal assumptions on the distribution of private parameters and on the auction rule and achieves -rates of convergence for many auctions used in practice, such as the Generalized Second Price (GSP) auction eos:2007 (); varian:2009 (). We applied our approach to a real world dataset from a sponsored search auction system and we portray the optimism of the data-dependent guarantees as compared to their worst-case counterparts C14 ().
We consider the single-dimensional mechanism design setting with bidders. The mechanism designer wants to allocate a unit of good to the bidders, subject to some feasibility constraint on the vector of allocations . Let be the space of feasible allocations. Each bidder has a private value per-unit of the good, and her utility when she gets allocation and is asked to make a payment is . The value of each bidder is drawn independently from distribution with CDF , supported in and let be the joint distribution.
An auction solicits a bid from each bidder and decides on the allocation vector based on an allocation rule and a payment rule . For a vector of values and bids, the utility of a bidder is:
A strategy , for each bidder , maps the value of the bidder to a bid. Given an auction and distribution of values , a strategy profile is a Bayes-Nash Equilibrium (BNE) if each bidder with any value maximizes her utility in expectation over her opponents bids, by bidding .
The welfare of an auction outcome is the expected utility generated for all the bidders, plus the revenue of the auctioneer, which due to the form of bidder utilities boils down to being the total value that the bidders get from the allocation. Thus the expected utility of a strategy profile is
We denote with the expected optimal welfare: .
Worst-case Bayes-Nash price of anarchy.
The Bayesian price of anarchy of an auction is defined as the worst-case ratio of welfare in the optimal auction to the welfare in a Bayes-Nash equilibrium of the original auction, taken over all value distributions and over all equilibria. Let be the set of Bayes-Nash equilibria of an auction , when values are drawn from distributions . Then:
3 Distributional Price of Anarchy: Refining the PoA with Data
We will assume that we observe samples of bid profiles from running times an auction . Each bid profile is drawn i.i.d. based on an unknown Bayes-Nash equilibrium of the auction, i.e.: let denote the distribution of the random variable , when is drawn from . Then are i.i.d. samples from . Our goal is to refine our prediction on the efficiency of the auction and compute a bound on the price of anarchy of the auction conditional on the observed data set. More formally, we want to derive statements of the form: conditional on , with probability at least : , where is the empirical analogue of the worst-case price of anarchy ratio.
Infinite data limit
We will tackle this question in two steps, as is standard in estimation theory. First we will look at the infinite data limit where we know the actual distribution of equilibrium bids . We define a notion of price of anarchy that is tailored to an equilibrium bid distribution, which we refer to as the distributional price of anarchy. In Section 4 we give a distribution-dependent upper bound on this ratio for any single-dimensional auction. Subsequently, in Section 5, we show how one can estimate this upper bound on the distributional price of anarchy from samples.
Given a value distribution and an equilibrium , let denote the resulting equilibrium bid distribution. We then define the distributional price of anarchy as follows:
Definition 1 (Distributional Price of Anarchy).
The distributional price of anarchy of an auction and a distribution of bid profiles , is the worst-case ratio of welfare in the optimal allocation to the welfare in an equilibrium, taken over all distributions of values and all equilibria that could generate the bid distribution :
This notion has nothing to do with sampled data-sets, but rather is a hypothetical worst-case quantity that we could calculate had we known the true bid generating distribution .
What does the extra information of knowing give us?
To answer this question, we first focus on the optimization problem each bidder faces. At any Bayes-Nash equilibrium each player must be best-responding in expectation over his opponent bids. Observe that if we know the rules of the auction and the equilibrium distribution of bids , then the expected allocation and payment function of a player as a function of his bid are uniquely determined:
Importantly, these functions do not depend on the distribution of values , other than through the distribution of bids . Moreover, the expected revenue of the auction is also uniquely determined:
Thus when bounding the distributional price of anarchy, we can assume that these functions and the expected revenue are known. The latter is unlike the standard price of anarchy analysis, which essentially needs to take a worst-case approach to these quantities.
Through the rest of the paper we will fix the distribution . Hence, for brevity we omit it from notation, using , and Rev instead of , and .
4 Bounding the Distributional Price of Anarchy
We first upper bound the distributional price of anarchy via a quantity that is relatively easy to calculate as a function of the bid distribution and hence will also be rather straightforward to estimate from samples of , which we defer to the next section. To give intuition about the upper bound, we start with a simple but relevant example of bounding the distributional price of anarchy in the case when the auction is the single-item first price auction. We then generalize the approach to any auction .
4.1 Example: Single-Item First Price Auction
In a single item first price auction, the designer wants to auction a single indivisible good. Thus the space of feasible allocations , are ones where only one player gets allocation and other players get allocation . The auctioneer solicits bids from each bidder and allocates the good to the highest bidder (breaking ties lexicographically), charging him his bid. Let be the equilibrium distribution of bids and let be the CDF of the bid of player . For simplicity we assume that is continuous (i.e. the distribution is atomless). Then the expected allocation of a player from submitting a bid is equal to and his expected payment is , leading to expected utility: .
The quantity DPoA is a complex object as it involves the structure of the set of equilibria of the given auction. The set of equilibria of a first price auction when bidders values are drawn from different distributions is an horrific object.111Even for two bidders with uniformly distributed values and , the equilibrium strategy requires solving a complex system of partial differential equations, which took several years of research in economics to solve (see Vickrey1961 (); Krishna2002 ()) However, we can upper bound this quantity by a much simpler data-dependent quantity by simply invoking the fact that under any equilibrium bid distribution no player wants to deviate from his equilibrium bid. Moreover, this data-dependent quantity can be much better than its worst-case counterpart used in the existing literature on the price of anarchy.
Let be the single item first price auction and let be the equilibrium distribution of bids, then , where .
Let be the CDF of the bid of each player under distribution . Moreover, let denote the equilibrium strategy that leads to distribution . By the equilibrium condition, we know that for all and for all ,
We will give a special deviating strategy used in the literature ST13 (), that will show that either the players equilibrium utility is large or the expected maximum other bid is high. Let denote the expected maximum other bid which can be expressed as . We consider the randomized deviation where the player submits a randomized bid in with PDF . Then the expected utility from this deviation is:
Adding the quantity on both sides, we get: . Invoking the equilibrium condition we get: . Subsequently, for any :
If is the expected allocation of player under the efficient allocation rule , then taking expectation of Equation (9) over and adding across all players we get:
The theorem then follows by invoking the fact that for any feasible allocation : , using the fact that expected total agent utility plus total revenue at equilibrium is equal to expected welfare at equilibrium and setting . ∎
Comparison with worst-case PoA
In the worst-case, is upper bounded by , leading to the well-known worst-case price of anarchy ratio of the single-item first price auction of , irrespective of the bid distribution . However, if we know the distribution then we can explicitly estimate , which can lead to a much better ratio (see Figure 2 in Appendix). Moreover, observe that even if we had samples from the bid distribution , then estimating is very easy as it corresponds to the ratio of two expectations, each of which can be estimating to within an error by a simple average and using standard concentration inequalities. Even thought this improvement, when compared to the worst-case bound might not be that drastic in the first price auction, the extension of the analysis in the next section will be applicable even to auctions where the analogue of the quantity is not even bounded in the worst-case. In those settings, the empirical version of the price of anarchy analysis is of crucial importance to get any efficiency bound.
Comparison with value inversion approach
Apart from being just a primer to our main general result in the next section, the latter result about the data-dependent efficiency bound for the first price auction, is itself a contribution to the literature. It is notable to compare the latter result with the standard econometric approach to estimating values in a first price auction pioneered by GPV () (see also Paarsch2006 ()). Traditional non-parametric auction econometrics use the equilibrium best response condition to pin-point the value of a player from his observed bid, by what is known as value inversion. In particular, if the function: has a unique maximum for each and this maximum is strictly monotone in , then given the equilibrium bid of a player and given a data distribution we can reverse engineer the value that the player must have. Thus if we know the bid distribution we can calculate the equilibrium welfare as . Moreover, we can calculate the expected optimal welfare as: . Thus we can pin-point the distributional price of anarchy.
However, the latter approach suffers from two main drawbacks: (i) estimating the value inversion function uniformly over from samples, can only happen at very slow rates that are at least and which require differentiability assumptions from the value and bid distribution as well as strong conditions that the density of the value distribution is bounded away from zero in all the support (with this lower bound constant entering the rates of convergence), (ii) the main assumption of the latter approach is that the optimal bid is an invertible function and that given a bid there is a single value that corresponds to that bid. This assumption might be slightly benign in a single item first price auction, but becomes a harsher assumption when one goes to more complex auction schemes. Our result in Lemma 1 suffers neither of these drawbacks: it admits fast estimation rates from samples, makes no assumption on properties of the value and bid distribution and does not require invertibility of the best-response correspondence. Hence it provides an upper bound on the distributional price of anarchy that is statistically robust to both sampling and mis-specification errors.
4.2 Generalizing to any Single-Dimensional Auction Setting
Our analysis on DPoA is based on the reformulation of the auction rules as an equivalent pay-your-bid auction and then bounding the price of anarchy as a function of the ratio of how much a player needs to pay in an equivalent pay-your-bid auction, so as to acquire his optimal allocation vs. how much revenue is the auctioneer collecting. For any auction, we can re-write the expected utility of a bid :
This can be viewed as the same form of utility if the auction was a pay-your-bid auction and the player submitted a bid of . We refer to this term as the price-per-unit and denote it . Our analysis will be based on the price-per-unit allocation rule , which determines the expected allocation of a player as a function of his price-per-unit. Given this notation, we can re-write the utility that an agent achieves if he submits a bid that corresponds to a price-per-unit of as: . The latter is exactly the form of a pay-your-bid auction.
Our upper bound on the DPoA, will be based on the inverse of the PPU allocation rule; let be the price-per-unit of the cheapest bid that achieves allocation at least . More formally, . For simplicity, we assume that any allocation is achieveable by some high enough bid .222The theory can be easily extended to allow for different maximum achievable allocations by each player, by simply integrating the average threshold only up until the largest such allocation. Given this we can define the threshold for an allocation:
Definition 2 (Average Threshold).
The average threshold for agent is
In the Appendix we provide a pictorial representation of these quantities. Connecting with the previous section, for a first price auction, the price-per-unit function is , the price-per-unit allocation function is and the threshold function is . The average threshold is equal to , i.e. the expected maximum other bid.
We now give our main Theorem, which is a distribution-dependent bound on DPoA, that is easy to compute give and which can be easily estimated from samples of . This theorem is a generalization of Lemma 1 in the previous section.
Theorem 2 (Distributional Price of Anarchy Bound).
For any auction in a single dimensional setting and for any bid distribution , the distributional price of anarchy is bounded by , where .
Theorem 2 provides our main method for bounding the distributional price of anarchy. All we need is to compute the revenue Rev of the auction and the quantity:
under the given bid distribution . Both of these are uniquely defined quantities if we are given . Moreover, once we compute , the optimization problem in Equation (13) is simply a welfare maximization problem, where each player’s value per-unit of the good is . Thus, the latter can be solved in polynomial time, whenever the welfare maximization problem over the feasible set is polynomial-time solvable.
Theorem 2 can be viewed as a bid distribution-dependent analogue of the revenue covering framework HHT14 () and of the smooth mechanism framework ST13 (). In particular, the quantity is the data-depenent analogue of the worst-case quantity used in the definition of -revenue covering in HHT14 () and is roughly related to the quantity used in the definition of a -smooth mechanism in ST13 ().
5 Distributional Price of Anarchy Bound from Samples
In the last section, we assumed we were given distribution and hence we could compute the quantity , which gave an upper bound on the DPoA. We now show how we can estimate this quantity when given access to i.i.d. samples from the bid distribution . We will separately estimate and Rev. The latter is simple expectation and thereby can be easily estimated by an average at rates. For the former we first need to estimate for each player , which requires estimation of the allocation and payment functions and .
Since both of these functions are expected values over the equilibrium bids of opponents, we will approximate them by their empirical analogues:
To bound the estimation error of the quantities produced by using the latter empirical estimates of the allocation and payment function, we need to provide a uniform convergence property for the error of these functions over the bid .
Since takes values in a continuous interval, we cannot simply apply a union bound. We need to make assumptions on the structure of the class of functions and , so as uniformly bound their estimation error. For this we resort to the technology of Rademacher complexity. For a generic class of functions and a sequence of random variables , the Rademacher complexity is defined as:
where each is an i.i.d. Rademacher random variable, which takes each of those values with equal probabilities. The following well known theorem will be useful in our derivations:
Theorem 3 (Shalev2014 ()).
Suppose that for any sample of size , and suppose that functions in take values in . Then with probability :
This Theorem reduces our uniform error problem to bounding the Rademacher complexity of classes and , since we immediately have the following corollary (where we also use that the allocation functions lie in and the payment functions lie in ):
Suppose that for any sample of size , the Rademacher complexity of classes and is at most . Then with probability , both and are at most .
We now provide conditions under which the Rademacher complexity of these classes is .
Suppose that and for each bidder and each , the functions and can be computed as finite superposition of (i) multiplication of bid vectors with constants; (ii) comparison indicators ; (iii) pairwise addition . The Rademacher complexity for both classes on a sample of size is .
The proof of this Lemma follows by standard arguments of Rademacher calculus, together with VC arguments on the class of pairwise comparisons. Those arguments can be found in Lemma 9.9 in kosorok:07 (). Thereby, we omit its proof. The assumptions of Lemma 5 can be directly verified, for instance, for the sponsored search auctions where the constants that multiply each bid correspond to quality factors of the bidders, e.g. as in eos:2007 () and varian:2009 () and then the allocation and the payment is a function of the rank of the weighted bid of a player. In that case the price and the allocation rule are determined solely by the ranks and the values of the score-weighted bids , as well as the position specific quality factors , for each position in the auction.
Next we turn to the analysis of the estimation errors on quantities . We consider the following plug-in estimator for : We consider the empirical analog of function by . Then the empirical analog of is obtained by:
To bound the estimation error of , we need to impose an additional condition that ensures that any non-zero allocation requires the payment from the bidder at least proportional to that allocation.
We assume that is Lipschitz-continuous and that the mechanism is worst-case interim individually rational, i.e. .
Under this assumption we can establish that rates of convergence of to and of the empirical analog of the optimized threshold to as well as the empirical analog of the revenue to Rev. Thus the quantity , will also converge to at that rate. This implies the following final conclusion of this section.
6 Sponsored Search Auction: Model, Methodology and Data Analysis
We consider a position auction setting where ordered positions are assigned to bidders. An outcome in a position auction is an allocation of positions to bidders. denotes the bidder who is allocated position ; refers to the position assigned to bidder . When bidder is assigned to slot , the probability of click is the product of the click-through-rate of the slot and the quality score of the bidder, , so (in the data the quality scores for each bidder are varying across different auctions and we used the average score as a proxy for the score of a bidder). Each advertiser has a value-per-click (VPC) , which is not observed in the data and which we assume is drawn from some distribution . Our benchmark for welfare will be the welfare of the auction that chooses a feasible allocation to maximize the welfare generated, thus .
We consider data generated by advertisers repeatedly participating in a sponsored search auction. The mechanism that is being repeated at each stage is an instance of a generalized second price auction triggered by a search query. The rules of each auction are as follows: Each advertiser is associated with a click probability and a scoring coefficient and is asked to submit a bid-per-click . Advertisers are ranked by their rank-score and allocated positions in decreasing order of rank-score as long as they pass a rank-score reserve . All the mentioned sets of parameters and the bids are observable in the data.
We will denote with the bidder allocated in slot under a bid profile and parameter profile . We denote with the slot allocated to bidder . If advertiser is allocated position , then he pays only when he is clicked and his payment, i.e. his cost-per-click is the minimal bid he had to place to keep his position, which is: . Mapping this setting to our general model, the allocation function of the auction is , the payment function is and the utility function is: .
We applied our analysis to the BingAds sponsored search auction system. We analyzed eleven phrases from multiple thematic categories. For each phrase we retrieved data of auctions for the phrase for the period of a week. For each phrase and bidder that participated in the auctions for the phrase we computed the allocation curve by simulating the auctions for the week under any alternative bid an advertiser could submit (bids are multiples of cents).
See Figure 1 for the price-per-unit allocation curves for a subset of the advertisers for a specific search phrase. We estimated the average threshold for each bidder by numerically integrating these allocation curves along the axis. We then applied the approach described in Section 3 for each of the search phrases, computing the quantity . The latter optimization is simply the optimal assignment problem where each player’s value-per-click is and can be performed by greedily assigning players to slots in decreasing order of . We then estimate the expected revenue by the empirical revenue .
We portray our results on the estimate and the implied bound on the distributional price of anarchy for each of the eleven search phrases in Table 1. Phrases are grouped based on thematic category. Even though the worst-case price of anarchy of this auction is unbounded (since scores are not equal to qualities , which is required in worst-case PoA proofs C14 ()), we observe that empirically the price of anarchy is very good and on average the guarantee is approximately of the optimal. Even if the worst-case bound on the PoA implies guarantees of approx. C14 (), while the DPoA we estimated implies significantly higher percentages, portraying the value of the empirical approach we propose.
-  Ioannis Caragiannis, Christos Kaklamanis, Maria Kyropoulou, Brendan Lucier, Renato Paes Leme, and Éva Tardos. Bounding the inefficiency of outcomes in generalized second price auctions. pages 1–45, 2014.
-  Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. The American economic review, 97(1):242–259, 2007.
-  Emmanuel Guerre, Isabelle Perrigne, and Quang Vuong. Optimal nonparametric estimation of first-price auctions. Econometrica, 68(3):525–574, 2000.
-  Jason Hartline, Darrell Hoy, and Sam Taggart. Price of Anarchy for Auction Revenue. In ACM Conference on Economics and Computation, pages 693–710, New York, New York, USA, 2014. ACM Press.
-  Michael R Kosorok. Introduction to empirical processes and semiparametric inference. Springer Science & Business Media, 2007.
-  Elias Koutsoupias and Christos Papadimitriou. Worst-case equilibria. In STACS 99, pages 404–413. Springer, 1999.
-  Vijay Krishna. Auction Theory. Academic Press, March 2002.
-  H. J. Paarsch and H. Hong. An Introduction to the Structural Econometrics of Auction Data. MIT Press, 2006.
-  Tim Roughgarden, Vasilis Syrgkanis, and Éva Tardos. The price of anarchy in auctions. CoRR, abs/1607.07684, 2016.
-  Tim Roughgarden and Eva Tardos. How bad is selfish routing? J. ACM, 49(2):236–259, March 2002.
-  S. Shalev-Shwartz and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
-  Vasilis Syrgkanis and Eva Tardos. Composable and efficient mechanisms. In ACM Symposium on Theory of Computing, pages 211–220, 2013.
-  Hal R Varian. Online ad auctions. The American Economic Review, pages 430–434, 2009.
-  William Vickrey. Counterspeculation, auctions, and competitive sealed tenders. The Journal of Finance, 16(1):8–37, 1961.
Supplementary material for
“ Efficiency Guarantees from Data ”
Appendix A Supplementary Figures
a.1 Distributional Price of Anarchy as a Function of
a.2 Pictorial Representation of Threshold Functions
Appendix B Omitted Proofs from Section 3
Theorem 2 (restatement) For any auction in a single dimensional setting and for any bid distribution , the distributional price of anarchy is bounded by , where .
Our proof is a based on a data-dependent analog of the value and revenue covering framework of . First we show that even without having distributional knowledge, the threshold functions are related to the equilibrium utility of a bidder and any target utility at any Bayes-Nash equilibrium. Specifically, either the utility of a bidder at a Bayes-Nash is high compared to his value or the average threshold is high.
Lemma 8 (Value Covering).
For any bidder with value , for any allocation amount and for any ,
The proof proceeds analogously to the proof of value covering in . For simplicity of notation we drop the subscript , as we are focusing on a single agent and some threshold function . Observe that since a player is at equilibrium it must be that for any target expected allocation he does not want to deviate to a bid that corresponds to a price-per-unit , which would yield him expected allocation at least :
Moreover, in any case , by definition. Thus if we define , then we have and hence .
Evaluating the integral gives . Thus
and by dividing over by :
The right side of Equation (21) is convex in , so we can minimize it by taking first-order conditions of the quantity with respect to variable , giving
Leading to a minimum value of that quantity of . Thus the right side of Equation (21) is at least this quantity, giving our desired result,
The Lemma follows by the fact that and , which allows us to multiply and divide the fraction by and then remove the in front of the quantity . ∎
Given the value covering lemma we now proceed to proving the Theorem. Let be the welfare optimal allocation rule for valuation profile , i.e. the one that solves the optimization problem . Applying the value covering inequality of Equation (19) with respect to the optimal allocation quantity gives that for each bidder with value ,
Appendix C Omitted Proofs from Section 5
We begin by showing convergence of to and to .
Lemma 9 (Bounding Estimated Average Thresholds).
Suppose that the premises of Lemma 5 hold and that the function is -Lipschitz continuous. Then for each player with probability :
Since we focus on a single player , we drop index and denote for and similarly for their estimated quantities. Recall that and . Moreover, we denote with and , the uniform errors on the payment and allocation curve, which be the assumptions of the theorem are upper bounded, with probability , by .
Our goal is to bound the quantity:
By individual rationality we have that . Thus we get that and therefore . Hence:
It remains to bound quantity . We consider any . By the definition of and , we obtain
We now upper bound separately the two terms and .
Bounding . For term we have:
Since , we have that for any , with , it must also be that: , which implies that (by individual rationality). Which leads to the bound:
Bounding . For quantity , we proceed as follows. Let and (note that in the second set, we still use to define the possible allocations, and only the set of bids is defined based on the estimated allocation function). Then: