ROI constrained Auctions
A standard result from auction theory is that bidding truthfully in a second price auction is a weakly dominant strategy, or, in the language of digital advertising, ’the cost per mille (eCPM) is equal to the click through rate (CTR) times the cost per clicks (CPC)’. However, such assertion is incorrect if the buyers are subject to ROI constraints. More generally, several predictions derived by the traditional auction theory literature fail to apply. This is what I propose to discuss in this article.
I formalize the notion of ROI constrained auctions and derive a Nash equilibrium for second price auctions. I then extend this result to any combination of first and second price payment rules and do an asymptotic analysis of the equilibrium as the number of bidders increases. Further I expose a revenue equivalence property, and finish with a proposal for a dynamic extension of the bidder ROI constrained optimization problem.
Keywords:Auction theory Ad auctions Bidding strategy Equilibrium Control HJB.
What should you bid in a second price ad auction for a display with a known click-through rate (CTR), for a given cost-per-click (CPC)? The most probable (and possibly incorrect) answer is "CPC times CTR". However, the right answer is "What do you mean by cost-per-click?". In this article, we challenge the current modeling approach for ad auctions. We argue that when some advertisers’ business constraints apply, the expected outcome of the auction may depart from the traditional literature.
Advertising is a major source of revenue for Internet publishers, and as such, is financing a large portion of the Internet. About 200 billions USD were spend in 2017 on digital advertising. Banners for display advertising are usually sold through a high frequency one unit auction mechanism called RTB (Real Time Bidding) by or on behalf of advertisers.
When a user reaches a publisher page, it triggers a call to an RTB platform. The RTB platform then calls potential buyers, who have a few millisecond to answer with a bid request. This results in an allocation of the display to a bidder, in exchange of a payment. The allocation and the payment are defined by the rules of the auction, which are not uniformly defined among the RTB platforms. The industry standard is currently shifting from Second Price Auctions to First Price Auctions.
The literature on display advertising auctions has been growing over the last decade, pushed by the emergence of Internet giants whose business models are centered on digital advertising. One track of research takes the viewpoint of the seller and focuses on how to build "good" auctions, and relies mostly on mechanism design theory [2, 8], while the dual track brings the buyer view point under scrutiny, and focuses on the design of bidding strategies [13, 1, 6, 7, 5]. The reader can refer to  and  for an introduction to auction theory.
Usually, in performance markets, the advertiser has a target Return on Investment (ROI) and/or a budget in mind. We will assume that the budget constraint is absent or not binding. This topic of budget constrains has already been addressed in the literature[2, 1, 7]. We will focus the analysis on the ROI constraint. The ROI is measured in term of average action per unit of money spent, the action being a click, a visit, a conversion… In the literature and in the industry we usually use the inverse of the ROI, expressed in Cost Per Click (CPC), Cost per Sale (COS) or more generally, Cost Per Action (CPA). For instance, if (1) I am facing a competition uniformly distributed between and , (2) the CTR of every display is , (3) the auction is second price and (4) I am ready to pay 1 USD on average for a click. Then if I bid , I will win half of the time. For every display won, I get on average half a click, and I pay , thus my expected CPC is . I should increase my bid to raise my empirical CPC.
We pinpoint the following paradox: on the one hand, a lot of research effort is being invested in the design of good, sophisticated auctions and bidding strategies, but on the other hand, one key aspect of the end buyer (the brand) objective program is being neglected more often than not.
This work brings several contributions to the table: we introduce the buyer’s ROI constrained optimization problem, and exhibit a solution, we find the symmetric equilibrium in this setting and compare its properties with standard results from the literature, in particular, no reserve price should be used in the symmetric setting. Since in practice the ROI is computed in average over time, it create a dependency between auction, we thus also discuss extensions to dynamic settings and derive some properties of the solutions.
2 Modeling Assumptions and Notations
We start by introducing the simplest ROI constrained optimization problem for a buyer. There is one item (a display opportunity in our context) to be sold through an auction. In all but Section 6, we assume that there is only one auction to be held. Unless otherwise stated, we also assume the auction to follow a second price rule, with bidders competing for the item.
Each item brings a value to the bidder of interest. This value is independently distributed among bidders, with the same distribution of cumulative and density of support . At bid time, each bidder knows his value for the item to be sold. For example, in the context of ad auction, can be thought of as the CTR or the conversion probability.
We will denote by the bid of a given bidder of interest, and by the maximum of the bids of the other bidders (the price to beat). In the proofs, we will use the letter and for the density and the cumulative distributions of the competitors best bid (or best value, depending on the context). Without loss of generality, we assume .
For a random variable , we denote by the binary random variable that takes the value when , and otherwise.
We assume the bidders do not want to have their ROI below a given threshold. Throughout the paper, we will not refer to the ROI directly, but to its inverse (homogeneous to a cost per click, a cost per sale, a cost per action…), that we will refer to with the letter , because it is our "targeted cost per something". We assume to be the same for all buyers.
The bidder wants to maximize his expected value, subject to an ex ante constraint in expectancy representing the targeted ROI. The constraint is ex ante because in practice, the same bidding strategy is going to be applied repeatedly (or even simultaneously) on several similar auctions. More formally, the bidder is looking for a bid function solution of
subject to (ROI). The average is computed over the distribution of values and prices to beat. The RHS of the constraint corresponds to the expected value earned by the buyer times the target, while the LHS is the expected spend. Observe that if we remove the constraint, the buyer would buy all the opportunities no matter the cost.
The decisional interactions among buyers can be modeled as a game with constraints on the strategies profile. For a given set of opponent bidding strategies, the solution of (1) is the best reply of the bidder. If we find a strategies profile such that all components are best replies against the others, we say that this strategy profile is a (constrained) Nash equilibrium.
3 Bidding Behavior and Symmetric Equilibrium
The main result of this section is the derivation of a symmetric Nash equilibrium for an ROI constrained Second Price Auction. We then generalize this result to linear combinations of First Price and Second Price Auctions.
3.1 Second Price
Theorem 3.1 (Best Reply)
For any distribution of the price to beat , there exists such that a solution of 1 writes:
Lemma 1 (ROI monotony)
If the bidder bid proportionally to the value i.e. there exists such that , then the ROI is non-increasing in .
Fix , , then the expected spend knowing that the value is and the auction is won is , where is the probability density of the price to beat. This quantity is increasing in . Meanwhile the value is . We get the result by integration on .
If are independent draws from , we denote by the average of the greater draw.
Theorem 3.2 (Equilibrium Bid)
The unique symmetric constrained Nash equilibrium strategy is
Theorem 3.1 is the answer to the introductory question. Despite its simplicity, it shows something that is probably overlooked: it may happen in a second price auction that the bidder’s optimal bid depends on the competition. Another useful insight is that the bid is linear in the value, which implies that simple bid multipliers could be used optimally. Observe that the result holds also in non-symmetric settings.
The proof relies on the strong incentive compatibility of the Second Price Auction in standard setting reinterpreted on the Lagrangian of the optimization problem. The parameter should be interpreted as the Lagrangian multiplier associated with the ROI constraint.
Informally speaking, the harder the constraint, the bigger , the smaller the bid. When is set to , one bids only for displays that do not consume the constraint. When goes to zero,the bid diverges.
Lemma 1 is a useful observation to understand what is happening. Observe that since the bid is linear in the value and the objective increasing in the bid multiplier, the optimal bid multiplier is the maximal admissible one.
Theorem 3.2 is one of our main result. Observe that is only a function of the number of opponents and . It is smaller than one (see Appendix). In particular, the bid is proportional to the target with a factor greater than one. Compared with the standard setting, the seller’s expected revenue is multiplied by . The limit of as the number of bidders goes to infinity, as well as other natural questions concerning the equilibrium will be studied in in Section 5. Note that Theorem 3.2 is a necessary and sufficient condition for a symmetric Nash equilibrium, therefore it is unique, but we cannot claim that we have identified the only NE (see Section 5).
We now generalize the reasoning to auctions which are convex combinations of First Price and Second Price. We motivate this extension by the use in the industry of auctions which mix together first and second pricing rules, such as soft-floor.
If we denote by the payment rule of the auction when the bidder bid and the best competing bid is , then we can make the following observations: (1) for any , , (2) , (3) is the sum of a linear function of and a linear function of .
We denote by symmetric equilibrium bidding strategy profile of the standard auction with payment rule and i.i.d. values distributed according to (and no ROI constraints).
The bid , is a symmetric equilibrium strategy,
: We can check this formula on a FPA with 2 buyers and uniform distribution to recover
4.1 Power Law Case: derivation of
We take over with . We get that . Therefore we can write , which leads to
We observe that is increasing in and . It converges to 1 as (or ) goes to infinity. A plot of is displayed in Figure 1(a).
If we consider the 2 bidders case, and denote by and their respective bid multipliers, then the payment and of the first bidder are respectively equal to
We see that the ROI of the first bidder does not depends on the second bidder strategy. Therefore, we have in this case something similar to a strategy dominance.
4.2 Numerical estimation of
The trick is to remark that is equal to the ratio of spend vs value when . We derive the following algorithm to estimate in with different and :
Choose , and a number of samples
If then : and
Repeat from 3. (N-1) times
The results are displayed in Figure 1. For the lognormal and the Poisson distributions, the results are similar to those of the power distribution: monotony and convergence to 1. It seems that the bigger the variance, the bigger . Without providing a proof, we see that we can get an insightful intuition from Equation 8 of the next section.
The exponential distribution provides more exotic results: seems to be only a function of (i.e. independent of the scale parameter). This can be easily proved analytically using the formula and a recursion on . It is however still increasing in , and seems to converge to 1 as goes to infinity.
5 Properties of the Equilibrium
We recover a standard result ( from auction theory, adapted to constrained auctions.
Theorem 5.1 (Revenue equivalence)
All the auctions described in Section 3.2 bring the same expected revenue to the seller.
The expected payment of a buyer is equal to his expected value time . Since the total welfare is fixed, the expected payment of every buyer is the same no matter the auction.
On the other hand, the following observation is quite unusual.
Theorem 5.2 (Optimal Reserve Price)
With ROI constraints, the optimal reserve price in a symmetric setting is zero.
Same argument as before: a reserve price would decrease the social welfare, and thus decrease the expected payment.
This completely departs from the idea that reserve prices should be used to increase the seller’s revenue. Observe that in practice in the case of display advertising, the buyer may take some time to react to a change of ROI. Consequently, measuring the seller’s long term expected revenue uplift is a technical and business challenge. Moreover, the reserve price, by decreasing the welfare, may on the long run trigger a decrease of the target ROI, since buyers have to do an arbitrage between volume and ROI.
Theorem 5.3 (Convergence)
For any number of bidders , . Moreover if the value is bounded, then
Let us denote by the maximum of random variables drawn with the distribution . One just need to observe that
This can be interpreted in the following way: when their is a great number of bidders, the competition is such that the payment tends to become first price.
There may be a non symmetric equilibrium.
The basic idea is that one bidder can bid higher than necessary to force another bidder to leave the auction because his (ROI) constraint cannot be satisfied.
We concentrate on exhibiting a counter-example. Take , , is the uniform distribution over , and denote by the bid multipliers of the two bidders. Set . Then if , bidder 1 do not win any auction, but the (ROI) constraint is satisfied.
On the other hand, if , then the spend vs value ratio is bigger than 4, and the (ROI) constraint is violated. We can check that if the (ROI) constraint is satisfied for . Therefore is an asymmetric Nash equilibrium.
In addition to this, observe that the bidders may be tempted to bid other strategies than the linear best reply. For example consider this example: Take , an exponential distribution with parameter 1, . One agent may increase its profit by bidding with a affine function. Compare with .
On a simulation with auctions, we get in the first case an effective inverse ROI of , and a revenue of for bidder 1 (resp. and for bidder 2), while in the second case, we get an effective inverse ROI of , and a revenue of for bidder 1 (resp. and for bidder 2). Those simple, informal examples indicate that the bidders may be tempted to bid aggressively to weaken the other bidders ROI.
6 Dynamic Bidder Problem
In practice, the buyer behaviors may differ from the static case: (1) the dynamics allows the buyer to take decisions based on the past events, (2) the linear constraint in expectation does not reflect the buyer risk aversion, (3) the benefit the buyer gets from a won auction is stochastic (for example, if he is only interested in clicks or conversions), (4) from a business perspective, the constraint is "smooth", and there is a trade-off between and volumes that can be expressed in different ways.
We propose in this section a continuous time optimal control framework to express and study the buyer’s problem in a dynamic fashion. The approach combines the advantages of a powerful and mathematically clean expressiveness with theoretical insights and numerical tools. We only model one individual buyer. Yet note that the optimal control formulation is a first building block for the study of the full market dynamics (with mean field games or differential games)
6.1 Deterministic Dynamic Formulation
When we neglect the stochastic aspect of the reward, the problem is very similar to the static one. In this section, is the cumulative distribution of the price to beat. We denote by the time horizon. We denote by the instantaneous revenue and by the instantaneous cost:
The bidder is now maximizing with respect to
subject to , and .
We use the Pontryagin Maximum Principle to argue that a necessary condition for an optimal control is that there exists a multiplier so that maximizes . Moreover . We thus recover the main result of the static case: the bid is linear in . Moreover in the absence of stochasticity, the multiplier is constant. Observe that in practice, one could use the solution of problem (10) to implement a Markov Decision Process (MDP), using as the current state and thus get an online bidding engine.
6.2 Stochastic Formulation
The number of displays is much higher than the number of clicks/sales, therefore I neglect the stochasticity of the competition/price over the stochasticity that comes from the actions. Since we see a lot of displays, we apply the law of large number to model the uncertainty with a white noise. Therefore there is an additional, stochastic term in the revenue, , where is a Brownian motion. We get that an optimal, stochastic bid should maximize (see  for a reference)
Since the realization of the action follows a binomial law, . Assuming , we can approximate it as . Therefore every infinitesimal terms of the Hamiltonian becomes reproducing the previous reasoning we get
Once again, the bid factor approach is justified ! Observe that this conclusion comes from an approximation, (therefore the bid is not strictly speaking optimal for the initial problem). Last but not least, we have reduced the control set from to . This reduction gives access to a well understood approach to solve this kind of problem: the continuous time equivalent of Dynamic Programming.
Observe that our problem definition is incomplete: we need to take into account the constraint on the final state. We restate this constraint with a penalty . Using dynamic programing, the buyer problem is equivalent to the following Hamilton-Jacobi-Bellman equation:
6.3 Numerical Resolution
We solve an instance of the problem with the numerical optimal stochastic control solver BocopHJB . BocopHJB compute the Bellman function by solving the non-linear partial differential equation (HJB). Observe that the state is one dimensional, which make this kind of solver very efficient.
We take on and uniformly distributed on . We get that if and otherwise, similarly, for and for (see Figure 2. We take and a linear penalty for negative . The result of simulated trajectories are displayed on Figure 3.
We have formalized the existence of ROI constraints in the buyers bidding problem. These constraints are of paramount importance in performance markets, yet they tend to be neglected in the current literature. We have seen how standard results translate into this context, sometimes with surprising insights. We have given a proposal to study the dynamic, stochastic setting by combining a first order approximation with the Hamilton-Jacobi-Bellman approach. This allowed us in particular to propose a numerical method.
This work raises questions that would probably deserve more extended study. In particular, can we generalized the first three theorems to other constraints, auction rules and settings ? Can the existence of the aggressive behaviors pointed out at the end of Section 5 poses a problem to the market ? We may also want to put under scrutiny the harder, but more realistic case of correlated values. The dynamic case would also probably deserve a specific study.
I would like to thank Jérémie Mary for his helpful comments.
-  Agarwal, D., Ghosh, S., Wei, K., You, S.: Budget pacing for targeted online advertisements at linkedin. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 1613–1619. ACM (2014)
-  Balseiro, S.R., Besbes, O., Weintraub, G.Y.: Repeated auctions with budgets in ad exchanges: Approximations and design. Management Science 61(4), 864–884 (2015)
-  Bonnans, F., Giorgi, D., Heymann, B., Martinon, P., Tissot, O.: BocopHJB 1.0.1 – User Guide. Technical Report RT-0467, INRIA (Sep 2015), https://hal.inria.fr/hal-01192610
-  Bonnans, J.: Stochastic Optimization, Lecture Notes
-  Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Yu, Y., Guo, D.: Real-time bidding by reinforcement learning in display advertising. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. pp. 661–670. ACM (2017)
-  Diemert, E., Meynet, J., Galland, P., Lefortier, D.: Attribution modeling increases efficiency of bidding in display advertising. In: Proceedings of the ADKDD’17. p. 2. ACM (2017)
-  Fernandez-Tapia, J., Guéant, O., Lasry, J.M.: Optimal real-time bidding strategies. Applied Mathematics Research eXpress 2017(1), 142–183 (2016)
-  Golrezaei, N., Lin, M., Mirrokni, V., Nazerzadeh, H.: Boosted second-price auctions for heterogeneous bidders (2017)
-  Krishna, V.: Auction theory. Academic press (2009)
-  Pham, H.: Continuous-time stochastic control and optimization with financial applications, vol. 61. Springer Science & Business Media (2009)
-  Roughgarden, T.: Twenty lectures on algorithmic game theory. Cambridge University Press (2016)
-  Vickrey, W.: Auctions and bidding games. Recent advances in game theory 29, 15–27 (1962)
-  Zhang, Y., Wei, Y., Ren, J.: Multi-touch attribution in online advertising with survival theory. In: 2014 IEEE International Conference on Data Mining (ICDM). pp. 687–696. IEEE (2014)
Appendix A Proof of Theorem 3.1
The Lagrangian of the optimization problem writes
By recognizing the payoff of a Second Price auction with value , we educe that for each and , a maximizing is . The existence of a stationary is a consequence of the minmax Theorem (see Theorem 1.1129 in )
Appendix B Proof of Theorem 3.2
To derive the first order condition, we follow the same path as in the proof of Theorem 3.3. We thus need to argue on the uniqueness.
Let us go back to the Lagrangian expression, and consider a value . We start with a small technical consideration: observe that the bids can be modified on a zero measure set without impacting the profit. To simplify the technical analysis, we assume without loss of generality that the bids are maximizers of the integrand. We denote by a symmetric equilibrium bidding strategy.
Remember that bidding truthfully is a weakly dominant strategy for a second price auction. Similarly, we can say that for a given , is not strictly dominated, no matter the competition. Still, it is possible that some other bid provide as much in the Lagrangian expression. Such thing may only happen if has a discontinuity around . Since is linear where it is not discontinuous, and increasing, the only possibility is that has positive jumps on a set of positive measures. Since is bounded this is not possible (by Baire’s theorem).
Appendix C Proof of Theorem 3.3
First observe that, the Lagrangian of the bidder optimization problems writes:
thus we can look for a point-wise maximizer. Denote by the equilibrium bid of the same auction without any ROI constraints. Observe that if the price to beat is distributed like (for a given ), then:
Therefore, for a given , if the competition uses the proposed linear bid, one should also bid linearly to maximize the Lagrangian. We denote by the price to beat distribution and by the degree of "second priceness": . Observe that the first term of the intergand in the Lagrangian definition is equal to . So, let us deal with the second term. Since
we get that
The first order condition on writes
Then using successively the homogeneity of , a simplification by , and the de facto definition of as , we get
which we simplify using the following formula on the order statistics (see ): .