Sponsored Search Auctions with Markovian Users
Abstract
Sponsored search involves running an auction among advertisers who bid in order to have their ad shown next to search results for specific keywords. Currently, the most popular auction for sponsored search is the “Generalized Second Price” (GSP) auction in which advertisers are assigned to slots in the decreasing order of their score, which is defined as the product of their bid and clickthrough rate. In the past few years, there has been significant research on the gametheoretic issues that arise in an advertiser’s interaction with the mechanism as well as possible redesigns of the mechanism, but this ranking order has remained standard.
From a search engine’s perspective, the fundamental question is: what is the best assignment of advertisers to slots? Here “best” could mean “maximizing user satisfaction,” “most efficient,” “revenuemaximizing,” “simplest to interact with,” or a combination of these. To answer this question we need to understand the behavior of a search engine user when she sees the displayed ads, since that defines the commodity the advertisers are bidding on, and its value. Most prior work has assumed that the probability of a user clicking on an ad is independent of the other ads shown on the page.
We propose a simple Markovian user model that does not make this assumption. We then present an algorithm to determine the most efficient assignment under this model, which turns out to be different than that of GSP. A truthful auction then follows from an application of the VickreyClarkeGroves (VCG) mechanism. Further, we show that our assignment has many of the desirable properties of GSP that makes bidding intuitive. At the technical core of our result are a number of insights about the structure of the optimal assignment.
1 Introduction
Targeted advertisements on search queries is an increasingly important advertising medium, attracting large numbers of advertisers and users. When a user poses a query, the search engine returns search results together with advertisements that are placed into positions, usually arranged linearly down the page, top to bottom. On most major search engines, the assignment of ads to positions is determined by an auction among all advertisers who placed a bid on a keyword that matches the query. The user might click on one or more of the ads, in which case (in the payperclick model) the advertiser receiving the click pays the search engine a price determined by the auction.
In the past few years, the sponsored search model has been highly successful commercially, and the research community is attempting to understand the underlying dynamics, explain the behavior of the market and to improve the auction algorithms. The most common auction being run today is the Generalized Second Price (GSP) auction: Each bidder submits a bid stating the maximum amount they are willing to pay for a click, and the bidders are placed in descending order of , where is what is called the clickthroughrate of advertiser ; i.e., the probability that a user will click on the ad, given that the user looks at it. Much of previous research on sponsored search auctions has fixed this sort order, and focused on understanding the implications of different pricing schemes, assuming strategic behavior on the part of the advertisers. We now know something about GSP’s equilibrium properties [9, 22, 4], alternative pricing that will make it truthful [4], and to some extent, impact on the revenue in principle [9] and via simulations [21].
However, by fixing this sort order, prior work exogenizes an important third party in sponsored search; i.e., the search engine user. Unfortunately, there is very little guidance on this in the literature, even though the user’s behavior is the essential ingredient that defines the commodity the advertisers are bidding on, and its value. We suggest a different framework for principled understanding of sponsored search auctions:

Define a suitable probabilistic model for search engine user behavior upon being presented the ads.

Once this model is fixed, ask the traditional mechanism design questions of how do assign the ads to slots, and how to price them.

Analyze the given mechanism from the perspective of the bidders (e.g., strategies) and the search engine (e.g., user satisfaction, efficiency and revenue).
There are certain wellaccepted observations about the user’s interaction with the sponsored search ads that should inform the model: (i) The higher the ad is on the page, the more clicks it gets. (ii) The “better” the ad is, the more clicks it gets, where the “goodness” of an ad is related to the inherent quality of the ad, and how well it matches the user’s query. These properties govern not only how the auction is run but also how advertisers think about their bidding strategy (they prefer to appear higher and get more clicks). Thus it is important for an auction to have what we call intuitive bidding: a higher bid translates to a higher position and more clicks.
In this paper, we propose a natural Markov model for user clicks, taking the above observations into account and design an algorithm to determine an optimal assignment of ads to positions in terms of economic efficiency. Together with VCG pricing, this gives a truthful auction. We further show that the optimal assignment under this model has certain monotonicity properties that allow for intuitive bidding. In what follows, we will describe our technical contributions in more detail.
Modeling the Search Engine User
Previous work on sponsored search has (implicitly) modeled the user using two types of parameters: adspecific clickthrough rates and positionspecific visibility factors . There are some intuitive user behavior models that express overall clickthrough probabilities in terms of these parameters. One possibility is “for each position independently, the user looks at the ad in that position with probability then clicks on the ad with probability .” Alternatively: “The user picks a single position according to the distribution implied by the ’s, and then clicks on the ad in that position with probability .” Under both these models, it follows that the probability of an ad in position receiving a click is equal to , which is the socalled separability assumption [4]. From separability it follows that GSP ordering of ads will be suitable, because GSP ordering maximizes the total advertiser value on the page.
In both these models there is no reason a priori that the position factors should be decreasing; this is simply imposed because it makes sense, and it is verifiable empirically. Also, both suggested models assume that the probability of an ad getting clicked is independent of other ads that appear with it on the page, an assumption made without much justification. It is hard to imagine that seeing an ad, perhaps followed by a click, has no effect on the subsequent behavior of the user.
In designing a user model, we would like to have the monotonicity of the positions arise naturally. Also, each ad should have parameters dictating their effect on the user both in terms of clicking on that ad, as well as looking at other ads. We propose a model based on a user who starts to scan the list of ads from the top, and makes decisions (about whether to click, continue scanning, or give up altogether) based on what he sees. More specifically, we model the user as the following Markov process: “Begin scanning the ads from the top down. When position is reached, click on the ad with probability . Continue scanning with probability .” In this model, if we try to write the click probability of an ad in position as , we get that , where is the set of ads placed above^{1}^{1}1Throughout the paper, we will often refer to a position or an ad being “higher” or “above” another position or ad; this means that it is earlier on the list, and is looked at first by the user. position . Thus the “position factor” in the click probability decreases with position, and does so naturally from the model. Also note that we do not have separability anymore, since depends on which ads are above position . Consequently, it can be shown that GSP assignment of ads is no longer the most efficient.
Auction with Markovian users
Given this new user model, we can now ask what the best assignment is of ads to slots. We will study the most efficient assignment; i.e., the one that maximizes total advertiser value derived from user clicks. It turns out that the structure of this assignment is different than that of GSP, and indeed is more sophisticated than any simple ranking. The presence of the ’s requires a delicate tradeoff between the click probability of an ad and its effect on the slots below it. In this paper, we identify certain structural properties of the optimal assignment and use them to find such an optimal assignment efficiently, not only in polynomial time, but in nearlinear time. Given this algorithm, a natural candidate for pricing is VCG [23, 8, 12], which is clearly truthful in this setting.
Intuitive Bidding
One of the reasons why GSP is successful is perhaps because bidding strategy is intuitive: Under GSP ranking, if an advertiser bids more, they get to a higher position, and consequently, if they bid more, their click probability increases. Now that we have defined a more sophisticated assignment function, even though VCG pricing is truthful, the auction still may not have these intuitive properties. Our main technical result is to show that in our model, if a mechanism uses the most efficient assignment, indeed position and click probabilities are monotonic in an ad’s bid (with all other bids fixed), thus preserving this important property. While not surprising, positionmonotonicity turns out to be rather involved to prove, requiring some delicate combinatorial arguments, and insights into the optimal substructure of bidder assignments.
In summary, we approach sponsored search auctions as a three party process by modeling the behavior of users first and then designing suitable mechanisms to affect the game theory between the advertiser and the search engine. Our work sheds some light on the intricate connection between the user models and the mechanisms; for example, the sort order of GSP that is currently popular (sort by ) is not optimal under the Markov user model we propose here. More powerful models will be of great interest, such as making the continuation probability a function of position as well, endogenizing the actions of the user as they navigate on the landing page, etc. We leave it open to design truthful auctions under such extended models.
1.1 Related Work
Sponsored search has been an active area of research in the last several years after the early papers explored the foundational models [9, 4, 22, 14]. In general, the motivation for the work that followed is that sponsored search in practice is much more complex than as described by the first models. Some papers have taken on the effect of advertiser budgets [6, 18, 2], as well as analyzing bidder strategy and dynamics [5, 20, 7, 10, 25, 24, 15]. There have also been several papers offering extensions to GSP, or entirely new models and mechanisms [3, 13, 16, 11, 19, 17, 1].
1.2 Outline
In Section 2 we define our model formally. In Section 3, we establish several properties of optimal assignments in this model, including our main technical result that position and click probability will be monotone in bid and match our intuition. We give our algorithm for finding an optimal assignment in Section 4 which gives the truthful auction via VCG pricing, and conclude in Section 5.
2 Markov User Click Model
We consider a sponsored search auction with bidders and positions. We will also refer to “ad ,” meaning the advertisement submitted by bidder . Each bidder has two parameters, and . The clickthroughrate is the probability that a user will click on ad , given that they look at it. The continuation probability is the probability that a user will look at the next ad in a list, given that they look at ad .
Each bidder submits a bid to the auction, representing the amount that they value a click. The quantity then represents the value of an “impression,” i.e., how much they value a user looking at their ad. This is commonly referred to as their “ecpm.”^{2}^{2}2The acronym ecpm stands for “expected cost per thousand” impressions, where M is the roman numeral for one thousand. We will drop the factor of one thousand and refer to as the “ecpm.” Throughout, we will use the notation for convenience.
Given an assignment of bidders to the positions, the user looks at the first ad , clicks on it with probability , and then continues looking with probability .^{3}^{3}3The click event and the continuation event could in principle have some correlation, and all our results will still hold. However since we only consider expected value, we never use this correlation explicitly in our analysis. This is repeated with the second bidder, etc., until the last ad is reached, or some continuation test has failed. Thus the overall expected value of the assignment to the bidders is
The goal of the auctioneer is to compute an assignment of ads to positions that maximizes the overall expected value. Given this assignment, prices can be computed using VCG [23, 8, 12]: for each assigned bidder we compute the change in others’ value if that bidder were to disappear. This assures truthful reporting of bids under a profitmaximizing utility function.
3 Properties of Optimal Assignments
We will start analyzing some basic properties of the optimal assignment. Our insights will allow us to give our main results regarding monotonicity of position and click probability, as well as an efficient algorithm for finding this assignment.
3.1 Adjusted ECPM
It turns out that the quantity , which we will refer to as the “adjusted ecpm (aecpm),” plays a central role in this model. Intuitively, this quantity is the impression value adjusted by the negative effect this bid has on the ads below it. We use for convenience. The following theorem tells us how to assign a set of selected ads to the positions:
Theorem 1
In the most efficient assignment, the ads that are placed are sorted in decreasing order of adjusted ecpm .
Proof.
Suppose not. Then in the ranking there are two consecutive ads and in positions and where
(1) 
The contribution of positions to the efficiency of the ranking (given that position is reached) is
where is the efficiency of positions given that position is reached. If and are switched, then the contribution would change to
and nothing else would change. So since the former is the most efficient assignment, we have
and so
This contradicts (1). ∎
While this theorem tells us how to sort the ads selected, it does not tell us which ads to select. One is tempted to say that choosing the top ads by aecpm would do the trick; however the following example proves otherwise:
Example 1
Suppose we have three bidders and two slots, and the bidders have the following parameters:
Bidder  

1  $1  .75  4 
2  $2  .2  2.5 
3  $0.85  .8  4.25 
Let’s consider some possible assignments and their efficiency. If we use simple ranking by ecpm , we get the assignment , which has efficiency . If we use simple ranking by aecpm we get the assignment with efficiency . It turns out that the optimal assignment is with efficiency . The assigned bidders are ordered by aecpm in the assignment, but are not the top 2 bidders by aecpm.
Now suppose we have the same set of bidders, but now we have three slots. The optimal assignment in this case is ; note how bidder 3 goes from being unassigned to being assigned the first position.
3.2 Notation for Assignments
Theorem 1 implies that the optimal assignment can be described by the set of assigned bidders, since this set will always be sorted by aecpm. For a set of bidders, let be the efficiency of an assignment that places only the set , regardless of the number of bidders in ; i.e., if and wlog is sorted by aecpm, then
Similarly, let be the overall continuation probability of the set of ads: . If we have and .
Throughout the paper, we will also make use of the notation for a particular assignment, where uppercase letters denote sets of bidders and lowercase letters denote single bidders. This denotes a solution where the order of the bidders is as given, and where the order within a set is by aecpm (breaking ties using a lexicographic order on bidders). So in this example, the solution would put all the bidders of first in aecpm order, followed by bidder , followed by the bidders of in aecpm order, etc. Note that this notation could express suboptimal solutions (for the particular set of bidders) if the elements are not in order of ecpm. We use and to denote the efficiency and continuation probability of such solutions.
3.3 Bidder Dominance
In classical sponsored search with simple ranking, a bidder can dominate another bidder by having higher ecpm; i.e., bidder will always appear whenever does, and in a higher position. Example 1 above shows that having a higher ecpm (or aecpm) does not allow a bidder to dominate another bidder in our new model. However, we show in this section that if she has higher ecpm and aecpm, then this does suffice. This is not only interesting in its own right, it is essential for proving deeper structural properties in later sections.
Theorem 2
For all bidders in an optimal assignment, if some bidder is not in the assignment, and and , then we may substitute for , and the assignment is no worse.
Proof.
Consider some bidder that appears in an optimal assignment, and some that does not appear in the assignment, such that and . Let be the optimal assignment, where is the sequence of bidders above and is the sequence below . The efficiency of the assignment is The efficiency of the assignment is Suppose ; then clearly is as efficient as since , and the theorem is proven. Thus we assume that for the remainder of the proof. Note that is equivalent to:
(2) 
Now consider the assignment , with efficiency . Since is optimal, we get
i.e., . Combining this with (2), and using the fact that , we get which can be rewritten as
This implies that the solution is as efficient as . ∎
3.4 Subset Substructure in Optimal Assignments
In this section we give a theorem that shows some subset structure between optimal assignments to different numbers of slots. This theorem is used to prove position monotonicity, and is an essential ingredient of our algorithm. Let denote the set of all optimal solutions for filling positions with bidders from the set .
Theorem 3
Let be some number of positions, and let be an arbitrary set of bidders. Then, for all , there is some where .
Proof.
We proceed by induction on , the base case being simple. Let be some solution in , and let be the solution in with the most bidders in common with . We will refer to an ad being “above” another ad if it has higher aecpm. Let be the highest bidder in .
If does not appear in , then we claim that the solution must be in : Once is chosen for , taking any set in for the remaining positions will result in an optimal solution; the set is such a set, since by assumption it does not include , and is in . But if the theorem is proven, so we may assume .
Let be the set of ads in above , and so we can write and . We claim . To see this, consider the set of all ads that have lower aecpm than . By the optimality of , we have for . By induction there is a where . Thus we must have since is the solution with the most bidders in common with . Decompose as where

is the lowest ad in that does not appear in ,

is the set of ads below in (this can be empty),

is the maximal set of ads immediately above in that also appear in (this can be empty),

are the remaining ads in that do not appear in ,

are the remaining ads (besides ) that appear in both and .
Let . Note that by the definitions above we may write . We have since and . Decomposing this a bit gives
(3) 
We also note that
(4)  
Let . The remainder of the proof will show that , which implies the theorem since , and . We can rewrite as follows:
(6)  
In the above, (6) follows from (3), and (6) follows from (4). Rearranging, and using
we get
We know that since otherwise , and this cannot be since . We claim that , which would imply and thus complete the proof. This is trivially true if . Since by the definitions above, we can assume both and are nonempty. consider some and . We have by the definition of . Since is the solution in with the most bidders in common with , we must not be able to substitute for in , and thus by Lemma 2 we must have that or . But by the definitions of and , we have . Therefore . The previous two inequalities imply . Since and were arbitrary and , this gives . ∎
3.5 Monotonicity of Position and Click Probability
In this section we give our main theorem regarding the structure of the optimal assignments in the Markovian click model: that position and click probability are monotonic in a bidder’s bid, with all other bids fixed. This is a fundamental property that makes the bidder’s interaction with the system intuitive, and allows the bidder to adjust her bid intelligently without global knowledge of the other bids.
Theorem 4
With all other bids fixed, the probability of receiving a click in the optimal solution is nondecreasing in one’s bid. In addition, the position of a particular bidder in the optimal solution is monotonic that bidder’s bid.
Proof.
As bidder increases her bid with all other bids fixed, the value of a particular solution increases linearly as . (Solutions not involving stay constant.)
Let denote the sequence of optimal solutions that occurs as increases from . Solution is the best solution not involving , and is the best solution that puts in the first position. By the fact that each solution increases linearly by the term , which is the probability that receives a click in that solution, it must be the case that for a new solution to become optimal it gives a higher click probability than in the previous solution; i.e., for all , gives a higher click probability than . This proves the first part of the theorem.
Now suppose the second part of the theorem is false. Then, there must be some consecutive solutions and where has a higher position in than in . Let be the bid that makes both and optimal, which must exist since they are consecutive in the list of optimal solutions, and fix for the remainder of the proof. Decompose the two solutions as and where by assumption, and by the argument that proved the first part of the theorem. Since both solutions are optimal, they are both sorted by aecpm, with ties broken lexicographically.
We claim that . If this were not the case then some bidder would appear in both and , but since both and are sorted by ecpm, and appears on different sides of , this must mean that has the same aecpm as . But, this violates our assumption on how the algorithm breaks ties among different orderings of the same set. Using similar logic, we get .
By the optimality of and the fact that , we get . Since , Theorem 3 then implies that there is some where . Even if , the set could replace in (since ) and still be optimal, and so we define and have . Note that by the definition of , we have .
Since and , there must be some bidder in that does not appear in . Let be the first such bidder (by aecpm). Decompose into where and are those bidders with higher and lower aecpm than , respectively. Note that , by the definition of . Let be the smallest prefix of (by aecpm) that contains all of , and let .
Similarly, since and , we let be the first bidder (by aecpm) that does not appear in . Let be the bidders in with higher aecpm than , and . Note is also a prefix of , and let .
Given these definitions, we define nine different solutions that we will use in our proof (renaming and for clarity):
In the following claims we will often use the generic identity
(7) 
Claim 1
Proof.
Using (7), we can rewrite the claim as
(8) 
Since , and , we get , and so it remains to prove . But this follows from the optimality of , since could replace in solution (indeed, ). ∎
Claim 2
Proof.
Using (7), we can rewrite the claim as
(9) 
Since has the same length as , and does not contain or any bidders in , it could replace in ; but since is optimal, we may conclude that , which proves the claim. ∎
Note that , since all bidders in have a higher aecpm than . Finally, note that a simple application of (7) gives
(10) 
We now conclude the proof with the following contradiction:
∎
4 Computing the Optimal Assignment
In this section we give algorithms for computing the optimal assignment of bidders to positions using the structural properties we proved in the previous section. We begin with a simple dynamic program that gives an time algorithm. We then show how our insights from the previous sections give a faster time algorithm.
4.1 Optimal Assignment using Dynamic Programming
The algorithm proceeds as follows. First, sort the ads in decreasing order of aecpm in time . Then, let be the efficiency obtained (given that you reach slot ) by filling slots with bidders from the set . We get the following recurrence:
Solving this recurrence for yields the optimal assignment, and can be done in time.
4.2 Nearlinear Time Algorithm
Let be the set of bidders, sorted by aecpm. Suppose we had an oracle that told us, for any , the bidder with that maximizes for an arbitrary linear function . We will later show how to construct this oracle, but first we describe our algorithm that uses this oracle.
Our algorithm will construct a solution for all , the final one being the overall optimum. By Theorem 3, we may assume that . Using this fact, our algorithm builds from by simply finding . To perform this max, the algorithm first guesses (i.e., searches exhaustively for) the aecpm rank of the new bidder among the bidders in ; this a number from to . Let be the elements of sorted by decreasing aecpm. The new bidder has aecpm between and , and so . Since is linear in , we may appeal to the oracle to find the bidder that maximizes among all bidders with aecpm between that of and . We make calls to this oracle for each , and thus calls overall. To get the coefficients of to pass to the oracle, we precompute the quantities and for all . (This can be done in time per , for time overall.)
It remains to show how to implement the oracle. We first preprocess the sequence of bidders as follows. We consider the dyadic intervals for each possible , for a total of intervals. Note that any subsequence is made up of at most such intervals. For each such interval, we will make a data structure that can find over bidders in that interval in time. So overall, given , the oracle takes the max of calls to the data structure, and completes in time.
The data structure we compute for a particular interval is simply the convex hull of the points in twodimensional space defined by bidders in the interval. We can compute all these convex hulls in time by successively merging convex hulls for increasing . Given the convex hull (with the points sorted in order of for example), a simple binary search can find the point maximizing in time.
This gives,
Theorem 5
Consider the auction with Markovian bidders and slots. There is an optimal assignment which can be determined in time.
It follows that using VCG pricing with this optimal assignment, we obtain a truthful mechanism for sponsored search with Markovian users.
5 Concluding Remarks
We approached sponsored search auctions as a three party process by modeling the behavior of users first and then designing suitable mechanisms to affect the game theory between the advertiser and the search engine. This formal approach shows an intricate connection between the user models and the mechanisms.
There are some interesting open issues to understand about our model and mechanism. For example, in order to implement our mechanism, the search engine needs to devise methods to estimate the parameters of our model, in particular, ’s. This is a challenging statistical and machine learning problem. Also, we could ask how much improvement in efficiency and/or revenue is gained by using our model as opposed to VCG without using our model.
More powerful models will also be of great interest. One small extension of our model is to make the continuation probability a function of location as well, which makes the optimization problem more difficult. We can also generalize the Markov model to handle arbitrary configurations of ads on a web page (not necessarily a search results page), or to allow various other user states (such as navigating a landing page). Finally, since page layout can be performed dynamically, we could ask what would happen if the layout of a web page were a part of the mechanism; i.e., a function of the bids.
References
 [1] Z. Abrams and A. Ghosh. Auctions with revenue guarantees for sponsored search. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [2] Z. Abrams, O. Mendelevitch, and J. Tomlin. Optimal delivery of sponsored search advertisements subject to budget constraints. In ACM Conference on Electronic Commerce, 2007.
 [3] G. Aggarwal, J. Feldman, and S. Muthukrishnan. Bidding to the top: VCG and equilibria of positionbased auctions. In Proc. Workshop on Approximation and Online Algorithms (WAOA), 2006.
 [4] G. Aggarwal, A. Goel, and R. Motwani. Truthful auctions for pricing search keywords. In ACM Conference on Electronic Commerce (EC), 2006.
 [5] C. Borgs, J. Chayes, O. Etesami, N. Immorlica, K. Jain, and M. Mahdian. Dynamics of bid optimization in online advertisement auctions. In Proc. WWW, 2007.
 [6] C. Borgs, J. T. Chayes, N. Immorlica, M. Mahdian, and A. Saberi. Multiunit auctions with budgetconstrained bidders. In ACM Conference on Electronic Commerce (EC), 2005.
 [7] M. Cary, A. Das, B. Edelman, I. Giotis, K. Heimerl, A. R. Karlin, C. Mathieu, and M. Schwarz. Greedy bidding strategies for keyword auctions. In ACM Conference on Electronic Commerce, 2007.
 [8] E. Clarke. Multipart pricing of public goods. Public Choice, 11:17–33, 1971.
 [9] B. Edelman, M. Ostrovsky, and M. Schwarz. Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. In Second workshop on sponsored search auctions, 2006.
 [10] J. Feldman, S. Muthukrishnan, M. Pál, and C. Stein. Budget optimization in searchbased advertising auctions. In ACM Conference on Electronic Commerce, 2007.
 [11] J. Feldman, E. Nikolova, S. Muthukrishnan, and M. Pál. A truthful mechanism for offline ad slot scheduling. In Proc. Symposium on Algorithmic Game Theory (SAGT), 2008. To appear.
 [12] T. Groves. Incentives in teams. Econometrica, 41(4):617–631, 1973.
 [13] S. Lahaie. An analysis of alternative slot auction designs for sponsored search. In ACM Conference on Electronic Commerce (EC), 2006.
 [14] S. Lahie, D. Pennock, A. Saberi, and R. Vohra. Sponsored Search Auctions, in: Algorithmic Game Theory, pages 699–716. Cambirdge University Press, 2007.
 [15] Li Liang and Qi Qi. Cooperative or vindictive: Bidding strategies in sponsored search auction. In WINE, pages 167–178, 2007.
 [16] M. Mahdian, H. Nazerzadeh, and A. Saberi. Allocating online advertisement space with unreliable estimates. In ACM Conference on Electronic Commerce, 2007.
 [17] M. Mahdian and A. Saberi. Multiunit auctions with unknown supply. In ACM conference on Electronic commerce (EC), 2006.
 [18] A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani. AdWords and generalized online matching. In FOCS, 2005.
 [19] S. Muthukrishnan, M. Pál, and Z. Svitkina. Stochastic models for budget optimization in searchbased advertising. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [20] P. Rusmevichientong and D. Williamson. An adaptive algorithm for selecting profitable keywords for searchbased advertising services. In Proc. 7th ACM conference on Electronic commerce, 2006.
 [21] B. K. Szymanski and J.S. Lee. Impact of roi on bidding and revenue in sponsored search advertisement auctions. In 2nd Workshop on Sponsored Search Auctions in ACM Conference on Electronic Commerce (EC), 2006.
 [22] H. Varian. Position auctions. International Journal of Industrial Organization, 25(6):1163–1178, December 2007.
 [23] W. Vickrey. Counterspeculation, auctions and competitivesealed tenders. Finance, 16(1):8–37, 1961.
 [24] Y. Vorobeychik and D. M. Reeves. Equilibrium analysis of dynamic bidding in sponsored search auctions. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [25] J. Wortman, Y. Vorobeychik, L. Li, and J. Langford. Maintaining equilibria during exploration in sponsored search auctions. In Proc. Workshop on Internet and Network Economics (WINE), 2007.