Algorithmic Methods for Sponsored Search Advertising
Abstract
Modern commercial Internet search engines display advertisements along side the search results in response to user queries. Such sponsored search relies on market mechanisms to elicit prices for these advertisements, making use of an auction among advertisers who bid in order to have their ads shown for specific keywords. We present an overview of the current systems for such auctions and also describe the underlying gametheoretic aspects. The game involves three parties—advertisers, the search engine, and search users—and we present example research directions that emphasize the role of each. The algorithms for bidding and pricing in these games use techniques from three mathematical areas: mechanism design, optimization, and statistical estimation. Finally, we present some challenges in sponsored search advertising.
1 Introduction
Targeted advertisements on search queries is an increasingly important advertising medium, attracting large numbers of advertisers and users. When a user poses a query, the search engine returns search results together with advertisements that are placed into positions, usually arranged linearly down the page, top to bottom. On most major search engines, the assignment of ads to positions is determined by an auction among all advertisers who placed a bid on a keyword that matches the query. The user might click on one or more of the ads, in which case (in the payperclick modelPayperclick model) the advertiser receiving the click pays the search engine a price determined by the auction.
In the past few years, the sponsored search model has been highly successful commercially, and the research community is attempting to understand the underlying dynamics, explain the behavior of the market and improve the auction algorithms. This survey will provide an overview of the algorithmic issues in sponsored search.
The basic view we emphasize is the role of the three partiesThree parties in sponsored search.

The first party is the advertisers who have multiple objectives in seeking to place advertisements. Some advertisers want to develop their brand, some seek to make sales, and yet others advertise for defensive purposes on specific keywords central to their business. Some have budget constraints, while others are willing to spend as much as it takes to achieve their goal. Some seek to obtain many clicks and eyeballs, yet others attempt to optimize their return on investment. So, in general, advertisers are of varied types.

The second party is the auctioneer, in this case, the search engine. The search engines have to balance many needs. They must maintain useful search results and have advertisements enhance, rather than interfere with, the search experience. They need to make sure the advertisers get their needs fulfilled, and at the same time ensure that the market the advertisers participate in is efficient and conducive to business.

The third party is perhaps the most important in the game: these are search users. Users come to search engines for information and pointers. In addition, they also come to discover shopping opportunities, good deals, and new products. There are millions of users with different goals and behavior patterns with respect to advertisements.
These three parties induce a fairly sophisticated dynamics. While economic and game theory provide a welldeveloped framework for understanding the auction game between the advertisers and the auctioneer, the community has had to generalize such methods and apply them carefully to understand the currently popular Internet auctions. Likewise, while there has been recent work on understanding models of user behavior for posing search queries and their click behavior for search responses, little is known about user behavior on advertisements, and crucially, these affect the value of the slots and thus the very goods that are sold in auction.
In this survey, we will show examples of research themes in algorithmic, optimization and gametheoretic issues in sponsored search. In particular, we present three examples each emphasizing the perspective of one of the three different parties involved in sponsored search: the advertisers (who act as the bidders), the search engine (who acts as the auctioneer), and the search engine user (who determines the commodity). More specifically,

We present results for how an advertiser should choose their bids given the currently used auction mechanism and implicit user behavior models. This result appears as FMPS (). It shows that a very simple bidding strategy is very effective for the advertiser.

We study a new mechanism for the auctioneer to allocate advertisements to slots in order to optimize efficiency, and analyze the gametheoretic aspects of this mechanism. This result appears in FMNP (). It shows that a simple pricesetting mechanism is suitable for determining the outcome of several auctions simultaneously for the auctioneer.

We present a novel Markovian model of user behavior when shown advertisements, and for this model, develop mechanisms and game theory. This result appears in AFMP08 (). It shows that under a model of user behavior more general than the one that is implicit in existing auctions, entirely different allocation and pricing will be optimal. Hence, user models have significant impact.
The results above are joint work with Gagan Aggarwal, Evdokia Nikolova, Martin Pál and Cliff Stein, and represent work done at Google Research.
In the rest of the document, we will first describe the foundations behind the existing auctions. Then we will describe the three results above. After that, we will be able to point to open issues and provide concluding remarks more generally on Internet advertising and auctions.
2 Existing Auctions
The basic auction behind sponsored searchSponsored search auction occurs when a user submits a query to the search engine. The screen shot in Figure 1 shows an example user query “soda” and search results page returned from the search engine. This page includes web search results on the left, and independently, a set of three text ads on the right, arranged linearly top to bottom, clearly marked “Sponsored links.” Each advertiser has previously submitted a bid stating their value for a click, tying their bid to a specific keyword. The auction is held in realtime among advertisers whose keywords match that user’s query. The result of the auction is the list of advertisements on the right. So, after selecting the set of eligible (matching) ads, running the auction involves the search engine determining (a) the ordering of bidders and (b) pricing.

Ordering: The most natural ordering is to sort by decreasing bid, but that does not take into account the quality of ads and their suitability to users. Thus, it is common practice to place the bidders in descending order of , where is what is called the clickthroughrate (ctr)Clickthrough rate (ctr)ctr (clickthrough rate) of advertiser , i.e., the probability that a user will click on the ad, given that the user looks at it. (The ctr is usually measured by the search engine.) This is the ordering currently in use by search engines like Yahoo!Yahoo! and Google.Google

Pricing: The natural method is to make bidders pay what they bid, but that leads to wellknown race conditions EOS (). Instead, the most common method is to use a “generalized second price” (GSP) auctionGSP (Generalized SecondPrice) auction. Say the positions are numbered starting at the top and going down, and the bidder at position has bid . In GSP, the price for a click for the advertiser in position is determined by the advertisement below it and is given by , which is the minimum they would have needed to bid to attain their position.
The first academic treatments of the sponsored search auction were naturally from the perspective of auction and game theoryAuction theory. Fixing this ordering of the bidders, authors in EOS (); Varian (); AGM () focused on understanding the implications of different pricing schemes, assuming strategic behavior on the part of the advertisers. The setting of the game that is modeled in this work is as follows: each advertiser has a private value for a click from this user, and wants to set a bid that maximizes her utility . The natural economic utility model in this context would be profit:Profit utility model i.e.,
where is the price per click, and is the probability of a click occurring. Of course is determined by the user, and may depend on any number of factors. One common model is to assume that the click probability is separable AGM ():Separable click probability if ad is placed into position , then where is the adspecific “clickthrough rate” and is a positionspecific visibility factor.Positionspecific visibility factor (We will later explore other utility models in Sections 3 and 4 when we incorporate budgets, as well as nonseparable user models in Section 5.)
Natural questions in this context include asking whether there is a purestrategyPurestrategy Nash equilibrium Nash equilibriumNash Equilibrium of this game, and analyzing the economic efficiency and revenue of such equilibria. By economic efficiencyEconomic efficiency we mean the total advertiser value generated by the assignment. This is also commonly referred to as the social welfare.Social welfare In the context of sponsored search, the efficiency is the sum of the individual advertisers’ values; i.e., , where is the probability that will receive a click under this assignment and is ’s private value for a click. By a purestrategy Nash equilibrium we mean a set of bids such that no single bidder can change her bid and increase her utility.
Among the most desirable properties of a mechanism is to be truthful,Truthful mechanism which is also referred to as being incentive compatible.Incentive compatible This property says that each bidder’s best strategy, regardless of the actions of other bidders, is simply to report her true value; i.e., submit as her bid. Truthfulness immediately implies the existence of a purestrategy Nash equilibrium (where every bidder reports ). Furthermore, it is simple to compute economic efficiency, since the assignment (and thus the efficiency) is simply a function of the values . Unfortunately, it turns out that the GSP auction is not truthful. However, there is a pricing scheme that is truthful, which is based on an application of the famous VickreyClarkeGroves (VCG) mechanism V (); C (); G ()VCG (VickreyClarkeGroves) mechanism. Furthermore, the GSP auction, while not truthful, still has a wellunderstood purestrategy Nash equilibrium:
Theorem 2.1 (Eos (); Varian (); Agm ())
Suppose we have a set of bidders participating in a particular sponsored search auction. Assume each bidder has a private value and a profitmaximizing utility function. Suppose further that the click probabilities are separable. Then, the GSP auction is not truthful, but it does have a purestrategy Nash equilibrium whose outcome (in terms of assignment and prices) is equivalent to an application of the truthful VCG auction.
For a more detailed discussion of this line of research, we refer the reader to nisan_chapter (). Authors in AGM () also show that under a more general click probability model, there is a pricing method that is truthful. (This pricing method reduces to the VCG pricing method when the clickthrough rates are separable.) Furthermore, they show that in this more general setting the GSP has a Nash equilibrium that has the same outcome as their mechanism.
2.1 Practical Aspects
The results described above regard GSP as an isolated auction, abstracting away the context of the larger system of which it is a part. While this is useful from a modeling perspective, there are many other elements that make sponsored search a more complex environment. Here we list some of those complicating factors, and mention examples of work done to address them.

Multiple queries, multiple keywords. Each sponsored auction is conducted for a particular search engine user with a potentially unique query. There are perhaps millions of such queries every day. Advertisers must submit bids on keywords,Keywords in sponsored search and cannot adjust those bids on a perquery basis. The degree to which the keyword matches a particular query determines not only whether the advertiser will participate in the auction (and also who her competitors will be), but also can factor into the clickthrough rate that is used for ranking. Theorem 2.1 only applies to the case where the same auction—with the same set of advertisers, and the same clickthrough rates—is repeated, and the bids qualify only for that set of auctions. A lot of the work mentioned below takes on this complication in various ways; we give two such examples in Sections 3 and 4.

Budgets. In the privatevalue modelPrivatevalue model each advertiser has a value per click, but is willing to spend an arbitrary amount to maximize her profit. In reality, many advertisers have operating budgets or spending targets, and simply want to maximize their value given the constraints of that budget. This budget can be reported to the search engine, who can then employ techniques to use the budget efficiently. Analysis of incentives becomes more difficult in the presence of budgets. This has been addressed in e.g., BCIMS (); MS (); MNS (); MSVV (); MNS (); AMT (); mps (); RW (); FMPS (); AFMP08 (), and we discuss two examples in much more detail in Sections 3 and 4.

Reserve prices. The major search engines enforce reserve prices,Reserve prices in sponsored search dictating the minimum price that an advertiser can pay for a click. Sometimes these reserve prices will even be specific to a particular bidder. Reserve prices are useful for controlling quality on the search results page, and also have implications for revenue. The effect of reserve prices on the game theory of sponsored search is discussed in detail in EFMM08 ().

Interdependent click probabilities. The “separable” assumption implies that an advertiser’s click probability depends only on the properties and position of her own ad. This ignores the other ads on the search results page, which certainly affect the user experience, and therefore the click probability of this advertiser. We discuss this further in Section 5.

Branding. The private clickvalue model assumes that a click is what the advertiser is ultimately interested in. However a branding advertiser could be interested in her ad appearing in a high position, but not really care whether or not it gets a click (other than due to the fact that they only pay if it does). (Indeed, a recent empirical study by the Interactive Advertising Bureau and Nielsen//NetRatings concluded that higher ad positions in paid search have a significant brand awareness effect iabstudy ().) Thus we might be interested in an auction where an advertiser can express the lowest position she is willing to tolerate for her ad. This is the approach taken in afm (), where Theorem 2.1 is generalized to this setting.

Conversions. The private clickvalue model also assumes that each click is worth the same to an advertiser, which is not always the case in practice. Indeed many advertisers track whether or not a click leads to a conversion,Conversions in sponsored search which is some sort of event on the linked page (e.g., a sale, a signup, etc.) Given this data, the advertiser can learn which keywords lead to conversions and therefore which clicks are worth more to them.

Estimating various parameters. Most work in the context of the game theory of sponsored search has assumed that the parameters like clickthrough rate and position visibility are known. However, estimating these parameters is a difficult task (e.g., IJMT (); RDR ()). Indeed, there is an inherent tradeoff between learning these parameters and applying them; one cannot learn that an ad has a bad ctr unless it is exposed to the user, but then it was a bad idea to show it in the first place. This “exploration/exploitation” tradeoff turns out to be related to the “multiarmed bandit”Multiarmed bandit problem problem (see e.g. GP (); WVLL (); EMM02 ()).

Incomplete Knowledge. Both the advertisers and the search engine have incomplete knowledge of the “inventory” available to them, since they do not know which queries will arrive. In addition the bidders do not know the other bids or clickthrough rates. This makes the advertiser’s optimization problem much more difficult (see e.g., Lahaie (); BCEIJM (); CDEGHKMS (); FMPS (); VR (); LQ (); ZL07 (); SL (); RW ()). From the search engine’s point of view, we can model incomplete knowledge of the future as an online algorithm; see e.g. MSVV (); RW (); mps (); WVLL (); MNS (); MS (); AG (); GP (); GM07 (); GM08 ().
3 The Advertiser’s Point of View: Budget Optimization
The perspective in this section is the advertisers. The challenge from an advertiser’s point of view is to understand and interact with the auction mechanism. The advertiser determines a set of keywords of their interest^{1}^{1}1The choice of keywords is related to the domainknowledge of the advertiser, user behavior and strategic considerations. Internet search companies provide the advertisers with summaries of the query traffic which is useful for them to optimize their keyword choices interactively. We do not directly address the choice of keywords in this section, which is addressed elsewhere RW (). and then must create ads, set the bids for each keyword, and provide a total (often daily) budget.
While the effect of an ad campaign in any medium is a sophisticated phenomenon that is difficult to quantify, one commonly accepted (and easily quantified) notion in searchbased advertising on the Internet is to maximize the number of clicks. The Internet search companies are supportive towards advertisers and provide statistics about the history of click volumes and prediction about the future performance of various keywords. Still, this is a sophisticated problem for the following reasons (among others):

Individual keywords have significantly different characteristics from each other; e.g., while “fishing” is a broad keyword that matches many user queries and has many competing advertisers, “humane fishing bait” is a niche keyword that matches only a few queries, but might have less competition.

There are complex interactions between keywords because a user query may match two or more keywords, since the advertiser is trying to cover all the possible keywords in some domain. In effect the advertiser ends up competing with herself.
As a result, the advertisers face a challenging optimization problem. The focus of the work in FMPS () is to solve this optimization problem.
Problem Formulation. We present a short discussion and formulation of the optimization problem faced by advertisers; a more detailed description is in Section 3.1.
A given advertiser sees the state of the auctions for searchbased advertising as follows. There is a set of keywords of interest; in practice, even small advertisers typically have a large set . There is a set of queries posed by the users. For each query , there are functions giving the and that result from bidding a particular amount in the auction for that query, which we will see a more formal model of in the next section. There is a bipartite graph on the two vertex sets representing and . For any query , the neighbors of in are the keywords that are said to “match” the query .^{2}^{2}2The particulars of the matching rule are determined by the Internet search company; here we treat the function as arbitrary.
The budget optimization problem is as follows. Given graph together with the functions and on the queries, as well as a budget , determine the bids for each keyword such that is maximized subject to , where the “effective bid” on a query is some function of the keyword bids in the neighborhood of .
While we can cast this problem as a traditional optimization problem, there are different challenges in practice depending on the advertiser’s access to the query and graph information, and indeed the reliability of this information (e.g., it could be based on unstable historical data). Thus it is important to find solutions to this problem that not only get many clicks, but are also simple, robust and less reliant on the information. The notion of a “uniform” strategy is defined in FMPS () which is essentially a strategy that bids uniformly on all keywords. Since this type of strategy obviates the need to know anything about the particulars of the graph, and effectively aggregates the click and cost functions on the queries, it is quite robust, and thus desirable in practice. What is surprising is that uniform strategy actually performs well, which is proved in FMPS ().
Main Results and Technical Overview. Some positive and negative results are given in FMPS () for the budget optimization problem:

Nearly all formulations of the problem are NPHard. In cases slightly more general than the formulation above, where the clicks have weights, the problem is inapproximable better than a factor of , unless P=NP.

There is a approximation algorithm for the budget optimization problem. The strategy found by the algorithm is a twobid uniform strategy, which means that it randomizes between bidding some value on all keywords, and bidding some other value on all keywords until the budget is exhausted^{3}^{3}3This type of strategy can also be interpreted as bidding one value (on all keywords) for part of the day, and a different value for the rest of the day.. This approximation ratio is tight for uniform strategies. There is also a approximation algorithm that offers a singlebid uniform strategy, only using one value . (This is tight for singlebid uniform strategies.) These strategies can be computed in time nearly linear in , the input size.
Uniform strategies may appear to be naive in first consideration because the keywords vary significantly in their click and cost functions, and there may be complex interaction between them when multiple keywords are relevant to a query. After all, the optimum can configure arbitrary bids on each of the keywords. Even for the simple case when the graph is a matching, the optimal algorithm involves placing different bids on different keywords via a knapsacklike packing (Section 3.1). So, it might be surprising that a simple twobid uniform strategy is or more effective compared to the optimum. In fact, our proof is stronger, showing that this strategy is effective against a strictly more powerful adversary who can bid independently on the individual queries, i.e., not be constrained by the interaction imposed by the graph .
We will also look at the simulations conducted in FMPS () using real auction data from Google. The results of these simulations suggest that uniform bidding strategies could be useful in practice. However, important questions remain about (among other things) alternate bidding goals, online or stochastic bidding models mps (), and gametheoretic concerns BCIMS (), which we briefly discuss in Section 3.4.
3.1 Modeling a Keyword Auction
We begin by considering the case of a single keyword that matches a single user query. In this section we define the notion of a “query landscape” that describes the relationship between the advertiser’s bid and what will happen on this query as a result of this bid autobidder (). This definition will be central to the discussion as we continue to more general cases.
The search results page for a query contains possible positions in which our ad can appear. We denote the highest (most favorable) position by and lowest by . Assuming a separable user model, associated with each position is a value that denotes the click probability if the ad appears in position .^{4}^{4}4We leave out the adspecific factor from this section for clarity, but all the results in FMPS () generalize to this case as well. We assume throughout this section that that if , that is, higher positions receive at least as many clicks as lower positions.
In order to place an ad on this page, we must enter the GSP auction that is carried out among all advertisers that have submitted a bid on a keyword that matches the user’s query. We will refer to such an auction as a query auction,Query auction to emphasize that there is an auction for each query rather than for each keyword. In GSP, the advertisers are ranked in decreasing order of bid, and each advertiser is assigned a price equal to the amount bid by the advertiser below them in the ranking. Let denote the bids of the top advertisers in this query auction. For notational convenience, we assume that and . Since the auction is a generalized second price auction, higher bids win higher positions; i.e. . Suppose that we bid on some keyword that matches the user’s query, then our position is defined by the largest that is at most , that is,
(1) 
Since we only pay if the user clicks (and that happens with probability ), our expected cost for winning position would be We use and to denote the expected cost and clicks that result from having a bid that qualifies for a query auction , and thus
(2) 
(3) 
When the context is clear, we drop the subscript . The following observations about cost and clicks follow immediately from the definitions and equations (1), (2) and (3). We use to denote the nonnegative reals.
Proposition 1
For ,

The tuple () can only take on one of a finite set of values .

Both and are nondecreasing functions of .

Costperclick (cpc)cpc (CostPerClick)Costperclick (cpc) is nondecreasing in , and is always at most the bid; i.e., .
Query LandscapesQuery landscape We can summarize the data contained in the functions and as a collection of points in a plot of cost vs. clicks, which we refer to as a landscape.Landscape in a query auction For example, for a query with four slots, a landscape might look like Table 1.
bid range  cost per click  cost  clicks 

[$2.60, )  $2.60  $1.30  .5 
[$2.00, $2.60)  $2.00  $0.90  .45 
[$1.60, $2.00)  $1.60  $0.40  .25 
[$0.50, $1.60)  $0.50  $0.10  .2 
[$0, $0.50)  $0  $0  0 
It is convenient to represent this data graphically as in Figure 2 (ignore the dashed line for now). Here we graph clicks as a function of cost. Observe that in this graph, the cpc of each point is the reciprocal of the slope of the line from the origin to the point. Since , and are nondecreasing, the slope of the line from the origin to successive points on the plot decreases. This condition is slightly weaker than concavity.
Suppose we would like to solve the budget optimization problem for a single query landscape.^{5}^{5}5Of course it is a bit unrealistic to imagine that an advertiser would have to worry about a budget if only one user query was being considered; however one could imagine multiple instances of the same query and the problem scales. As we increase our bid from zero, our cost increases and our expected number of clicks increases, and so we simply submit the highest bid such that we remain within our budget.
One problem we see right away is that since there are only a finite set of points in this landscape, we may not be able to target arbitrary budgets efficiently. Suppose in the example from Table 1 and Figure 2 that we had a budget of . Bidding between and uses only , and so we are underspending. Bidding more than is not an option, since we would then incur a cost of and overspend our budget.
Randomized strategies To rectify this problem and better utilize our available budget, we allow randomized bidding strategies.Randomized bidding strategy Let be a distribution on bids . Now we define and . Graphically, the possible values of lie in the convex hullConvex hull of the landscape points. This is represented in Figure 2 by the dashed line.
To find a bid distribution that maximizes clicks subject to a budget, we simply draw a vertical line on the plot where the cost is equal to the budget, and find the highest point on this line in the convex hull. This point will always be the convex combination of at most two original landscape points which themselves lie on the convex hull. Thus, given the point on the convex hull, it is easy to compute a distribution on two bids which led to this point. Summarizing,
Lemma 1
FMPS () If an advertiser is bidding on one query, subject to a budget , then the optimal strategy is to pick a convex combination of (at most) two bids which are at the endpoints of the line on the convex hull at the highest point for cost .
There is one subtlety in this formulation. Given any bidding strategy, randomized or otherwise, the resulting cost is itself a random variable representing the expected cost. Thus if our budget constraint is a hard budget, we have to deal with the difficulties that arise if our strategy would be over budget. Therefore, we think of our budget constraint as soft,Soft budget constraint that is, we only require that our expected cost be less than the budget. In practice, the budget is often an average daily budget, and thus we don’t worry if we exceed it one day, as long as we are meeting the budget in expectation. Further, either the advertiser or the search engine (possibly both), monitor the cost incurred over the day; hence, the advertiser’s bid can be changed to zero for part of the day, so that the budget is not overspent.^{6}^{6}6See https://adwords.google.com/support/bin/answer.py?answer=22183, for example. Thus in the remainder of this section, we will formulate a budget constraint that only needs to be respected in expectation.
Multiple Queries: a Knapsack Problem As a warmup, we will consider next the case when we have a set of queries, each with its own landscapeKnapsack problem. We want to bid on each query independently subject to our budget: the resulting optimization problem is a small generalization of the fractional knapsackFractional knapsack problem problem, and was solved in autobidder ().
The first step of the algorithm is to take the convex hull of each landscape, as in Figure 2, and remove any landscape points not on the convex hull. Each piecewise linear section of the curve represents the incremental number of clicks and cost incurred by moving one’s bid from one particular value to another. We regard these “pieces” as items in an instance of fractional knapsack with value equal to the incremental number of clicks and size equal to the incremental cost. More precisely, for each piece connecting two consecutive bids and on the convex hull, we create a knapsack item with value and size . We then emulate the greedy algorithm for knapsack, sorting by value/size (costperclick), and choosing greedily until the budget is exhausted.
In this reduction to knapsack we have ignored the fact that some of the pieces come from the same landscape and cannot be treated independently. However, since each curve is concave, the pieces that come from a particular query curve are in increasing order of costperclick; thus from each landscape we have chosen for our “knapsack” a set of pieces that form a prefix of the curve.
Keyword Interaction
Keyword interaction
In reality, search advertisers can bid on a large set of keywords, each of them qualifying for a different (possibly overlapping) set of queries, but most search engines do not allow an advertiser to appear twice in the same search results page.^{7}^{7}7See https://adwords.google.com/support/bin/answer.py?answer=14179, for example. Thus, if an advertiser has a bid on two different keywords that match the same query, this conflict must be resolved somehow. For example, if an advertiser has a bid out on the keywords “shoes” and “highheel,” then if a user issues the query “highheel shoes,” it will match on two different keywords. The search engine specifies, in advance, a rule for resolution based on the query the keyword and the bid. A natural rule is to take the keyword with the highest bid, which we adopt here, but our results apply to other resolution rules.
We model the keyword interaction problem using an undirected bipartite graph where is a set of keywords and is a set of queries. Each has an associated landscape, as defined by and . An edge means that keyword matches query .
The advertiser can control their individual keyword bid vector specifying a bid for each keyword . (For now, we do not consider randomized bids, but we will introduce that shortly.) Given a particular bid vector on the keywords, we use the resolution rule of taking the maximum to define the “effective bid” on query as
By submitting a bid vector , the advertiser receives some number of clicks and pays some cost on each keyword. We use the term spend to denote the total cost; similarly, we use the term traffic to denote the total number of clicks:
We also allow randomized strategies, where an advertiser gives a distribution over bid vectors . The resulting spend and traffic are given by
We can now state the problem in its full generality:Budget Optimization Problem
Budget Optimization
Input: a budget , a keywordquery graph , and landscapes for each .
Find: a distribution over bid vectors such that and is maximized.
We conclude this section with a small example to illustrate some feature of the budget optimization problem. Suppose you have two keywords and two queries and edges Suppose query has one position with ctr , and there is one bid . Query has two positions with ctrs , and bids and To get any clicks from , an advertiser must bid at least on . However, because of the structure of the graph, if the advertiser sets to $1, then his effective bid is on both and . Thus he must tradeoff between getting the clicks from and getting the bargain of a click for that would be possible otherwise.
3.2 Uniform Bidding Strategies
As shown in FMPS (), solving the Budget Optimization problem in its full generality is difficult. In addition, it may be difficult to reason about strategies that involve arbitrary distributions over arbitrary bid vectors. Advertisers generally prefer strategies that are easy to understand, evaluate and use within their larger goals. With this motivation, we look at restricted classes of strategies that we can easily compute, explain and analyze.
We define a uniform bidding strategy to be a distribution over bid vectors where each bid vector in the distribution is of the form for some realvalued bid . In other words, each vector in the distribution bids the same value on every keyword.Uniform bidding strategy
Uniform strategies have several advantages. First, they do not depend on the edges of the interaction graph, since all effective bids on queries are the same. Thus, they are effective in the face of limited or noisy information about the keyword interaction graph. Second, uniform strategies are also independent of the priority rule being used. Third, any algorithm that gives an approximation guarantee will then be valid for any interaction graph over those keywords and queries.
Define a twobid strategyTwobid strategy to be a uniform strategy which puts nonzero weight on at most two bid vectors. Given the landscapes for all the queries, we can compute the best uniform strategy in linear time; the proof also directly implies that there is always an optimal twobid strategy:
Lemma 2
FMPS () Given an instance of Budget Optimization in which there are a total of points in all the landscapes, we can find the best uniform strategy in time. Furthermore, this strategy will always be a twobid strategy.
The authors in FMPS () also consider singlebid strategies,Singlebid strategy which are uniform strategies that put nonzero weight on at most one nonzero vector, i.e. advertiser randomizes between bidding a certain amount on all keywords, and not bidding at all. A singlebid strategy is even easier to implement in practice than a twobid strategy. For example, the search engines often allow advertisers to set a maximum daily budget. In this case, the advertiser would simply bid until her budget runs out, and the ad serving system would remove her from all subsequent auctions until the end of the day. One could also use an “ad scheduling”Ad scheduling tool offered by some search companies^{8}^{8}8See https://adwords.google.com/support/bin/answer.py?answer=33227, for example. to implement this strategy. The best singlebid strategy can also be computed easily from the aggregate landscape. The optimal strategy for a budget will either be the point s.t. is as large as possible without exceeding , or a convex combination of zero and the point , where is as small as possible while larger than .
Approximation Guarantees of Uniform Strategies. In fact, not only are uniform strategies easy to optimize over, they are also guaranteed to have good performance compared to the optimal solution. In the case of singlebid strategies, we have the following:
Theorem 3.1
FMPS () There always exists a uniform singlebid strategy that is optimal. Furthermore, for any there exists an instance for which all singlebid strategies are at most optimal.
For general uniform strategies—where a twobid strategy is always optimal—FMPS () proves a tighter approximation ratio:
Theorem 3.2
FMPS () There always exists a uniform bidding strategy that is optimal. Furthermore, for any , there exists an instance for which all uniform strategies are at most optimal.
Thus if given full information about the landscapes, a bidder has an efficient strategy to get a large fraction of the available clicks at her budget. But perhaps more importantly, these theorems show that the simple uniform bidding heuristic can perform well.
3.3 Experimental Results
The authors in FMPS () ran simulations using the data available at GoogleGoogle which we briefly summarize here. They took a large advertising campaign, and, using the set of keywords in the campaign, computed three different curves (see Figure 3) for three different bidding strategies. The xaxis is the budget (units removed), and the yaxis is the number of clicks obtained (again without units) by the optimal bid(s) under each respective strategy. “Query bidding” represents the (unachievable) upper bound , bidding on each query independently. The “uniform bidding” curves represent the results of applying the algorithm: “deterministic” uses a single bid level, while “randomized” uses a distribution. For reference, we include the lower bound of a fraction of the top curve.
The data clearly demonstrate that the best single uniform bid obtains almost all the possible clicks in practice. Of course in a more realistic environment without full knowledge, it is not always possible to find the best such bid, so further investigation is required to make this approach useful. However, just knowing that there is such a bid available should make the online versions of the problem simpler.
3.4 Extensions
The algorithmic result presented here gives an intriguing heuristic in practice: bid a single value on all keywords; at the end of the day, if the budget is underspent, adjust to be higher; if budget is overspent, adjust to be lower; else, maintain . If the scenario does not change from day to day, this simple strategy will have the same theoretical properties as the onebid strategy, and in practice, is likely to be much better. Of course the scenario does change, however, and so coming up with a “stochastic” bidding strategyStochastic bidding strategy remains an important open direction, explored somewhat by mps (); RW ().
Another interesting generalization is to consider weights on the clicks, which is a way to model conversions. (A conversion corresponds to an action on the part of the user who clicked through to the advertiser site; e.g., a sale or an account signup.) Finally, we have looked at this system as a black box returning clicks as a function of bid, whereas in reality it is a complex repeated game involving multiple advertisers. In BCIMS (), it was shown that when a set of advertisers use a strategy similar to the one suggested in FMPS (), under a slightly modified firstprice auction, the prices approach a wellunderstood market equilibrium.
4 The Search Engine’s Point of View: Offline Slot Scheduling
In the previous section we saw that when we take the GSP auction as given, and view the world through the lens of the bidder, the practical problem becomes more complex than what the individual auction was designed for. But we could take the question back to the search engine and ask if there is a more general mechanism that regards the entire day’s worth of queries as part of a single overall game. This question is addressed in FMNP (), where the Offline Ad Slot Scheduling problem is defined: given a set of bidders with bids (per click) and budgets (per day), and a set of slots over the entire day where we know the expected number of clicks in each slot, find a schedule that places bidders into slots. The schedule must not place a bidder into two different slots at the same time. In addition, we must find a price for each bidder that does not exceed the bidder’s budget constraint, nor their perclick bid. (See below for a formal statement of the problem.)
A good algorithm for this problem will have high revenue. Also, we would like the algorithm to be truthful; i.e., each bidder will be incented to report her true bid and budget. In order to prove something like this, we need a utility function for the bidder that captures the degree to which she is happy with her allocation. Natural models in this context (with clicks, bids and budgets) areClick maximization utility function clickmaximization—where she wishes to maximize her number of clicks subject to her personal bid and budget constraints, or profitmaximization—where she wishes to maximize her profit (clicks profit per click). The work in FMNP () is focused on clickmaximization.^{9}^{9}9This choice is in part motivated by the presence of budgets, which have a natural interpretation in this application: if an overall advertising campaign allocates a fixed portion of its budget to online media, then the agent responsible for that budget is incented to spend the entire budget to maximize exposure. In contrast, under the profitmaximizing utility, a weak motivation for budgets is a limit on liquidity. Also, this choice of utility function is out of analytical necessity: Borgs et al. BCIMS () show that under some reasonable assumptions, truthful mechanisms are impossible under a profitmaximizing utility.
We present the efficient mechanism of FMNP () for Offline Ad Slot Scheduling, which is truthful under clickmaximization. Also, the revenueoptimal mechanism for Offline Ad Slot Scheduling is not truthful, but has a Nash equilibrium (under the same utility model) whose outcome is equivalent to the FMNP () mechanism; this result is strong evidence that the mechanism has desirable revenue properties.
Problem Definition. The Offline Ad Slot Scheduling problem FMNP () is defined as followsAd slot schedulingOffline ad slot scheduling. We have bidders interested in clicks. Each bidder has a budget and a maximum costperclick (maxcpc) . Given a number of clicks , and a price per click , the utility of bidder is if both the true maxcpc and the true budget are satisfied, and otherwise. In other words, if and ; and otherwise. We have advertising slots where slot receives clicks during the time interval . We assume .
In a schedule, each bidder is assigned to a set of (slot, time interval) pairs , where and . A feasible scheduleFeasible schedule in ad slot scheduling is one where no more than one bidder is assigned to a slot at any given time, and no bidder is assigned to more than one slot at any given time. (In other words, the intervals for a particular slot do not overlap, and the intervals for a particular bidder do not overlap.) A feasible schedule can be applied as follows: when a user query comes at some time , the schedule for that time instant is used to populate the ad slots. If we assume that clicks come at a constant rate throughout the interval , the number of clicks a bidder is expected to receive from a schedule is the sum of over all pairs in her schedule.^{10}^{10}10All the results of FMNP () generalize to the setting where each bidder has a bidderspecific factor in the clickthrough rate and thus receives clicks (see Section 4.3). We leave this out for clarity.
A mechanism for Offline Ad Slot Scheduling takes as input a declared budget and declared maxcpc (the “bid”) , and returns a feasible schedule, as well as a price per click for each bidder. The schedule gives some number of clicks to each bidder that must respect the budget at the given price; i.e., we have . The revenue of a mechanism is . A mechanism is truthful if it is a weakly dominant strategy to declare one’s true budget and maxcpc; i.e., for any bidder , given any set of bids and budgets declared by the other bidders, declaring her true budget and maxcpc maximizes . In this setting, a (pure strategy) Nash equilibrium is a set of declared bids and budgets such that no bidder wants to change her declaration of bid or budget, given that all other declarations stay fixed. An Nash equilibriumNash equilibrium is a set of bids and budgets where no bidder can increase her by more than by changing her bid or budget.
Throughout the presentation we assume some arbitrary lexicographic ordering on the bidders, that does not necessarily match the subscripts. When we compare two bids and we say that iff either , or but occurs first lexicographically.
We comment that for this problem one is tempted to apply a Fisher Market model: here divisible goods are available to buyers with money , and denotes ’s utility of receiving amount of good . It is known AD (); EG (); DPS () that under certain conditions a vector of prices for goods exists (and can be found efficiently DPSV ()) such that the market clears, in that there is no surplus of goods, and all the money is spent. The natural way to apply a Fisher model to a slot auction is to regard the slots as commodities and have the utilities be in proportion to the number of clicks. However this becomes problematic because there does not seem to be a way to encode the scheduling constraints in the Fisher model; this constraint could make an apparently “marketclearing”Marketclearing equilibrium equilibrium infeasible.
4.1 Special Case: One Slot
In this section we consider the case , where there is only one advertising slot, with some number of clicks. A truthful mechanism for this case is derived by first considering the two extreme cases of infinite bids and infinite budgets.
Suppose all budgets . Then, our input amounts to bids . The obvious mechanism is simply to give all the clicks to the highest bidder. We charge bidder 1 her full price . A simple argument shows that reporting the truth is a weakly dominant strategy for this mechanism. Clearly all bidders will report , since the price is set to if they win. The losing bidders cannot gain from decreasing . The winning bidder can lower her price by lowering , but this will not gain her any more clicks, since she is already getting all of them.
Now suppose all bids ; our input is just a set of budgets , and we need to allocate clicks, with no ceiling on the perclick price. Here we apply a simple rule known as proportional sharingProportional sharing (see Kelly (); JT ()^{11}^{11}11Nguyen and Tardos NT () give a generalization of JT () to general polyhedral constraints, and also discuss the application to sponsored search. Both their bidding language and utility function differ from FMNP (), but it would be interesting to see if there are any connections between their approach and FMNP ().): Let . Now to each bidder , allocate clicks. Set all prices the same: . The mechanism guarantees that each bidder exactly spends her budget, thus no bidder will report . Now suppose some bidder reports , for . Then this bidder is allocated clicks, which is less than , since and all .
Greedy FirstPrice Mechanism. A natural mechanism for the general singleslot case is to solve the associated “fractional knapsack”Fractional knapsack problem problem, and charge bidders their bid; i.e., starting with the highest bidder, greedily add bidders to the allocation, charging them their bid, until all the clicks are allocated. We refer to this as the greedy firstpriceGFP (Greedy FirstPrice) mechanismGreedy firstprice (GFP) mechanism (GFP) mechanism. Though natural (and revenuemaximizing as a function of bids) this is easily seen to be not truthful:
Example 1
FMNP () Suppose there are two bidders and clicks. Bidder 1 has (, ) and bidder 2 has (, ). In the GFP mechanism, if both bidders tell the truth, then bidder 1 gets 50 clicks for each, and 50 of the remaining 70 clicks go to bidder 2 for each. However, if bidder 1 instead declares , then she gets (roughly) 100 clicks, and bidder 2 is left with (roughly) 20 clicks.
The problem here is that the high bidders can get away with bidding lower, thus getting a lower price. The difference between this and the unlimitedbudget case above is that a lower price now results in more clicks. It turns out that in equilibrium, this mechanism will result in an allocation where a prefix of the top bidders are allocated, but their prices equalize to (roughly) the lowest bid in the prefix (as in the example above).
The PriceSetting Mechanism. An equilibrium allocation of GFP can be computed directly via the following mechanism, which FMNP () refers to as the pricesetting (PS) mechanism.Pricesetting (PS) mechanismPS (PriceSetting) mechanism Essentially this is a descending price mechanism: the price stops descending when the bidders willing to pay at that price have enough budget to purchase all the clicks. We have to be careful at the moment a bidder is added to the pool of the willing bidders; if this new bidder has a large enough budget, then suddenly the willing bidders have more than enough budget to pay for all of the clicks. To compensate, the mechanism decreases this “threshold” bidder’s effective budget until the clicks are paid for exactly.
PriceSetting (PS) Mechanism (Single Slot) FMNP () 
Assume wlog that . 
Let be the first bidder such that . Compute price . 
Allocate clicks to each . Allocate clicks to bidder , where . 
Example 2
FMNP () Suppose there are three bidders with , , and , , , and clicks. Running the PS mechanism, we get since , but . The price is set to , and bidders 1 and 2 get 200 and 100 clicks at that price, respectively. There is no threshold bidder.
Example 3
FMNP () Suppose now bidder 2 changes her bid to (everything else remains the same as Example 2). We still get since . But now the price is set to , and bidders 1 and 2 get 250 and 50 clicks at that price, respectively. Note that bidder 2 is now a threshold bidder, does not use her entire budget, and gets fewer clicks.
Theorem 4.1
FMNP () The pricesetting mechanism (single slot) is truthful.
PriceSetting Mechanism Computes Nash Equilibrium of GFP. Consider the greedy firstprice auction in which the highest bidder receives clicks, the second clicks and so on, until the supply of clicks is exhausted. It is immediate that truthfully reporting budgets is a dominant strategy in this mechanism, since when a bidder is considered, her reported budget is exhausted as much as possible, at a fixed price. However, reporting is not a dominant strategy. Nevertheless, it turns out that GFP has an equilibrium whose outcome is (roughly) the same as the PS mechanism. One cannot show that there is a plain Nash equilibrium because of the way ties are resolved lexicographically, so FMNP () proves instead that the bidders reach an Nash equilibrium:
Theorem 4.2
Suppose the PS mechanism is run on the truthful input, resulting in price and clicks for each bidder. Then, for any there is a purestrategy Nash equilibrium of the GFP mechanism where each bidder receives clicks.
4.2 Multiple Slots
Generalizing to multiple slots makes the scheduling constraint nontrivial. Now instead of splitting a pool of clicks arbitrarily, we need to assign clicks that correspond to a feasible schedule of bidders to slots. The conditions under which this is possible add a complexity that needs to be incorporated into the mechanism.
As in the singleslot case it will be instructive to consider first the cases of infinite bids or budgets. Suppose all . In this case, the input consists of bids only . Naturally, what we do here is rank by bid, and allocate the slots to the bidders in that order. Since each budget is infinite, we can always set the prices equal to the bids . By the same logic as in the singleslot case, this is easily seen to be truthful. In the other case, when , there is a lot more work to do.
Without loss of generality, we may assume the number of slots equals the number of bids (i.e., ); if this is not the case, then we add dummy bidders with , or dummy slots with , as appropriate. We keep this assumption for the remainder of the section.
Assigning Slots Using a Classical Scheduling Algorithm. First we give an important lemma that characterizes the conditions under which a set of bidders can be allocated to a set of slots, which turns out to be just a restatement of a classical result HLS () from scheduling theory.
Lemma 3
Proof
In scheduling theory, we say a job with service requirement is a task that needs units of time to complete on a machine with speed . The question of whether there is a feasible allocation is equivalent to the following scheduling problem: Given jobs with service requirements , and machines with speeds , is there a schedule of jobs to machines (with preemption allowed) that completes in one unit of time?
A MultipleSlot BudgetsOnly Mechanism. The mechanism in FMNP () is roughly a descendingprice mechanismDescendingprice mechanism where we decrease the price until a prefix of budgets fits tightly into a prefix of positions at that price, whereupon we allocate that prefix, and continue to decrease the price for the remaining bidders. More formally, it can be written as follows:
PriceSetting Mechanism (Multiple Slots, Budgets Only) FMNP () 
If all , assign bidders to slots arbitrarily and exit. 
Sort the bidders by budget and assume wlog that . 
Define . Set price . 
Let be the largest such that . Allocate slots to bidders at price , using all of their budgets; i.e., . 
Repeat the steps above on the remaining bidders and slots until all slots are allocated. 
Note that the allocation step is always possible since for all , we have , which rewritten is , and so we can apply Lemma 3. An example run of the pricesetting mechanism is shown in Figure 4.
Theorem 4.3
FMNP () The pricesetting mechanism (multislot, budgets only) is truthful.
The PriceSetting Mechanism (General Case). The generalization of the multipleslot PS mechanism to use both bids and budgets combines the ideas from the bidsandbudgets version of the single slot mechanism with the budgetsonly version of the multipleslot mechanism. As our price descends, we maintain a set of “active” bidders with bids at or above this price, as in the singleslot mechanism. These active bidders are kept ranked by budget, and when the price reaches the point where a prefix of bidders fits into a prefix of slots (as in the budgetsonly mechanism) we allocate them and repeat. As in the singleslot case, we must be careful when a bidder enters the active set and suddenly causes an overfit; in this case we again reduce the budget of this “threshold” bidder until it fits. For details on this mechanism and a proof that it is also truthful, we refer the reader to the paper FMNP ().
Greedy FirstPrice Mechanism for Multiple Slots. In the multipleslot case, as in the singleslot case, there is a natural greedy firstprice (GFP)Greedy firstprice (GFP) mechanismGFP (Greedy firstprice) mechanism mechanism when the bidding language includes both bids and budgets: Order the bidders by bid . Starting from the highest bidder, for each bidder compute the maximum possible number of clicks that one could allocate to bidder at price , given the budget constraint and the commitments to previous bidders . This reduces to the “fractional knapsack” problem in the singleslot case, and so one would hope that it maximizes revenue for the given bids and budgets, as in the singleslot case. This is not immediately clear, but does turn out to be true (see FMNP () for details). As in the singleslot case, the GFP mechanism is not a truthful mechanism. However, FMNP () give a generalization of Theorem 4.2 showing that the multipleslot GFP mechanism does have a purestrategy equilibrium, and that equilibrium has prices and allocation equivalent to the multipleslot price setting mechanism.
4.3 Extensions
There are several natural generalizations of the Online Ad Slot Scheduling problem where it would be interesting to extend or apply the results of FMNP ():

Clickthrough rates. To incorporate adspecific clickthrough rates into this model, we would say that a bidder assigned to slot for a time period of length would receive clicks. All the results of FMNP () can be generalized to this setting by simply scaling the bids using . However, now the mechanism does not necessarily prefer more efficient solutions; i.e., ones that generate more overall clicks. It would be interesting to analyze a possible tradeoff between efficiency and revenue in this setting.

Multiple Keywords. To model multiple keywords in this model, we could say that each query had its own set of click totals , and each bidder is interested in a subset of queries. The greedy firstprice mechanism is easily generalized to this case: maximally allocate clicks to bidders in order of their bid (at price ) while respecting the budgets, the query preferences, and the click commitments to previous bidders. It would not be surprising if there was an equilibrium of this extension of the greedy mechanism that could be computed directly with a generalization of the PS mechanism.

Online queries, uncertain supply. In sponsored search, allocations must be made online in response to user queries, and some of the previous literature has focused on this aspect of the problem (e.g., MSVV (); MNS ()). Perhaps the ideas from FMNP () could be used to help make online allocation decisions using (unreliable) estimates of the supply, a setting considered in MNS (), with gametheoretic considerations.
5 The User’s Point of View: a Markov Model for Clicks
In the GSP auction, by fixing the sort order, we leave out an important third party in sponsored search; i.e., the search engine user. Unfortunately, there is very little guidance on this in the literature, even though the user’s behavior is the essential ingredient that defines the commodity the advertisers are bidding on, and its value. In AFMP08 () a different framework is suggested for principled understanding of sponsored search auctions:

Define a suitable probabilistic model for search engine user behavior upon being presented the ads.

Once this model is fixed, ask the traditional mechanism design questions of how do assign the ads to slots, and how to price them.

Analyze the given mechanism from the perspective of the bidders (e.g., strategies) and the search engine (e.g., user satisfaction, efficiency and revenue).
There are certain wellaccepted observations about the user’s interaction with the sponsored search ads that should inform the model:

The higher the ad is on the page, the more clicks it gets.

The “better” the ad is, the more clicks it gets, where the “goodness” of an ad is related to the inherent quality of the ad, and how well it matches the user’s query.
These properties govern not only how the auction is run but also how advertisers think about their bidding strategy (they prefer to appear higher and get more clicks). Thus it is important for an auction to have what we call intuitive bidding: a higher bid translates to a higher position and more clicks.
In AFMP08 (), a natural Markov model is proposed for user clicks, taking the above observations into account. An algorithm is given to determine an optimal assignment of ads to positions in terms of economic efficiency. Together with VCG pricing, this gives a truthful auction. They further show that the optimal assignment under this model has certain monotonicity properties that allow for intuitive bidding. In what follows, we will describe these contributions in more detail.
Modeling the Search Engine User. Previous work on sponsored search has (implicitly) modeled the user using two types of parameters: adspecific clickthrough rates and positionspecific visibility factors . There are some intuitive user behavior models that express overall clickthrough probabilities in terms of these parameters. One possibility is “for each position independently, the user looks at the ad in that position with probability then clicks on the ad with probability .” Alternatively: “The user picks a single position according to the distribution implied by the ’s, and then clicks on the ad in that position with probability .” Under both these models, it follows that the probability of an ad in position receiving a click is equal to , which is the socalled separability assumption (see AGM () or the discussion in Section 2). From separability it follows that GSP ordering of ads will be suitable, because GSP ordering maximizes the total advertiser value on the page.
In both these models there is no reason a priori that the position factors should be decreasing; this is simply imposed because it makes sense, and it is verifiable empirically. Also, both suggested models assume that the probability of an ad getting clicked is independent of other ads that appear with it on the page, an assumption made without much justification. It is hard to imagine that seeing an ad, perhaps followed by a click, has no effect on the subsequent behavior of the user.
In designing a user model, we would like to have the monotonicity of the positions arise naturally. Also, each ad should have parameters dictating their effect on the user both in terms of clicking on that ad, as well as looking at other ads. In AFMP08 (), a model is proposed of a user who starts to scan the list of ads from the top, and makes decisions (about whether to click, continue scanning, or give up altogether) based on what he sees. More specifically, the user is modeled as the following Markov process: “Begin scanning the ads from the top down. When position is reached, click on the ad with probability . Continue scanning with probability .” In this model, if we try to write the click probability of an ad in position as , we get that , where is the set of ads placed above^{12}^{12}12Throughout the section, we will often refer to a position or an ad being “higher” or “above” another position or ad; this means that it is earlier on the list, and is looked at first by the user. position . Thus the “position factor” in the click probability decreases with position, and does so naturally from the model. Also note that we do not have separability anymore, since depends on which ads are above position . Consequently, it can be shown that GSP assignment of ads is no longer the most efficient.
Auction with Markovian users. Given this new user model, we can now ask what the best assignment is of ads to slots. In AFMP08 (), the most efficient assignment is studied; i.e., the one that maximizes total advertiser value derived from user clicks. It turns out that the structure of this assignment is different than that of GSP, and indeed is more sophisticated than any simple ranking. The presence of the ’s requires a delicate tradeoff between the click probability of an ad and its effect on the slots below it. In AFMP08 (), certain structural properties of the optimal assignment are identified and used to find such an optimal assignment efficiently, not only in polynomial time, but in nearlinear time. Given this algorithm, a natural candidate for pricing is VCG V (); C (); G (), which is clearly truthful in this setting, at least under a profitmaximizing utility.
Intuitive Bidding.Intuitive bidding in sponsored search One of the reasons why GSP is successful is perhaps because bidding strategy is intuitive: Under GSP ranking, if an advertiser bids more, they get to a higher position, and consequently, if they bid more, their click probability increases. Now that we have defined a more sophisticated assignment function, even though VCG pricing is truthful, the auction still may not have these intuitive properties. The main technical result in AFMP08 () is to show that in the Markov user model, if a mechanism uses the most efficient assignment, indeed position and click probabilities are monotonic in an ad’s bid (with all other bids fixed), thus preserving this important property. While not surprising, positionmonotonicity turns out to be rather involved to prove, requiring some delicate combinatorial arguments, and insights into the optimal substructure of bidder assignments.
In summary, sponsored search auctions are a three party process which can be studied by modeling the behavior of users first and then designing suitable mechanisms to affect the game theory between the advertiser and the search engine. The work of AFMP08 () sheds some light on the intricate connection between the user models and the mechanisms; for example, the sort order of GSP that is currently popular (sort by ) is not optimal under the Markov user model.
5.1 A Simple Markov User Click Model
We consider a sponsored search auction with bidders and positions. We will also refer to “ad ,” meaning the advertisement submitted by bidder . Each bidder has two parameters, and . The clickthroughrate is the probability that a user will click on ad , given that they look at it. The continuation probability is the probability that a user will look at the next ad in a list, given that they look at ad Markovian user model.
Each bidder submits a bid to the auction, representing the amount that they value a click. The quantity then represents the value of an “impression,” i.e., how much they value a user looking at their ad. This is commonly referred to as their “ecpm.”^{13}^{13}13The acronym ecpm stands for “expected cost per thousand” impressions, where M is the roman numeral for one thousand. We will drop the factor of one thousand and refer to as the “ecpm.”ecpm (Expected Cost Per Thousand [Impressions])Expected cost per thousand [impressions] (ecpm) Throughout, we will use the notation for convenience.
Given an assignment of bidders to the positions, the user looks at the first ad , clicks on it with probability , and then continues looking with probability .^{14}^{14}14The click event and the continuation event could in principle have some correlation, and the results mentioned here will still hold. This is repeated with the second bidder, etc., until the last ad is reached, or some continuation test has failed. Thus the overall expected value of the assignment to the bidders is
Now that we have defined the user model, and characterized the value of an assignment in that model, we can now define a new auction mechanism: First, the search engine computes an assignment of ads to positions that maximizes the overall expected value. Given this assignment, prices can then be computed using VCG V (); C (); G (); for each assigned bidder we compute the change in others’ value if that bidder were to disappear. This assures truthful reporting of bids under a profitmaximizing utility function.
5.2 Properties of Optimal Assignments for Markovian Users
Since the optimal assignment used by the mechanism is no longer simple ranking by ecpm, it is essential to understand the structure of this assignment. This understanding will allow us to compute the assignment more efficiently, and prove some important gametheoretic properties of the mechanism.
It turns out that the quantity , which we will refer to as the “adjusted ecpm (aecpm),”Adjusted ecpm (aecpm)aecpm (Adjusted ecpm) plays a central role in this model. Intuitively, this quantity is the impression value adjusted by the negative effect this ad has on the ads below it. We use for convenience. The following theorem tells us how to assign a set of selected ads to the positions:
Theorem 5.1
AFMP08 () In the most efficient assignment, the ads that are placed are sorted in decreasing order of adjusted ecpm .
While this theorem tells us how to sort the ads selected, it does not tell us which ads to select. One is tempted to say that choosing the top ads by aecpm would do the trick; however the following example proves otherwise:
Example 4
AFMP08 () Suppose we have three bidders and two slots, and the bidders have the following parameters:
Bidder  

1  $1  .75  4 
2  $2  .2  2.5 
3  $0.85  .8  4.25 
Let’s consider some possible assignments and their efficiency. If we use simple ranking by ecpm , we get the assignment , which has efficiency . If we use simple ranking by aecpm we get the assignment with efficiency . It turns out that the optimal assignment is with efficiency . The assigned bidders are ordered by aecpm in the assignment, but are not the top 2 bidders by aecpm.
Now suppose we have the same set of bidders, but now we have three slots. The optimal assignment in this case is ; note how bidder 3 goes from being unassigned to being assigned the first position.
In classical sponsored search with simple ranking, a bidder can dominate another bidder by having higher ecpm; i.e., bidder will always appear whenever does, and in a higher position. Example 4 above shows that having a higher ecpm (or aecpm) does not allow a bidder to dominate another bidder in our new model. However, we show that if she has higher ecpm and aecpm, then this does suffice. This is not only interesting in its own right, it is essential for proving deeper structural properties.
Theorem 5.2
AFMP08 () For all bidders in an optimal assignment, if some bidder is not in the assignment, and and , then we may substitute for , and the assignment is no worse.
The following theorem shows some subset structure between optimal assignments to different numbers of slots. This theorem is used to prove position monotonicity, and is an essential ingredient of the more efficient algorithm for finding the optimal assignment. Let denote the set of all optimal solutions for filling positions with bidders from the set .
Theorem 5.3
AFMP08 () Let be some number of positions, and let be an arbitrary set of bidders. Then, for all , there is some where .
Finally, we state a main technical theorem of AFMP08 (), which shows that bidding is intuitive under a mechanism that maximizes value in the Markovian model.
Theorem 5.4
AFMP08 () With all other bids fixed, the probability of receiving a click in the optimal solution is nondecreasing in one’s bid. In addition, the position of a particular bidder in the optimal solution is monotonic that bidder’s bid.
This theorem, whose proof relies on all the previous results in this section, implies that from the perspective of a bidder participating in the auction, all the complexities of the underlying assignment still do not interfere with the intuitive nature of bidding; if you bid more, you still get more clicks, and get to a higher position.
5.3 Computing the Optimal Assignment
A simple algorithm for computing the optimal assignment proceeds as follows. First, sort the ads in decreasing order of aecpm in time . Then, let be the efficiency obtained (given that you reach slot ) by filling slots with bidders from the set . We get the following recurrence:
Solving this recurrence for yields the optimal assignment, and can be done in time. Using the properties about optimal assignments proved in the previous section, this can be improved to
Theorem 5.5
AFMP08 () Consider the auction with Markovian bidders and slots. There is an optimal assignment which can be determined in time.
6 Open Issues
We emphasize three open directions, besides the various gametheoretic and algorithmic open problems already proposed so far.

Estimating Parameters. In order to run the basic auction and its extensions, the search companies need to estimate a number of parameters: ctr, positionspecific factors, minimum bidderspecific reserve prices, etc. In addition, for operational reasons, search engines have to provide traffic estimates to potential advertisers, that is, for each keyword, they need to show landscape functions such as the ones in Section 3. An open research problem is, given a log of search and ad traffic over a significant period of time, design and validate efficient learning methods for estimating these parameters, and perhaps, even identify the models that fit the variation of these parameters over time. This is a significant research challenge since these parameters have intricate dependencies, and in addition, there is a long tail effect in the logs, that is, there is a significant amount of rare queries and keywords, as well as rare clicks for particular keywords.

Grand Simulation. In order for the academic world to develop intuition into the world of sponsored search auctions and the associated dynamics, we need a grand simulation platform that can generate search traffic, ad inventories, ad clicks, and market specifics at the “Internet scale” that search engines face. Such a platform will help us understand the many tradeoffs: increasing keywords vs increasing budget for a campaign, making better bids vs choosing different search engines, choosing to bid for impressions vs clicks vs action, etc. Some auction programs are currently available^{15}^{15}15For example, see http://www.hss.caltech.edu/~jkg/jAuctions.html, but a systematic, large scale effort by academia will have tremendous impact for research.

Grand Models. In general, we need more detailed models for the behavior of users, advertisers as well as the impact of the search engine design on them. We described a highly preliminary effort here in which the users were Markovian, but more powerful models will also be of great interest. For example, a small extension is to make the continuation probability a function of location as well, which makes the optimization problem more difficult. One can also generalize the Markov model to handle arbitrary configurations of ads on a web page (not necessarily the linear order in current search results page), or to allow various other user states (such as navigating a landing page,Landing page in sponsored search that is the page that is the target of an ad). Finally, since page layout can be performed dynamically, we could ask what would happen if the layout of a web page were a part of the model, which would combine both users as well as the search engine into a model. In general, there may be grand, unified models that capture the relationship between all the three parties in sponsored search.
7 Concluding Remarks
We have discussed algorithmic and gametheoretic issues in auctions for sponsored search.
Auctions are used for other products in Internet advertising, for example Google’s AdSense,GoogleAdSense (Google) where an Internet publisher (like an online newspaper) can sign up with an ad network (in this case Google) to place ads on their site. Here, an additional aspect of the problem from the auctioneer’s perspective is how to target ads, that is, how to choose the keywords from the surrounding context, to run auctions like the ones we discussed thus far. This also introduces the fourth player in the game, i.e., the publisher, and consequently, the game theory is more intricate and largely unexplored.
Internet ads like sponsored search or AdSense may combine different types of ads, i.e., text, image or video ads. Each has its own specifications in terms of dimensions, user engagement and effectiveness. How to combine them into a unified auction is an interesting challenge.
Beyond Internet ads, the Internet medium is also used for enabling ads in traditional media including TV, Radio, Print etc. In such cases, the auction problems may take on a richer combinatorial component, and also, a component based on ability to reserve ad slots ahead of time. The resulting algorithmic and gametheoretic problems are largely unexplored in the research community.
8 Acknowledgements
We gratefully thank numerous engineers and researchers at Google.
Bibliography
 [1] Z. Abrams and A. Ghosh. Auctions with revenue guarantees for sponsored search. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [2] Z. Abrams, O. Mendelevitch, and J. Tomlin. Optimal delivery of sponsored search advertisements subject to budget constraints. In ACM Conference on Electronic Commerce, 2007.
 [3] G. Aggarwal, J. Feldman, and S. Muthukrishnan. Bidding to the top: VCG and equilibria of positionbased auctions. In Proc. Workshop on Approximation and Online Algorithms (WAOA), 2006.
 [4] G. Aggarwal, J. Feldman, S. Muthukrishnan, and M. Pál. Sponsored search auctions with Markovian users, 2008. Submitted.
 [5] G. Aggarwal, A. Goel, and R. Motwani. Truthful auctions for pricing search keywords. In ACM Conference on Electronic Commerce (EC), 2006.
 [6] K. Arrow and G. Debreu. Existence of an equilibrium for a competitive economy. Econometrica, 22:265–290, 1954.
 [7] C. Borgs, J. Chayes, O. Etesami, N. Immorlica, K. Jain, and M. Mahdian. Dynamics of bid optimization in online advertisement auctions. In Proc. WWW, 2007.
 [8] C. Borgs, J. T. Chayes, N. Immorlica, M. Mahdian, and A. Saberi. Multiunit auctions with budgetconstrained bidders. In ACM Conference on Electronic Commerce (EC), 2005.
 [9] M. Cary, A. Das, B. Edelman, I. Giotis, K. Heimerl, A. R. Karlin, C. Mathieu, and M. Schwarz. Greedy bidding strategies for keyword auctions. In ACM Conference on Electronic Commerce, 2007.
 [10] E. Clarke. Multipart pricing of public goods. Public Choice, 11:17–33, 1971.
 [11] X. Deng, C. Papadimitriou, and S. Safra. On the complexity of equilibria. In Proceedings of ACM Symposium on Theory of Computing., 2002.
 [12] N. Devanur, C. Papadimitriou, A. Saberi, and V. Vazirani. Market equilibrium via a primaldualtype algorithm, 2002.
 [13] B. Edelman, M. Ostrovsky, and M. Schwarz. Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. In Second workshop on sponsored search auctions, 2006.
 [14] E. Eisenberg and D. Gale. Consensus of subjective probabilities: The parimutuel method. Annals Of Mathematical Statistics, 30:165–168, 1959.
 [15] E. EvenDar, J. Feldman, Y. Mansour, and S. Muthukrishnan. Position auctions with bidderspecific minimum prices, 2008. Submitted.
 [16] E. EvenDar, S. Mannor, and Y. Mansour. Pac bounds for multiarmed bandit and markov decision processes. In COLT ’02: Proceedings of the 15th Annual Conference on Computational Learning Theory, pages 255–270, London, UK, 2002. SpringerVerlag.
 [17] J. Feldman, S. Muthukrishnan, M. Pál, and C. Stein. Budget optimization in searchbased advertising auctions. In ACM Conference on Electronic Commerce, 2007.
 [18] J. Feldman, E. Nikolova, S. Muthukrishnan, and M. Pál. A truthful mechanism for offline ad slot scheduling. In Proc. Symposium on Algorithmic Game Theory (SAGT), 2008. To appear.
 [19] G. Goel and A. Mehta. Adwords auctions with decreasing valuation bids. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [20] G. Goel and A. Mehta. Online budgeted matching in random input models with applications to adwords. In Proc. Symposium on Discrete Algorithms (SODA), 2008.
 [21] Rica Gonen and Elan Pavlov. An adaptive sponsored search mechanism deltagain truthful in valuation, time, and budget. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [22] T. Gonzalez and S. Sahni. Preemptive scheduling of uniform processing systems. Journal of the ACM, 25:92–101, January 1978.
 [23] T. Groves. Incentives in teams. Econometrica, 41(4):617–631, 1973.
 [24] E. C. Horvath, S. Lam, and R. Sethi. A level algorithm for preemptive scheduling. J. ACM, 24(1):32–43, 1977.
 [25] R. Johari and J.N. Tsitsiklis. Efficiency loss in a network resource allocation game. Mathematics of Operations Research, 29(3):407–435, 2004.
 [26] F. Kelly. Charging and rate control for elastic traffic. European Transactions on Telecommunications, 8:33–37, January 1997.
 [27] W. Labio, M. Rose, and S. Ramaswamy, May 2004. Internal Document, Google, Inc.
 [28] S. Lahaie. An analysis of alternative slot auction designs for sponsored search. In ACM Conference on Electronic Commerce (EC), 2006.
 [29] S. Lahaie, D. Pennock, A. Saberi, and R. Vohra. Sponsored Search Auctions, chapter 28. Algorithmic Game Theory. Cambridge University Press, 2007.
 [30] L. Liang and Q. Qi. Cooperative or vindictive: Bidding strategies in sponsored search auction. In WINE, pages 167–178, 2007.
 [31] E. Dominowska M. Richardson and R. Ragno. Predicting clicks: Estimating the clickthrough rates for new ads. In WWW 2007, Track: Search, 2007.
 [32] M. Mahdian, H. Nazerzadeh, and A. Saberi. Allocating online advertisement space with unreliable estimates. In ACM Conference on Electronic Commerce, 2007.
 [33] M. Mahdian and A. Saberi. Multiunit auctions with unknown supply. In ACM conference on Electronic commerce (EC), 2006.
 [34] A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani. AdWords and generalized online matching. In FOCS, 2005.
 [35] S. Muthukrishnan, M. Pál, and Z. Svitkina. Stochastic models for budget optimization in searchbased advertising. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [36] M. Mahdian N. Immorlica, K. Jain and K. Talwar. Click fraud resistant methods for learning clickthrough rates. In Workshop on Internet and Network Economics (WINE), 2005.
 [37] T. Nguyen and E. Tardos. Approximately maximizing efficiency and revenue in polyhedral environments. In ACM Conference on Electronic Commerce (EC), 2007.
 [38] Nielsen//NetRatings. Interactive advertising bureau (IAB) search branding study, August 2004. Commissioned by the IAB Search Engine Committee. Available at http://www.iab.net/resources/iab_searchbrand.asp.
 [39] P. Rusmevichientong and D. Williamson. An adaptive algorithm for selecting profitable keywords for searchbased advertising services. In Proc. 7th ACM conference on Electronic commerce (EC), 2006.
 [40] B. K. Szymanski and J.S. Lee. Impact of roi on bidding and revenue in sponsored search advertisement auctions. In 2nd Workshop on Sponsored Search Auctions in ACM Conference on Electronic Commerce (EC), 2006.
 [41] H. Varian. Position auctions. International Journal of Industrial Organization, 25(6):1163–1178, December 2007.
 [42] W. Vickrey. Counterspeculation, auctions and competitivesealed tenders. Finance, 16(1):8–37, 1961.
 [43] Y. Vorobeychik and D. M. Reeves. Equilibrium analysis of dynamic bidding in sponsored search auctions. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [44] J. Wortman, Y. Vorobeychik, L. Li, and J. Langford. Maintaining equilibria during exploration in sponsored search auctions. In Proc. Workshop on Internet and Network Economics (WINE), 2007.
 [45] Y. Zhou and R. Lukose. Vindictive bidding in keyword auctions. In ICEC ’07: Proceedings of the ninth international conference on Electronic commerce, pages 141–146, New York, NY, USA, 2007. ACM.