Revenue Optimization in Posted-Price Auctions with Strategic Buyers

# Revenue Optimization in Posted-Price Auctions with Strategic Buyers

Mehryar Mohri
251 Mercer Street
New York, NY 10012
mohri@cims.nyu.edu &Andres Muñoz Medina
Courant Institute
251 Mercer Street
New York, NY 10012
munoz@cims.nyu.edu
###### Abstract

We study revenue optimization learning algorithms for posted-price auctions with strategic buyers. We analyze a very broad family of monotone regret minimization algorithms for this problem, which includes the previously best known algorithm, and show that no algorithm in that family admits a strategic regret more favorable than . We then introduce a new algorithm that achieves a strategic regret differing from the lower bound only by a factor in , an exponential improvement upon the previous best algorithm. Our new algorithm admits a natural analysis and simpler proofs, and the ideas behind its design are general. We also report the results of empirical evaluations comparing our algorithm with the previous state of the art and show a consistent exponential improvement in several different scenarios.

Revenue Optimization in Posted-Price Auctions with Strategic Buyers

Mehryar Mohri Courant Institute and Google Research 251 Mercer Street New York, NY 10012 mohri@cims.nyu.edu Andres Muñoz Medina Courant Institute 251 Mercer Street New York, NY 10012 munoz@cims.nyu.edu

## 1 Introduction

Auctions have long been an active area of research in Economics and Game Theory (Vickrey, 2012; Milgrom and Weber, 1982; Ostrovsky and Schwarz, 2011). In the past decade, however, the advent of online advertisement has prompted a more algorithmic study of auctions, including the design of learning algorithms for revenue maximization for generalized second-price auctions or second-price auctions with reserve (Cesa-Bianchi et al., 2013; Mohri and Muñoz Medina, 2014; He et al., 2013).

The scenario we consider is that of posted-price auctions, which, albeit simpler than other mechanisms, in fact matches a common situation in AdExchanges where many auctions admit a single bidder. In this setting, second-price auctions with reserve are equivalent to posted-price auctions: a seller sets a reserve price for a good and the buyer decides whether or not to accept it (that is to bid higher than the reserve price). In order to capture the buyer’s strategic behavior, we will analyze an online scenario: at each time , a price is offered by the seller and the buyer must decide to either accept it or leave it. This scenario can be modeled as a two-player repeated non-zero sum game with incomplete information, where the seller’s objective is to maximize his revenue, while the advertiser seeks to maximize her surplus as described in more detail in Section 2.

The literature on non-zero sum games is very rich (Nachbar, 1997, 2001; Morris, 1994), but much of the work in that area has focused on characterizing different types of equilibria, which is not directly relevant to the algorithmic questions arising here. Furthermore, the problem we consider admits a particular structure that can be exploited to design efficient revenue optimization algorithms.

We will adopt instead the definition of strategic-regret, which was introduced by Amin et al. (2013) precisely for the study of this problem. This notion of regret also matches the concept of learning loss introduced by (Agrawal, 1995) when facing an oblivious adversary. Using this definition, Amin et al. (2013) presented both upper and lower bounds for the regret of a seller facing a strategic buyer and showed that the buyer’s surplus must be discounted over time in order to be able to achieve sublinear regret (see Section 2). However, the gap between the upper and lower bounds they presented is in . In the following, we analyze a very broad family of monotone regret minimization algorithms for this problem (Section 3), which includes the algorithm of Amin et al. (2013), and show that no algorithm in that family admits a strategic regret more favorable than . Next, we introduce a nearly-optimal algorithm that achieves a strategic regret differing from the lower bound at most by a factor in (Section 4). This represents an exponential improvement upon the existing best algorithm for this setting. Our new algorithm admits a natural analysis and simpler proofs. A key idea behind its design is a method deterring the buyer from lying, that is rejecting prices below her valuation.

## 2 Setup

For any , define the discounted surplus of the buyer as follows:

 Sur(A,v)=T∑t=1γt−1at(v−pt). (1)

The value of the discount factor indicates the strength of the preference of the buyer for current surpluses versus future ones. The performance of a seller’s algorithm is measured by the notion of strategic-regret (Amin et al., 2013) defined as follows:

 Reg(A,v)=Tv−T∑t=1atpt. (2)

The buyer’s objective is to maximize his discounted surplus, while the seller seeks to minimize his regret. Note that, in view of the discounting factor , the buyer is not fully adversarial. The problem consists of designing algorithms achieving sublinear strategic regret (that is a regret in ).

The motivation behind the definition of strategic-regret is straightforward: a seller, with access to the buyer’s valuation, can set a fixed price for the good close to this value. The buyer, having no control on the prices offered, has no option but to accept this price in order to optimize his utility. The revenue per round of the seller is therefore . Since there is no scenario where higher revenue can be achieved, this is a natural setting to compare the performance of our algorithm.

To gain more intuition about the problem, let us examine some of the complications arising when dealing with a strategic buyer. Suppose the seller attempts to learn the buyer’s valuation by performing a binary search. This would be a natural algorithm when facing a truthful buyer. However, in view of the buyer’s knowledge of the algorithm, for , it is in her best interest to lie on the initial rounds, thereby quickly, in fact exponentially, decreasing the price offered by the seller. The seller would then incur an regret. A binary search approach is therefore “too aggressive”. Indeed, an untruthful buyer can manipulate the seller into offering prices less than by lying about her value even just once! This discussion suggests following a more conservative approach. In the next section, we discuss a natural family of conservative algorithms for this problem.

## 3 Monotone algorithms

The following conservative pricing strategy was introduced by Amin et al. (2013). Let and . If price is rejected at round , the lower price is offered at the next round. If at any time price is accepted, then this price is offered for all the remaining rounds. We will denote this algorithm by monotone. The motivation behind its design is clear: for a suitable choice of , the seller can slowly decrease the prices offered, thereby pressing the buyer to reject many prices (which is not convenient for her) before obtaining a favorable price. The authors present an regret bound for this algorithm, with . A more careful analysis shows that this bound can be further tightened to when the discount factor is known to the seller.

Despite its sublinear regret, the monotone algorithm remains sub-optimal for certain choices of . Indeed, consider a scenario with . For this setting, the buyer would no longer have an incentive to lie, thus, an algorithm such as binary search would achieve logarithmic regret, while the regret achieved by the monotone algorithm is only guaranteed to be in .

One may argue that the monotone algorithm is too specific since it admits a single parameter and that perhaps a more complex algorithm with the same monotonic idea could achieve a more favorable regret. Let us therefore analyze a generic monotone algorithm defined by Algorithm 1.

###### Definition 1.

For any buyer’s valuation , define the acceptance time as the first time a price offered by the seller using algorithm is accepted.

###### Proposition 1.

For any decreasing sequence of prices , there exists a truthful buyer with valuation such that algorithm suffers regret of at least

 Reg(Am,v0)≥14√T−√T.
###### Proof.

By definition of the regret, we have . We can consider two cases: for some and for every . In the former case, we have , which implies the statement of the proposition. Thus, we can assume the latter condition.

Let be uniformly distributed over . In view of Lemma 4 (see Appendix A.1), we have

 E[vκ∗]+E[(T−κ∗)(v−pκ∗)]≥12E[κ∗]+(T−√T)E[(v−pκ∗)]≥12E[κ∗]+T−√T32E[κ∗].

The right-hand side is minimized for . Plugging in this value yields , which implies the existence of with . ∎

We have thus shown that any monotone algorithm suffers a regret of at least , even when facing a truthful buyer. A tighter lower bound can be given under a mild condition on the prices offered.

###### Definition 2.

A sequence is said to be convex if it verifies for .

An instance of a convex sequence is given by the prices offered by the monotone algorithm. A seller offering prices forming a decreasing convex sequence seeks to control the number of lies of the buyer by slowly reducing prices. The following proposition gives a lower bound on the regret of any algorithm in this family.

###### Proposition 2.

Let be a decreasing convex sequence of prices. There exists a valuation for the buyer such that the regret of the monotone algorithm defined by these prices is , where .

The full proof of this proposition is given in Appendix A.1. The proposition shows that when the discount factor is known, the monotone algorithm is in fact asymptotically optimal in its class.

The results just presented suggest that the dependency on cannot be improved by any monotone algorithm. In some sense, this family of algorithms is “too conservative”. Thus, to achieve a more favorable regret guarantee, an entirely different algorithmic idea must be introduced. In the next section, we describe a new algorithm that achieves a substantially more advantageous strategic regret by combining the fast convergence properties of a binary search-type algorithm (in a truthful setting) with a method penalizing untruthful behaviors of the buyer.

## 4 A nearly optimal algorithm

Let be an algorithm for revenue optimization used against a truthful buyer. Denote by the tree associated to after rounds. That is, is a full tree of height with nodes labeled with the prices offered by . The right and left children of are denoted by and respectively. The price offered when is accepted by the buyer is the label of while the price offered by if is rejected is the label of . Finally, we will denote the left and right subtrees rooted at node by and respectively. Figure 1 depicts the tree generated by an algorithm proposed by Kleinberg and Leighton (2003), which we will describe later.

Since the buyer holds a fixed valuation, we will consider algorithms that increase prices only after a price is accepted and decrease it only after a rejection. This is formalized in the following definition.

###### Definition 3.

An algorithm is said to be consistent if for any node .

For any consistent algorithm , we define a modified algorithm , parametrized by an integer , designed to face strategic buyers. Algorithm offers the same prices as , but it is defined with the following modification: when a price is rejected by the buyer, the seller offers the same price for rounds. The pseudocode of is given in Algorithm 2. The motivation behind the modified algorithm is given by the following simple observation: a strategic buyer will lie only if she is certain that rejecting a price will boost her surplus in the future. By forcing the buyer to reject a price for several rounds, the seller ensures that the future discounted surplus will be negligible, thereby coercing the buyer to be truthful.

We proceed to formally analyze algorithm . In particular, we will quantify the effect of the parameter on the choice of the buyer’s strategy. To do so, a measure of the spread of the prices offered by is needed.

###### Definition 4.

For any node define the right increment of as . Similarly, define its left increment to be .

The prices offered by define a path in . For each node in this path, we can define time to be the number of rounds needed for this node to be reached by . Note that, since may be greater than , the path chosen by might not necessarily reach the leaves of . Finally, let be the function representing the surplus obtained by the buyer when playing an optimal strategy against after node is reached.

###### Lemma 1.

The function satisfies the following recursive relation:

 S(n)=max(γt(n)−1(v−pn)+S(r(n)),S(l(n))). (3)
###### Proof.

Define a weighted tree of nodes reachable by algorithm . We assign weights to the edges in the following way: if an edge on is of the form , its weight is set to be , otherwise, it is set to . It is easy to see that the function evaluates the weight of the longest path from node to the leafs of . It thus follows from elementary graph algorithms that equation (3) holds. ∎

The previous lemma immediately gives us necessary conditions for a buyer to reject a price.

###### Proposition 3.

For any reachable node , if price is rejected by the buyer, then the following inequality holds:

 v−pn<γr(1−γ)(1−γr)(δln+γδrn).
###### Proof.

A direct implication of Lemma 1 is that price will be rejected by the buyer if and only if

 γt(n)−1(v−pn)+S(r(n))

However, by definition, the buyer’s surplus obtained by following any path in is bounded above by . In particular, this is true for the path which rejects and accepts every price afterwards. The surplus of this path is given by where are the prices the seller would offer if price were rejected. Furthermore, since algorithm is consistent, we must have . Therefore, can be bounded as follows:

 S(r(n))≥T∑t=t(n)+r+1γt−1(v−pn−δrn)=γt(n)+r−γT1−γ(v−pn−δrn). (5)

We proceed to upper bound . Since for all , and

 S(l(n))≤T∑t=tn+rγt−1(v−pn+δln)=γt(n)+r−1−γT1−γ(v−pn+δln). (6)

Combining inequalities (4), (5) and (6) we conclude that

 γt(n)−1(v−pn)+γt(n)+r−γT1−γ(v−pn−δrn) ≤γt(n)+r−1−γT1−γ(v−pn+δln) ⇒(v−pn)(1+γr+1−γr1−γ) ≤γrδln+γr+1δrn−γT−t(n)+1(δrn+δln)1−γ ⇒(v−pn)(1−γr) ≤γr(δln+γδrn)1−γ.

Rearranging the terms in the above inequality yields the desired result. ∎

Let us consider the following instantiation of algorithm introduced in (Kleinberg and Leighton, 2003). The algorithm keeps track of a feasible interval initialized to and an increment parameter initialized to . The algorithm works in phases. Within each phase, it offers prices until a price is rejected. If price is rejected, then a new phase starts with the feasible interval set to and the increment parameter set to . This process continues until at which point the last phase starts and price is offered for the remaining rounds. It is not hard to see that the number of phases needed by the algorithm is less than . A more surprising fact is that this algorithm has been shown to achieve regret when the seller faces a truthful buyer. We will show that the modification of this algorithm admits a particularly favorable regret bound. We will call this algorithm (penalized fast search algorithm).

###### Proposition 4.

For any value of and any , the regret of algorithm admits the following upper bound:

 Reg(PFSr,v)≤(vr+1)(⌈log2log2T⌉+1)+(1+γ)γrT2(1−γ)(1−γr). (7)

Note that for and the upper bound coincides with that of (Kleinberg and Leighton, 2003).

###### Proof.

Algorithm can accumulate regret in two ways: the price offered is rejected, in which case the regret is , or the price is accepted and its regret is .

Let be the number of phases run by algorithm . Since at most different prices are rejected by the buyer (one rejection per phase) and each price must be rejected for rounds, the cumulative regret of all rejections is upper bounded by .

The second type of regret can also be bounded straightforwardly. For any phase , let and denote the corresponding search parameter and feasible interval respectively. If , the regret accrued in the case where the buyer accepts a price in this interval is bounded by . If, on the other hand , then it readily follows that for all prices offered in phase . Therefore, the regret obtained in acceptance rounds is bounded by

 K∑i=1Ni((v−bi)\mathds1v>bi+√ϵi)≤K∑i=1(v−bi)\mathds1v>biNi+K,

where denotes the number of prices offered during the -th round.

Finally, notice that, in view of the algorithm’s definition, every corresponds to a rejected price. Thus, by Proposition 3, there exist nodes (not necessarily distinct) such that and

 v−bi=v−pni≤γr(1−γ)(1−γr)(δlni+γδrni).

It is immediate that and for any node , thus, we can write

 K∑i=1(v−bi)\mathds1v>biNi ≤γr(1+γ)2(1−γ)(1−γr)K∑i=1Ni≤γr(1+γ)2(1−γ)(1−γr)T.

The last inequality holds since at most prices are offered by our algorithm. Combining the bounds for both regret types yields the result. ∎

When an upper bound on the discount factor is known to the seller, he can leverage this information and optimize upper bound (7) with respect to the parameter .

###### Theorem 1.

Let and For any , if , the regret of satisfies

 Reg(PFSr∗,v)≤(2vγ0Tγ0logcT+1+v)(log2log2T+1)+4Tγ0,

where .

The proof of this theorem is fairly technical and is deferred to the Appendix. The theorem helps us define conditions under which logarithmic regret can be achieved. Indeed, if , using the inequality valid for all we obtain

 11−γ0≤log2T2logT−1≤logT.

It then follows from Theorem 1 that

 Reg(PFSr∗,v)≤(2vlogTlogcT+1+v)(log2log2T+1)+4logT.

Let us compare the regret bound given by Theorem 1 with the one given by Amin et al. (2013). The above discussion shows that for certain values of , an exponentially better regret can be achieved by our algorithm. It can be argued that the knowledge of an upper bound on is required, whereas this is not needed for the monotone algorithm. However, if , the regret bound on monotone is super-linear, and therefore uninformative. Thus, in order to properly compare both algorithms, we may assume that in which case, by Theorem 1, the regret of our algorithm is whereas only linear regret can be guaranteed by the monotone algorithm. Even under the more favorable bound of , for any and , the monotone algorithm will achieve regret while a strictly better regret is attained by ours.

## 5 Lower bound

The following lower bounds have been derived in previous work.

###### Theorem 2 ((Amin et al., 2013)).

Let be fixed. For any algorithm , there exists a valuation for the buyer such that .

This theorem is in fact given for the stochastic setting where the buyer’s valuation is a random variable taken from some fixed distribution . However, the proof of the theorem selects to be a point mass, therefore reducing the scenario to a fixed priced setting.

###### Theorem 3 ( (Kleinberg and Leighton, 2003)).

Given any algorithm to be played against a truthful buyer, there exists a value such that for some universal constant .

Combining these results leads immediately to the following.

###### Corollary 1.

Given any algorithm , there exists a buyer’s valuation such that , for a universal constant .

We now compare the upper bounds given in the previous section with the bound of Corollary 1. For , we have . On the other hand, for , we may choose , in which case, by Proposition 4, . Thus, the upper and lower bounds match up to an factor.

## 6 Empirical results

In this section, we present the result of simulations comparing the monotone algorithm and our algorithm . The experiments were carried out as follows: given a buyer’s valuation , a discrete set of false valuations were selected out of the set . Both algorithms were run against a buyer making the seller believe her valuation is instead of . The value of achieving the best utility for the buyer was chosen and the regret for both algorithms is reported in Figure 2.

We considered two sets of experiments. First, the value of parameter was left unknown to both algorithms and the value of was set to . This choice is motivated by the discussion following Theorem 1 since, for large values of , we can expect to achieve logarithmic regret. The first two plots (from left to right) in Figure 2 depict these results. The apparent stationarity in the regret of is just a consequence of the scale of the plots as the regret is in fact growing as . For the second set of experiments, we allowed access to the parameter to both algorithms. The value of was chosen optimally based on the results of Theorem 1 and the parameter of monotone was set to to ensure regret in . It is worth noting that even though our algorithm was designed under the assumption of some knowledge about the value of , the experimental results show that an exponentially better performance over the monotone algorithm is still attainable and in fact the performances of the optimized and unoptimized versions of our algorithm are comparable. A more comprehensive series of experiments is presented in Appendix B.

## 7 Conclusion

We presented a detailed analysis of revenue optimization algorithms against strategic buyers. In doing so, we reduced the gap between upper and lower bounds on strategic regret to a logarithmic factor. Furthermore, the algorithm we presented is simple to analyze and reduces to the truthful scenario in the limit of , an important property that previous algorithms did not admit. We believe that our analysis helps gain a deeper understanding of this problem and that it can serve as a tool for studying more complex scenarios such as that of strategic behavior in repeated second-price auctions, VCG auctions and general market strategies.

## Acknowledgments

We thank Kareem Amin, Afshin Rostamizadeh and Umar Syed for several discussions about the topic of this paper. This work was partly funded by the NSF award IIS-1117591.

## References

• Agrawal [1995] R. Agrawal. The continuum-armed bandit problem. SIAM journal on control and optimization, 33(6):1926–1951, 1995.
• Amin et al. [2013] K. Amin, A. Rostamizadeh, and U. Syed. Learning prices for repeated auctions with strategic buyers. In Proceedings of NIPS, pages 1169–1177, 2013.
• Arora et al. [2012] R. Arora, O. Dekel, and A. Tewari. Online bandit learning against an adaptive adversary: from regret to policy regret. In Proceedings of ICML, 2012.
• Auer et al. [2002a] P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235–256, 2002a.
• Auer et al. [2002b] P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48–77, 2002b.
• Cesa-Bianchi et al. [2013] N. Cesa-Bianchi, C. Gentile, and Y. Mansour. Regret minimization for reserve prices in second-price auctions. In Proceedings of SODA, pages 1190–1204, 2013.
• Edelman and Ostrovsky [2007] B. Edelman and M. Ostrovsky. Strategic bidder behavior in sponsored search auctions. Decision Support Systems, 43(1), 2007.
• He et al. [2013] D. He, W. Chen, L. Wang, and T. Liu. A game-theoretic machine learning approach for revenue maximization in sponsored search. In Proceedings of IJCAI, pages 206–213, 2013.
• Kleinberg and Leighton [2003] R. D. Kleinberg and F. T. Leighton. The value of knowing a demand curve: Bounds on regret for online posted-price auctions. In Proceedings of FOCS, pages 594–605, 2003.
• Kuleshov and Precup [2010] V. Kuleshov and D. Precup. Algorithms for the multi-armed bandit problem. Journal of Machine Learning, 2010.
• Milgrom and Weber [1982] P. Milgrom and R. Weber. A theory of auctions and competitive bidding. Econometrica: Journal of the Econometric Society, pages 1089–1122, 1982.
• Mohri and Muñoz Medina [2014] M. Mohri and A. Muñoz Medina. Learning theory and algorithms for revenue optimization in second-price auctions with reserve. In Proceedings of ICML, 2014.
• Morris [1994] P. Morris. Non-zero-sum games. In Introduction to Game Theory, pages 115–147. Springer, 1994.
• Nachbar [2001] J. Nachbar. Bayesian learning in repeated games of incomplete information. Social Choice and Welfare, 18(2):303–326, 2001.
• Nachbar [1997] J. H. Nachbar. Prediction, optimization, and learning in repeated games. Econometrica: Journal of the Econometric Society, pages 275–309, 1997.
• Ostrovsky and Schwarz [2011] M. Ostrovsky and M. Schwarz. Reserve prices in internet advertising auctions: A field experiment. In Proceedings of EC, pages 59–60. ACM, 2011.
• Robbins [1985] H. Robbins. Some aspects of the sequential design of experiments. In Herbert Robbins Selected Papers, pages 169–177. Springer, 1985.
• Vickrey [2012] W. Vickrey. Counterspeculation, auctions, and competitive sealed tenders. The Journal of finance, 16(1):8–37, 2012.

## Appendix A Appendix

###### Lemma 2.

The function is decreasing over the interval .

###### Proof.

This can be straightforwardly established:

 g′(γ)=−1−γγ+log1γ(1−γ)2=γlog(1−[1−1γ])−(1−γ)γ(1−γ)2<(1−γ)−(1−γ)γ(1−γ)2=0,

using the inequality valid for all . ∎

###### Lemma 3.

Let and let be a decreasing and differentiable function. Then, the function defined by

 F(γ)=g(γ)−√g(γ)2−b

is increasing for all values of .

###### Proof.

We will show that for all . Since and by hypothesis, the previous statement is equivalent to showing that which is trivially verified since . ∎

###### Theorem 1.

Let and For any , if , the regret of satisfies

 Reg(PFSr∗,v)≤(2vγ0Tγ0logcT+1+v)(log2log2T+1)+4Tγ0,

where .

###### Proof.

It is not hard to verify that the function is convex and approaches infinity as . Thus, it admits a minimizer whose explicit expression can be found by solving the following equation

 0=ddr(r+γr0T(1−γ0)(1−γr0))=1+γr0Tlogγ0(1−γ0)(1−γr0)2.

Solving the corresponding second-degree equation yields

 γ¯r∗0=2+Tlog(1/γ0)1−γ0−√(2+Tlog(1/γ0)1−γ0)2−42=:F(γ0).

By Lemmas 2 and 3, the function thereby defined is increasing. Therefore, and

 γ¯r∗0≤2+T−√(2+T)2−42=42(2+T+√(2+T)2−4)≤2T. (8)

By the same argument, we must have , that is

 γ¯r∗0≥F(1/2) =2+2Tlog2−√(2+2Tlog2)2−42 =42(2+2Tlog2+√(2+2Tlog2)2−4) ≥24+4Tlog2≥14Tlog2.

Thus,

 r∗=⌈¯r∗⌉≤log(1/F(1/2))log(1/γ0)+1≤log(4Tlog2)log1/γ0+1. (9)

Combining inequalities (8) and (9) with (7) gives

 Reg(PFSr∗,v) ≤(vlog(4Tlog2)log1/γ0+1+v)(⌈log2log2T⌉+1)+(1+γ0)T(1−γ0)(T−2) ≤(2vγ0Tγ0log(cT)+1+v)(⌈log2log2T⌉+1)+4Tγ0,

using the inequality valid for all . ∎

### a.1 Lower bound for monotone algorithms

###### Lemma 4.

Let be a decreasing sequence of prices. Assume that the seller faces a truthful buyer. Then, if is sampled uniformly at random in the interval , the following inequality holds:

 E[κ∗]≥132E[v−pκ∗].
###### Proof.

Since the buyer is truthful, if and only if . Thus, we can write

 E[v−pκ∗]=κmax∑κ=2E[\mathds1v∈[pκ,pκ−1](v−pκ)]=κmax∑κ=2∫pκ−1pκ(v−pκ)dv=κmax∑κ=2(pκ−1−pκ)22,

where . Thus, by the Cauchy-Schwarz inequality, we can write

 E[κ∗∑κ=2pκ−1−pκ] ≤E⎡⎢⎣ ⎷κ∗κ∗∑κ=2(pκ−1−pκ)2⎤⎥⎦ ≤E⎡⎣ ⎷κ∗κmax∑κ=2(pκ−1−pκ)2⎤⎦ =E[√2κ∗E[v−pκ∗]] ≤√E[κ∗]√2E[v−p∗κ],

where the last step holds by Jensen’s inequality. In view of that, since , it follows that:

 34=E[v]≥E[pκ∗]=E[κ∗∑κ=2pκ−pκ−1]+p1≥−√E[κ∗]√2E[v−pκ∗]+1.

Solving for concludes the proof. ∎

The following lemma characterizes the value of when facing a strategic buyer.

###### Lemma 5.

For any , satisfies with . Furthermore, when and , can be replaced by the universal constant .

###### Proof.

Since an optimal strategy is played by the buyer, the surplus obtained by accepting a price at time must be greater than the corresponding surplus obtained when accepting the first price at time . It thus follows that:

 T∑t=κ∗γt−1(v−pκ∗) ≥T∑t=κ∗+1γt−1(v−pκ∗+1) ⇒γκ∗−1(v−pκ∗) ≥T∑t=κ∗+1γt−1(pκ∗−pκ∗+1)=γκ∗−γT1−γ(pκ∗−pκ∗+1).

Dividing both sides of the inequality by yields the first statement of the lemma. Let us verify the second statement. A straightforward calculation shows that the conditions on imply , therefore

 Cκ∗γ≥γ−γT−√TγT1−γ≥γ−γlog(2/γ)log(1/γ)1−γ=γ−γ21−γ=γ2(1−γ).

###### Proposition 5.

For any convex decreasing sequence , if , then there exists a valuation for the buyer such that

 Reg(Am,v0)≥max⎛⎜ ⎜⎝18√T−√T,  ⎷Cγ(T−√TγT)⎛⎝12−√CγT⎞⎠⎞⎟ ⎟⎠=Ω(√T+√CγT).
###### Proof.

In view of Proposition 1, we only need to verify that there exists such that

 Reg(Am,v0)≥  ⎷Cγ(T−√TγT)⎛⎝12−√CγT⎞⎠.

Let , and . If , then , from which the statement of the proposition can be derived straightforwardly. Thus, in the following we will only consider the case . Since, by definition, the inequality holds, we can write

 12≥pκmax=κmax∑κ=κmin+1(pκ−pκ−1)+pκmin≥κmax(pκmin+1−pκmin)+pκmin,

where the last inequality holds by the convexity of the sequence and the fact that . The inequality is equivalent to . Furthermore, by Lemma 5, we have

 maxv∈[12,1]Reg(Am,v) ≥max(κmax,(T−κmin)(pκmin−pκmin+1)) ≥max(κmax,Cγ(T−κmin)(pκmin−12)κmax).

The right-hand side is minimized for . Thus, there exists a valuation for which the following inequality holds:

 Reg(Am,v0)

Furthermore, we can assume that otherwise , which is easily seen to imply the desired lower bound. Thus, there exists a valuation such that

 Reg(Am,v0)≥  ⎷Cγ(T−√TγT)⎛⎝12−√CγT⎞⎠,

which concludes the proof. ∎

## Appendix B Simulations

Here, we present the results of more extensive simulations for and the monotone algorithm. Again, we consider two different scenarios. Figure 3 shows the experimental results for an agnostic scenario where the value of the parameter remains unknown to both algorithms and where the parameter of is set to . The results reported in Figure 4 correspond to the second scenario where the discounting factor is known to the algorithms and where the parameter for the monotone algorithm is set to . The scale on the plots is logarithmic in the number of rounds and in the regret.