The Value of Service Rate Flexibility in an Queue with Admission Control 111Research supported by the Greek Secretariat of Research and Technology, via a Greece-Turkey bilateral research program.
We consider a single server queueing system with admission control and the possibility to switch dynamically between increasing service rate values with service cost rate being convex in service rate. We explore the benefit due to service rate flexibility on the optimal profit and the admission thresholds, when service payment is made upon customer’s admission. We formulate a Markov Decision Process model for the problem of joint admission and service control considering both discounted and average expected profit maximization, and show that the optimal policy has a threshold structure for both controls. Regarding the benefit due to flexibility, we show that it is increasing in system congestion, and that its effect on the admission policy is to increase the admission threshold. We also derive a simple approximate condition between the admission reward and the relative cost of service rate increase, so that the service rate flexibility is beneficial. We finally show that the results extend to the corresponding model where service payment is made at the end of each service completion and differences on the benefit due to service flexibility with respect to the original model are pursued numerically.
Admission control is a queue management tool that can increase the efficiency of resource utilization in many service systems. Depending on the particular application, it can be used to preserve system capacity for future customers who bring higher profit, to limit the number of admitted customers in order to provide a better quality of service to those already in, etc. Admission control is often employed indirectly via dynamic pricing, or by price discrimination (direct or indirect) among different customer types. However in several situations frequent price changes may not be feasible, and in order to contain congestion, denying service to certain arriving customers may be necessary.
On the other hand, in several queueing systems it is possible to alleviate the congestion effects by adjusting the service capacity. For example in banks or call centers the number of servers may change during the day to follow variations of the arrival rate. The service rate may also be varied dynamically when the queue length becomes too long. Increasing (and in some cases decreasing) the service rate comes at a cost, but it has the advantage over admission control policies that fewer or no customers are turned away. It is thus of interest to explore to what extent a flexibility in service capacity can interact with admission control and when it can alleviate its effects.
In this paper we investigate the interaction of service rate flexibility and admission control in an M/M/1 queueing system, where the flexibility is modeled as the option to switch the service rate dynamically based on the congestion. We analyze the problem of joint admission and service rate control by formulating a Markovian Decision Process model to maximize the infinite horizon expected discounted and average profits. We explore the effect of service rate flexibility on the optimal profit and the admission thresholds. Specifically, the benefit of the service rate switch option is compared against a baseline case where only the lowest service rate is used and admission control is employed. Thus, in principle, the benefit is due both to the existence of higher service rates as well as the flexibility to use them. On the other hand, higher service rates are not free but come at higher costs, thus the benefit of using them reflects the tradeoff between serving at a faster rate and paying higher operational costs. When we use the term “benefit of flexibility” we mean exactly how this tradeoff manifests itself in the presence of admission control.
Specifically, we consider a simple admission-service control model with a finite number of available service rates, a fixed service reward per customer, holding costs that are increasing and convex in the number of customers and service cost rates that are increasing and convex in the service rate. We first show that the optimal policy has a threshold structure for both service and admission controls. We define the value of flexibility as the expected profit increase in the optimal admission control subproblem when the dynamic service rate change option becomes available, and show that this value is nondecreasing with the level of congestion. Furthermore, the effect on the optimal policy is to increase the admission threshold. We derive a simple sufficient condition between the admission reward and the service costs under which the service flexibility brings no benefit. Finally, we show that these results are valid when the service reward is obtained before or after service completion.
The single server model studied in this paper is more relevant in a make-to-order production setting, as opposed to a service provider system in which the service rate adjustment is made by varying the number of servers. A make-to-order production firm may be forced to apply admission control to incoming orders, if it has to pay high penalties for delayed deliveries. In such cases, if there exist a possibility to increase the production rate at times of high congestion, the need for order rejections may be alleviated. Depending on the structure of the production process, the capacity increase could be achieved either by increasing the machine load, if the production equipment has speed flexibility, or by increasing the number of shifts in a day.
Motivated by such a framework, the assumptions of convex holding cost is realistic because it reflects constant or increasing marginal penalties for order delays. It is also reasonable to assume service cost rates that are convex in the service rate, because maintenance costs may increase disproportionately with a high production speed, or because additional shifts require higher overtime payment rates.
The paper develops a model of dynamic optimization of queueing systems, a large area with very extensive literature. Both admission and service control models have been studied thoroughly. Stidham and Weber (1993) and Walrand (1988) survey several dynamic optimization models developed for queueing control.
For models with a single class of customers, as the one analyzed in this paper, admission control makes sense when there is an exogenous holding cost rate function. A simple model in this direction was first presented in Naor (1969), where arriving customers are admitted or not based on the observed queue length with the objective to maximize the overall customer benefit from receiving a reward for service completion minus a linear increasing holding cost per unit time of delay. It is shown that a socially optimal policy admits fewer customers than those who would decide to enter based on an individual optimality criterion. Stidham (1985) considers a queue under infinite horizon discounted cost, assuming a convex and nondecreasing holding cost rate function. It is shown that the optimal policy has a threshold structure if and only if the optimal benefit is concave in the number of customers in the system which in turn depends on convexity of the holding cost rate function. As in Stidham (1985), we also consider a convex, nondecreasing holding cost rate which implies a threshold property of the optimal admission policy.
On the other hand, in multi-class systems with finite capacity, admission control may be useful even in the absence of holding costs, because in this case admitting a customer implies the possibility of a loss of profit from a future higher class customer. Miller (1969) considers a system with parallel and identical servers, no waiting room and customer classes which contribute to the system different fixed rewards. This model results in a threshold-type optimal policy with a preferred class. Lippman and Ross (1971) analyze the optimal admission rule for a system with one server and no waiting room which receives offers from customers according to a joint service time and reward probability distribution. Carrizosa et al. (1998) and Ormeci et al. (2001) also investigate properties of optimal admission policies for certain loss systems. Carrizosa et al. (1998) develop an optimal static admission policy in an queueing system with customer classes with generally different service requirements and service rewards. Ormeci et al. (2001) examine the problem of dynamic admission control in a two class loss Markovian queueing system with different service rates and different fixed rewards for the two customer classes.
The admission control problem has been also analyzed in queueing systems under heavy traffic. In this framework, the dynamic optimization is usually approximated by a diffusion control problem, following the approach of Harrison (1988). Recent works in this area include Ward and Kumar (2008) and Kocaga and Ward (2010), both analyzing admission control under customer abandonments. Ward and Kumar (2008) analyze a queue in the conventional heavy traffic regime, where the optimal control depends on the sample path of the diffusion and the resulting asymptotically admission control policy of threshold type depends on second moment data of the interarrival and service times. On the other hand, Kocaga and Ward (2010) consider the long run average cost minimization problem of a multi-server system with a single arrival and a single server class in the Halfin-Whitt heavy traffic regime.
Dynamic service control in queueing systems is an equally large field. Several problems can be viewed as service control models, including controlled server vacations, server allocation policies in polling systems, etc. In an early work, Crabill (1974) examines dynamic service control under infinite horizon expected average expected cost in a maintenance system with finite available service rates, a linear holding cost rate and a reward collected in service completions. It is shown that the optimal service rate is increasing in the number of customers waiting in line. The monotonicity of the optimal service rate is also shown in Lippman (1975) in the framework of an queue, with service rates varying in a closed set and the holding cost rate increasing and convex. George and Harrison (2001) consider the service control problem in an queue where service rates are dynamically selected from a close subset of , under no switching cost, state-dependent holding cost and rate dependent service cost. They develop an asymptotic method for computing the optimal policy under average cost minimization by solving a sequence of approximating problems, each involving a truncation of the holding cost function. They prove that the optimal policies of the approximating problems converge monotonically to the optimal policy of the original problem and derive an implementable policy and a performance bound at each iteration. In our model we also derive a monotonicity property of the service control component of the problem, under a convexity assumption on the holding cost function. In the works mentioned above there are no switching costs for changing the service rate. We refer to Lu and Serfozo (1984), Hipp and Holzbaur (1988) and Kitaev and Serfozo (1999) for models that include service rate switching costs, resulting in hysteretic policies.
There are several works that consider and analyze the joint control problem of admitting or rejecting an incoming customer and varying the service rate. Most of them are motivated by and extend the work of George and Harrison (2001). For instance, in the area of asymptotic analysis in the heavy traffic regime considering diffusion control problems we refer to Ghosh and Weerasinghe (2007) and Ghosh and Weerasinghe (2010). Ghosh and Weerasinghe (2007) examine a queueing network where a central planner dynamically selects the service rate and buffer size that minimize the long-run average expected cost. The optimal policy is derived from the solution of a Brownian control problem and it consists of a feedback-type drift control and a threshold type admission policy. Ghosh and Weerasinghe (2010) consider a Markovian system with customer abandonments and address the infinite horizon discounted problem. In contrast to Ghosh and Weerasinghe (2007), it is proved that the optimal joint dynamic policy derived from the solution of the Brownian control problem is asymptotically optimal for the original problem. A common feature of these papers as well as the admission control problem in Kocaga and Ward (2010) is that uniformization is not applicable, because the transition rates are generally unbounded.
More relevant to our work are Ata and Shneorson (2006) and Adusumilli and Hasenbein (2010), since they also consider the joint control problem in a simple setting of an queue under an average reward/cost criterion. More specifically, Ata and Shneorson (2006) consider the joint admission and service control problem in an M/M/1 queue with adjustable arrival and service rates, under long-run average welfare maximization. They also formulate and solve an associated dynamic pricing problem. They show that the optimal arrival and service rates are monotone in the system length. However the optimal prices, which are set to induce the optimal arrival and service rates, are not necessarily monotone. Finally, they find that dynamic policies can result in significantly higher profits compared to static policies. Similarly to Ata and Shneorson (2006), Adusumilli and Hasenbein (2010) develop an efficient iterative method for computing the optimal policy under an average cost criterion, providing a computable upper bound on the optimality gap at each iteration step. It is also shown that service rates are monotone increasing in the system state.
Although the model considered by Adusumilli and Hasenbein (2010) can be seen as more general than ours since they assume a continuum action set for service decisions, our simpler setting contributes to the research area in several ways. First, we analyze the discounted expected profit maximization in the infinite horizon and show that the optimal policy converges to the corresponding long run average optimal policy, under easily verifiable sufficient conditions on the cost functions. Moreover, we characterize the optimal policy as threshold-based and derive a simple intuitive condition which orders the admission and service thresholds. This ordering is helpful, because it allows identifying which of the available service rates are not useful. The question on which of the service rates are actually beneficial in a given problem motivated us to extend the discussion on Ata and Shneorson (2006) and Adusumilli and Hasenbein (2010) by introducing the service rate flexibility, in order to examine the effect of the switch option on the admission policy and the optimal profit.
The rest of the paper is organized as follows. In Section 2 we define the joint control model, show several properties of the value function and establish the threshold structure of the optimal policy. In Section 3 we analyze the value of service flexibility and the effect of the high service rate switch option on the admission policy. In Section 4 we analyze the problem under the average reward criterion. In Section 5 we consider a variation of the original model, in which the service reward is collected at departure epochs and show that the main results still hold. In Section 6 we present a set of computational experiments exploring the sensitivity in the system parameters. Section 7 concludes.
2 Model Description
We consider a single server Markovian queue under the FCFS discipline, where customers arrive according to a Poisson process with rate . The service rate may be dynamically switched among available values without any cost. The service provider receives a fixed reward per customer admitted, and incurs holding and service costs as follows. The holding cost is equal to per unit time, where is the number of customers in the system. The function is assumed to be increasing and convex. The service cost is equal to per unit time, for using the service rate , respectively, for . We assume that
which corresponds to the service cost rate being convex in the service rate. We let for , with and and the convexity assumption for service cost rate can be rewritten as . Finally, we assume that the server is not allowed to close down when the system is empty. It is obvious that in this state the optimal service rate is the lowest, and, thus, cost rate is incurred.
Since the system is Markovian, it suffices to assume that the system manager makes a decision at both arrival and departure epochs. Service rate decisions can be made at both arrival and departure epochs, whereas admission decisions are made only at arrival epochs. Assuming continuous time discounting at rate , the service provider’s objective is to maximize the infinite horizon expected discounted net profit. Thus, the problem can be framed as a continuous time Markov Decision Process as follows.
Let be the time of the arrival, a random variable denoting the number of customers in the system at time and the indicator of the event that is an arrival epoch. We define the state vector as the pair , thus the state space is . State denotes an empty system.
For the action sets, let denote the service rate employed at time , where stands for service rate , respectively, and the admission decision at the arrival epoch, where denote rejection and admission, respectively. In states corresponding to arrival epochs, the action is defined by the pair , thus the action set is . On the other hand, in states corresponding to departure epochs, the action is defined only by , thus the action set is . Finally, let be the space of history dependent policies and denote the infinite horizon expected discounted net profit with initial state
The optimal value function is
and a policy is optimal if .
2.1 An Equivalent Model in Discrete Time
We can construct a discrete-time version of the Markovian Decision problem as follows. Depending on the state and the action employed, the transition rate out of any state can take values , or , for some . Let denote the maximum transition rate out of any state. Using standard uniformization arguments (see Section 11.5 of Puterman (1994)), it follows that the model described in (2) is equivalent to a model where the transition rates are all equal to and the transition probabilities are appropriately modified. Since in the original continuous time model the transition rates out of a state are generally different in different states, the discrete time formulation allows for transitions from a state back to itself so that the expected sojourn times are equal in the two models. These are referred to as fictitious transitions.
Via this transformation, the problem can be written in a form equivalent to a discrete time discounted Markov Decision Process, as follows
where for simplicity we omit the subscript .
Note that in the discrete time formulation, the equivalent discount factor per transition is equal to and for it has the standard property . For ease of the exposition, we normalize the time scale so that . This normalization is without loss of generality. Indeed, if we can make the following transformation on the parameters: and , for , , and, finally, . Under this transformation, it follows that , which implies that the value function and, thus the optimal policy derived by the system of optimality equations in (3) - (5) are identical with and without the normalization. Note that in Section 3 where we consider the criterion of average reward per unit-time as a limit of the discounted reward problem when and thus , we do not make this normalization assumption.
The finite horizon version of this last model is the following, where denotes the optimal discounted profit for the remaining transitions starting at state .
Note that in (6) the iteration index on the right hand side is still , because after an admission decision in state there is an instantaneous state switch to state or , so that the corresponding service-rate decision can also be made at that instant. The advantage of writing the optimality equations in this form is that only admission decisions are made in states and only service decisions in states .
To this end, we make the following assumption ensuring that the
holding cost does not increase too rapidly with the queue length.
There exists a constant such that:
There exists a constant and a positive integer such that: for ,
Assumption 1 is quite general. It can be easily seen that it is satisfied for power cost functions , as well as, exponential cost functions with and .
Proof. The proof follows by applying Theorem 11.5.3. of Puterman (1994). To do this we must verify the following
Assumption 11.5.1 (Puterman (1994)) implies that all transitions rates are bounded above. This is satisfied here with being the upper bound.
We will verify that the function satisfies the above properties. Note that is increasing in .
To show 2a, note that
Thus, 2a holds for .
To show 2b, for any and the possible transitions are to states with or customers. Since is increasing,
From condition (c1), , thus 2b holds for .
Finally, for 2c, iterating the above inequality times we obtain . Therefore it suffices to show that for some . However the last inequality holds by condition (c2).
2.2 The Optimal Threshold Policy
In this subsection, we derive the structure of the optimal policy. Specifically, we show in Theorem 2 that both admission and service rate controls are based on respective thresholds on the queue length. Furthermore, in Proposition 1 we derive a sufficient condition between the values of the economic and service parameters, which makes the option to switch to a higher service service rate essentially of no value for the service provider.
Let be the optimal admission decision in state and the optimal service rate decision in state when transitions remain. In addition, define
as the loss in future rewards because of the increased load from an accepted arrival and the increase in holding cost rate induced by an additional customer.
From equation (6), it follows that for any
Moreover, from equation (7), we derive that
Since , it follows after some algebra that
thus for , the condition becomes .
Similarly, since we derive that
thus for , the condition becomes .
Therefore, the optimal service decision at any state can be summarized as follows
The latter inequality implies that there exist situations where two or more service rates could be optimal. In order to make the analysis more tractable and without loss of generality, we make the following convention. When more than one service rates are optimal in a specific state, we consider the lowest of them as the optimal. Following this convention, the optimal service rate decision can be rewritten as follows
Finally, as we have discussed in the beginning of this section,
In order to characterize the optimal policy, we first present some intermediate properties. Lemma 1 shows that the value function is nonincreasing in .
The value function is nonincreasing in .
Proof. We will prove that is nonincreasing in for any , or equivalently that , by induction on . Obviously, and the statement holds for .
Assume that is nonincreasing in for . Then, for , we consider two cases for .
from the induction hypothesis and the monotonicity of .
For , the terms of (7) are nonincreasing functions of by the induction hypothesis, the assumption that is increasing in and the fact that the maximum function of nonincreasing functions is nonincreasing.
Therefore is nonincreasing in for any .
Case II: . For , we obtain similarly that is nonincreasing in from (6) and Case I.
Therefore the statement holds for and the proof is complete.
The monotonicity of in is intuitive. It implies that is nonnegative, thus it can be seen as a burden or profit reduction induced by one additional customer in state .
In the next theorem we show that the optimal policy is characterized by service and admission thresholds. We first define a generic threshold-type function that will be used in all the results.
For a function and define
with the convention .
It is easy to see that has the following properties:
is non-decreasing in for all non-decreasing functions .
If are such that for all , then for any .
We now proceed to the Theorem.
The value function is concave in , for .
There exist service thresholds for and admission thresholds such that:
Proof. The concavity of in state is equivalent to
being nondecreasing in . The proof is by induction on .
For , since , we obtain .
It follows that for all , thus (16) and (17) hold with for .
Furthermore, from (7) and (8), , where is increasing by assumption, thus (18) holds with .
Now suppose that i. holds for some . In order to prove the theorem, it suffices to show that ii. holds for and i. holds for . To do this we will prove the following claims in sequence:
(a) Let for .
Note that , for and .
Since, by the induction hypothesis, is nondecreasing in and , it follows from (13) that
which completes (a).
Moreover, we derive that .
(b) Under (a), (7) is transformed to
if and only if, for .
From the convexity of and the concavity of for by the induction hypothesis, we derive that is concave in , for all except from the threshold values. Thus, in order to complete the proof of (b), we must show that the following inequalities hold.
We must consider the following cases for the service rate employed at states and .
From the convexity of and the induction hypothesis, we derive that and for .
In addition, from (11) it follows that
From the convexity of and the induction hypothesis, we derive that and for .
From this inequality and the induction hypothesis we derive that
As in the proof of , we must consider the following cases for the service rate employed at states and for any .
Case 2A: Let and for any .
By (20), we obtain the following after some algebra for any :
From the convexity of and the induction hypothesis, we obtain:
Since and , the following inequality hold from (13) and the definition of ,
Case 2B: Let and for any .
From case , we have proved that and it remains to show that .
By (20), we obtain the following after some algebra for any :
From the convexity of and the induction hypothesis, we obtain: