Inventory Control for Spectrally Positive Lévy Demand Processes

# Inventory Control for Spectrally Positive Lévy Demand Processes

Kazutoshi Yamazaki
###### Abstract.

A new approach to solve the continuous-time stochastic inventory problem using the fluctuation theory of Lévy processes is developed. This approach involves the recent developments of the scale function that is capable of expressing many fluctuation identities of spectrally one-sided Lévy processes. For the case with a fixed cost and a general spectrally positive Lévy demand process, we show the optimality of an -policy. The optimal policy and the value function are concisely expressed via the scale function. Numerical examples under a Lévy process in the -family with jumps of infinite activity are provided to confirm the analytical results. Furthermore, the case with no fixed ordering costs is studied.
AMS 2010 Subject Classifications: 60G51, 93E20, 49J40
Key words: inventory models; impulse control; -policy; spectrally one-sided Lévy processes; scale functions

This version: September 9, 2019.
(corresponding author) Department of Mathematics, Faculty of Engineering Science, Kansai University, Suita-shi, Osaka 564-8680, Japan. Email: kyamazak@kansai-u.ac.jp Phone: +81-6-6368-1527.

## 1. Introduction

In this study, we introduce the fluctuation theory of Lévy processes to solve inventory control problems. In the continuous-time model, the majority of existing studies use a certain type of Lévy process as a demand process; typically it is modeled by Brownian motion, a compound Poisson process, or a mixture of the two. Pursuing the connections between the inventory theory and the Lévy-process theory is naturally of great importance. However, recent advances on Lévy processes have not been used to study inventory models. Therefore, our objective in the current study is to fill this void. Toward this end, we introduce the so-called scale function to inventory control, which plays a key role in the fluctuation theory of Lévy processes. We demonstrate both analytically and numerically that it is in fact a powerful tool to solve inventory control problems.

The common aim of inventory control is to derive the optimal replenishment policy to minimize both inventory and ordering costs. This study focuses on the discounted continuous-time model with fixed and proportional costs. We assume that the order quantity is continuous and back-orders are allowed. Furthermore, we assume the absence of lead time, perishability, and lost sales. This setting is the same as that in the seminal study by [9] where they solved (complementing [12]) the case where the demand arrives as a combination of Brownian motion and a compound Poisson process. In the current study, a scale-function-based approach is used to solve the problem for any choice of spectrally positive Lévy demand process i.e. a Lévy process with only upward jumps.

To our best knowledge, this is the first study on inventory models where the theory of scale functions is applied to derive the optimal solution. We focus on the aforementioned simple setting and demonstrate that the existing known properties of the scale function can be efficiently used to follow the classical guess and verify procedure in a straightforward fashion. Of course, applying the same technique in other inventory models can be of great importance. We expect that this study can potentially serve as a guide on how to tackle these open problems using the theory of scale functions. For a detailed review of inventory control problems, we refer the readers to [12, 13, 43] and the recent book by Bensoussan [10].

Over the last decade or two, significant developments in the fluctuation theory of Lévy processes have been presented (see, e.g., the textbooks by Bertoin [14], Doney [18], and Kyprianou [31]). Whereas the majority of Lévy processes still remain to be analytically tractable, numerous computations have been made possible when it comes to the spectrally one-sided Lévy process (a Lévy process with only upward or downward jumps) because of the development of the scale function.

For every spectrally one-sided Lévy process, the corresponding scale function, which is concisely defined in terms of the Laplace exponent, plays a great role. Without specifying a particular process type, it can efficiently express, e.g., hitting time probabilities, resolvent (potential) measures, and overshoot/undershoot distributions. It has gradually come into use in several stochastic models. In particular, it is now considered as a key tool in insurance mathematics; the classical formulation that models the surplus of an insurance company using a compound Poisson process is now being replaced by the spectrally negative Lévy model. Other stochastic models where the scale function plays a key role include, e.g., optimal stopping [20, 32, 36, 41, 47], stochastic games [4, 5, 19, 26], and mathematical finance [34, 46].

In this study, we begin with the same formulation as in [12] and solve for the general spectrally positive Lévy process. We have no difficulty in conjecturing that the form of an optimal policy is again of the -type, i.e., bringing the inventory level up to whenever it goes below is optimal. However, the scale-function-based approach is worth studying for various reasons. Here we summarize its advantages over the classical approach that involves solutions to integro-differential equations (IDEs).

Generalization of the demand process: Essentially, all existing Lévy-based inventory models employ the Lévy processes with upward jumps of finite activity [as defined in (3.15)], i.e., the demand arrives as a (drifted) compound Poisson process with or without a Brownian motion. The set of these Lévy processes excludes those with infinite activity/variation. In other words, the existing models do not cover well-known processes such as the (spectrally positive versions of) variance gamma, CGMY, and normal inverse Gaussian processes as well as classical ones as the gamma process and a subset of stable processes. For these processes, the IDE involves integration with respect to infinite measures, and hence, the IDE-based approach can become intractable. On the other hand, the approach that uses the scale function can accommodate these processes without any additional work.

Lévy processes of infinite activity/variation have been applied in the field of finance and insurance. Empirical evidence shows that asset prices can be more precisely modeled by jumps of infinite activity as opposed to being modeled by Brownian motion (see the introduction in [16]). Because the demand is closely linked to the price of an item, modeling inventory systems using these processes also makes sense. This reason motivates us to work on the inventory control problem for a general spectrally positive Lévy demand process.

Conciseness of arguments: The scale function can explicitly express the expected total costs under the -policy. Using this function, the optimal solution can be derived in a straightforward manner. As typically conducted in solving stochastic control problems, the optimal solution is first “guessed,” and its optimality is then confirmed by “verification” arguments. The known properties of the scale function help one achieve these objectives.

The first guessing step is reduced to choosing the pair by imposing the following two conditions on the expected cost function as follows:

1. The continuous (resp. smooth) fit condition at the lower threshold when the demand process is of bounded (resp. unbounded) variation,

2. The condition at the upper threshold such that the slope at is equal to the negative of the unit proportional cost.

Because of the (semi-)analytical expression of the expected costs under the -policy, these two conditions can be concisely rewritten by the scale function. For the results in this study, become the zeros of the function defined in (4.6) and that of its derivative with respect to the second argument. Using this graphical interpretation and taking advantage again of the certain known properties of the scale function, the existence of such can be confirmed.

The second verification step reduces to indicating that the candidate value function is sufficiently smooth and satisfies the quasi-variational inequalities (QVIs). The former can be easily confirmed because of the known smoothness/continuity properties of the scale function that depend on the path variation of the process. Unfortunately, part of the latter can become difficult to verify. However, in this study, the optimality in fact holds because of several known facts on the scale function and the results obtained in [9].

Computability: The derived optimal thresholds and the associated optimal value function are concisely written via the scale function. Hence, the computation of these factors is essentially equivalent to that of the scale function. Because the scale function is defined by its Laplace transform written in terms of the Laplace exponent, the Laplace transform needs to be inverted either analytically or numerically to compute it.

Fortunately, some important classes of Lévy processes have rational forms of Laplace exponents. For these processes, analytical forms of scale functions can be easily obtained by partial fraction decomposition. In Section 3.2, we show examples of such processes. Among others, the phase-type Lévy process of [2] is of great interest both in analytical and numerical aspects. It admits a rational form of Laplace exponent and is known to be dense in the set of all Lévy processes. This means that, it has an explicit form of scale function, and, more importantly, any scale function can be approximated by the scale function of this form. Egami and Yamazaki [22] conducted a sequence of numerical experiments to confirm the accuracy of this approximation.

Alternatively, the scale function can always be directly computed via numerical Laplace inversion. As discussed in Kuznetsov et al. [29], a scale function can be written as the difference between an exponential function (whose parameter is defined by in the current paper) and the resolvent (potential) term [see the third equation in (3.14) below]. Hence, the computation is reduced to that of the resolvent term. It is a bounded function that asymptotically converges to zero, and hence, numerical Laplace inversion can be quickly and accurately conducted. For more details, we refer the readers to Section 5 of [29].

To confirm the analytical results obtained in this study, we provide numerical examples with a quadratic inventory cost and a demand process in the -family in [28] with jumps of infinite activity. We can see that the optimal levels and the value function can be instantaneously computed.

Sensitivity analysis: Because of the analytical expressions of optimal thresholds and the value function, sensitivity analysis can be more directly conducted.

The sensitivity with respect to the parameter of the underlying Lévy process is equivalent to that of the scale function. For example, the smoothness of the optimal value function is directly linked to the asymptotic behavior of the scale function near zero. In our numerical results, we consider the cases where the diffusion coefficient is zero and positive, and analyze how the optimal solutions differ.

Furthermore, the sensitivity with respect to the fixed and unit proportional costs is of great interest. In particular, the forms of the optimal solutions are different depending on the existence of a fixed ordering cost. The distance between the optimal thresholds and is expected to shrink as the fixed cost decreases. In this study, we also consider the case with no fixed ordering cost and show that the optimal policy is of the barrier type. Using the fluctuation theory of reflected Lévy processes as in [3] and [42], the value function can again be written using the scale function. We numerically confirm that, as the fixed cost decreases to zero, the optimal -policy converges to the optimal barrier strategy.

Before closing the introduction, we briefly review the scale-function approach used in other stochastic control problems and discuss the similarities and differences with the results of this study.

The most relevant factor is the optimal dividend problem (de Finetti’s problem) in insurance mathematics. After the development of the fluctuation theory of reflected Lévy processes, many authors have applied the results in the spectrally negative Lévy model of de Finetti’s problems [33, 37, 38, 39] (see also [7, 8, 49] for the spectrally positive Lévy models). The main difference with our inventory control problem is that whereas our problem has an infinite time horizon, de Finetti’s problem is terminated at the ruin time (or the first time the controlled process goes below the zero level). In our problem, the absence of a ruin makes the problem both easier and more difficult. Whereas in de Finetti’s problem the optimality holds only for a subset of Lévy processes (see [37]), it holds for any spectrally positive demand process in our problem (except that the Lévy measure needs to have a light tail). On the other hand, without the ruin, the value function becomes unbounded (even not Lipschitz continuous), and the verification arguments become significantly more difficult (see the proof of Theorem 7.1).

In our problem, with a fixed cost, two threshold levels for the optimal -policy need to be found, which is different from the optimal stopping and de Finetti’s problems (with the exceptions of [8] and [39]), where only one parameter describes the optimal policy. In general, choosing two parameters is significantly more difficult. However the scale function helps one achieve this objective. Whereas this technique remains to be established, similar arguments can be found in stochastic games [19, 26] where two parameters characterize the equilibrium between two players.

The rest of this study is organized as follows. Section 2 presents a mathematical model of the problem. Section 3 reviews spectrally one-sided Lévy processes and scale functions. Section 4 presents the computation of the expected total costs under the -policy via the scale function. Section 5 obtains a candidate policy via the continuous/smooth fit principle and shows its existence. Section 6 verifies its optimality. Section 7 studies the case without a fixed ordering cost. Section 8 presents numerical results and Section 9 concludes this study.

Throughout this study, and are used to indicate the right- and left-hand limits, respectively. Superscripts , , , and are used to indicate positive and negative parts. Finally, we let , for any right-continuous process .

## 2. Inventory Models

Let be a probability space on which a stochastic process with , which represents the demand of a single item, is defined. Under the conditional probability , the initial level of inventory is given by (in particular, we let ). Hence, the inventory in the absence of control follows the stochastic process

 Xt:=x−Dt,t≥0. (2.1)

We shall consider the case where is a spectrally positive Lévy process, or equivalently is a spectrally negative Lévy process; we will define these processes formally in the next section. Let be the filtration generated by (or equivalently by ).

An (ordering) policy is given in the form of an impulse control with and , , where is an increasing sequence of -stopping times and , for , is an -measurable random variable. Corresponding to every policy , the (controlled) inventory process is given by where and

 Uπt:=Xt+Lπt,t≥0.

We assume that the order quantity is continuous and backorders are allowed. In addition, we assume that there is no lead time, perishability, and lost sales.

The problem is to compute, for a given discount factor , the total expected costs given by

 vπ(x) :=Ex[∫∞0e−qtf(Uπt)dt+∞∑i=1e−qTπig(uπi)] =Ex[∫∞0e−qtf(Uπt)dt+∑0≤t<∞e−qtg(ΔLπt)1{ΔLπt>0}],x∈R,

and to obtain an admissible policy that minimizes it, if such a policy exists. We shall additionally assume that an admissible policy is such that and are both well-defined and finite -a.s.

Here, corresponds to the cost of holding and shortage when and , respectively. Regarding , we assume

 g(y):=Cy+K,y>0, (2.2)

for some unit (proportional) cost of the item and fixed ordering cost . We shall study the case separately in Section 7. As in [9, 12], we assume the following.

###### Assumption 2.1.
1. is continuous and piecewise continuously differentiable with , and grows (or decreases) at most polynomially (that is to say, there exist and such that for all such that ).

2. For some ,

 ~f(x) :=f(x)+Cqx,x∈R, (2.3)

is decreasing and convex on and increasing on .

3. For some and , we have for a.e. .

As in [9, 12], this is a crucial assumption for our analysis, and in particular will be used to verify the existence of the optimal policy.

Finally, the (optimal) value function is written as

 v(x):=infπ∈Πvπ(x),x∈R, (2.4)

where is the set of all admissible policies. If the infimum is attained by some admissible policy , then we call an optimal policy.

## 3. Spectrally Negative Lévy Processes and Scale Functions

Throughout this paper, we assume that the demand process is a spectrally positive Lévy process. Equivalently, the process as in (2.1) is a spectrally negative Lévy process. By the Lévy-Khintchine formula (see e.g. Bertoin [14]), any Lévy process can be characterized by its Laplace exponent. Here, Assumption 3.1 we shall assume below allows us to write the Laplace exponent of as

 ψ(s):=logE[esX1]=γs+12σ2s2+∫(−∞,0)(esz−1−sz)ν(dz),s≥0, (3.1)

where is a Lévy measure with the support that satisfies the integrability condition .

A Lévy process has paths of bounded variation a.s. or otherwise it has paths of unbounded variation a.s. The former holds if and only if and ; in this case, the expression (3.1) can be simplified to

 ψ(s)=δs+∫(−∞,0)(esz−1)ν(dz),s≥0,

with . We exclude the case in which is the negative of a subordinator (i.e., is monotonically decreasing a.s.). This assumption implies that when is of bounded variation.

Regarding the Lévy measure , we make the same assumption as Assumption 3.2 of [12] so that has a finite moment for some small .

###### Assumption 3.1.

We assume that there exists a such that

 ∫(−∞,−1]e¯β|x|ν(dx)<∞.

This guarantees that

 μ:=E[X1]=ψ′(0+), (3.2)

is well-defined and finite.

For the rest of this section, we briefly review the fluctuation theory of the spectrally negative Lévy process and the scale function, which will play significant roles in solving the problem. Note that, unless otherwise stated, Assumption 3.1 is not required for the results in the next subsection to hold.

### 3.1. Scale functions

Fix . For any spectrally negative Lévy process, there exists a function called the -scale function

 W(q):R→[0,∞),

which is zero on , continuous and strictly increasing on , and is characterized by the Laplace transform:

 ∫∞0e−sxW(q)(x)dx=1ψ(s)−q,s>Φ(q), (3.3)

where

 Φ(q):=sup{λ≥0:ψ(λ)=q}.

Here, the Laplace exponent in (3.1) is known to be zero at the origin and convex on ; therefore is well-defined and is strictly positive as . We also define, for ,

 ¯¯¯¯¯¯W(q)(x) :=∫x0W(q)(y)dy, Z(q)(x) :=1+q¯¯¯¯¯¯W(q)(x), ¯¯¯¯Z(q)(x) :=∫x0Z(q)(z)dz=x+q∫x0∫z0W(q)(w)dwdz.

Because for , we have

 ¯¯¯¯¯¯W(q)(x)=0,Z(q)(x)=1and¯¯¯¯Z(q)(x)=x,x≤0. (3.4)

The most well-known application of the scale function can be found in the two-sided exit problem. Let us define the first down- and up-crossing times, respectively, of by

 τ−b:=inf{t>0:Xt0:Xt>b},b∈R. (3.5)

Then, for any and ,

 Ex[e−qτ+b1{τ+b<τ−0}]=W(q)(x)W(q)(b),Ex[e−qτ−01{τ+b>τ−0}]=Z(q)(x)−Z(q)(b)W(q)(x)W(q)(b),Ex[e−qτ−0]=Z(q)(x)−qΦ(q)W(q)(x). (3.6)

The continuity and smoothness of the scale function depend on the path variation of . First, regarding the behaviors around zero, as in Lemmas 3.1 and 3.2 of [29],

 W(q)(0) ={0,if X is of unbounded variation,1δ,if X is of bounded variation, (3.7) W(q)′(0+) :=limx↓0W(q)′(x)=⎧⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪⎩2σ2,if σ>0,∞,if σ=0andν(−∞,0)=∞,q+ν(−∞,0)δ2,if σ=0andν(−∞,0)<∞. (3.8)

These properties are particularly useful in applying the continuous/smooth fit principle in stochastic control problems. In this paper, we use this to obtain the candidate thresholds of the optimal policy; see Section 5.1 below.

Regarding the smoothness on , we have the following; see [17] for more comprehensive results.

###### Remark 3.1.

If is of unbounded variation or the Lévy measure does not have an atom, then it is known that is . Hence,

1. is and for the bounded variation case, while it is and for the unbounded variation case,

2. is and for the bounded variation case, while it is and for the unbounded variation case.

These smoothness results are important in order to apply the Itô formula where the (candidate) value function must be (resp. ) for the case of unbounded (resp. bounded) variation.

In Figure 1, we show sample plots of the scale function on . As reviewed in (3.7), the behaviors around zero depend on the path variation of the process.

#### 3.1.1. Change of measures

Fix and define as the Laplace exponent of under with the change of measure

 (3.9)

see page 213 of [31]. Suppose and are the scale functions associated with under (or equivalently with ). Then, by Lemma 8.4 of [31], , , which is well-defined even for by Lemmas 8.3 and 8.5 of [31]. In particular, we define

 WΦ(q)(x):=W(0)Φ(q)(x)=e−Φ(q)xW(q)(x),x∈R, (3.10)

which is known to be an increasing function.

By this change of measure (3.9), one can express the expectation for using the scale function. Using this, we can compute the (discounted) moments; for example, by taking the derivative with respect to and then a limit, we have under (3.2)

 Ex[e−qτ−0Xτ−0]=¯¯¯¯Z(q)(x)−μ¯¯¯¯¯¯W(q)(x)−q−μΦ(q)Φ(q)2W(q)(x),x∈R; (3.11)

see Proposition 2 of [3] for more detailed computation.

#### 3.1.2. Martingale properties

By Proposition 2 of [3] and as in the proof of Theorem 8.10 of [31], the processes

 e−q(t∧τ−0∧τ+B)Z(q)(Xt∧τ−0∧τ+B)ande−q(t∧τ−0∧τ+B)R(q)(Xt∧τ−0∧τ+B),t≥0,

for any and , , are -martingales.

Let be the infinitesimal generator associated with the process applied to a (resp. ) function for the case is of bounded (resp. unbounded) variation: for ,

 Lh(x):=γh′(x)+12σ2h′′(x)+∫(−∞,0)[h(x+z)−h(x)−h′(x)z]ν(dz),(resp. Lh(x):=δh′(x)+∫(−∞,0)[h(x+z)−h(x)]ν(dz)). (3.12)

Thanks to the smoothness of and on as in Remark 3.1, we obtain

 (L−q)Z(q)(y)=(L−q)R(q)(y)=0,y>0. (3.13)

We use these properties in the proof of Lemma 6.1 below.

#### 3.1.3. q-resolvent (potential) measure

The scale function can express concisely the -resolvent (potential) density. As summarized in Theorem 8.7 and Corollaries 8.8 and 8.9 of [31] (see also Bertoin [15], Emery [23], and Suprun [45]), we have

 (3.14)

It is clear that these can be used in the computation of inventory costs (see (4.11)).

The same identities can be obtained when the process is replaced with the running infimum process , . In particular, by Corollary 2.2 of [29], for Borel subsets in the nonnegative half-line,

 E[∫∞0e−qt1{−X––t∈dx}dt]=1Φ(q)W(q)(dx)−W(q)(x)dx=1Φ(q)[Θ(q)(x)dx+W(q)(0)δ0(dx)],

where is the measure such that (see [31, (8.20)]) and is the Dirac measure at zero. Here, for all ,

 Θ(q)(x):=W(q)′(x+)−Φ(q)W(q)(x)=eΦ(q)xW′Φ(q)(x+)>0.

We shall use this function and

 ¯¯¯¯Θ(q)(x) :=W(q)(x)−Φ(q)¯¯¯¯¯¯W(q)(x)>0,

later in the paper. Here the positivity of holds because for and that of holds because it is the integral of . Their positivity will be important in deriving the existence of the optimal solution and the verification of optimality. See the proofs of Proposition 5.2 and Lemma 5.3 below.

### 3.2. Examples of scale functions

We shall conclude this section with concrete examples of scale functions. We refer the readers to, e.g., [27, 29] for other examples.

#### 3.2.1. Brownian motion

The simplest and nonetheless important example of a Lévy process is Brownian motion with a diffusion coefficient and a drift . In this case, the Laplace exponent (3.1) reduces to . The Laplace transform (3.3) can be analytically inverted; the scale function becomes

 W(q)(x)=e(√δ2+2q−δ)x−e−(√δ2+2q+δ)x√δ2+2q,x≥0.

#### 3.2.2. α-stable processes

The spectrally negative stable process with index has the Laplace exponent . As in Example 8.2 of [31], its scale function is given by

 W(q)(x)=αxα−1E′α(qxα),x≥0,

where is the Mittag-Leffler function of parameter (which is a generalization of the exponential function).

#### 3.2.3. Phase-type Lévy processes [2]

A spectrally negative Lévy process with a finite Lévy measure admits a decomposition

 Xt=X0+δt+σBt−Nt∑n=1Zn,0≤t<∞, (3.15)

for some , , standard Brownian motion , a Poisson process with arrival rate and a sequence of i.i.d. random variables . These processes are assumed to be mutually independent.

The spectrally negative phase-type Lévy process is a special case where is phase-type distributed; a distribution on is of phase-type if it is the distribution of the absorption time in a finite state continuous-time Markov chain consisting of one absorbing state and transient states. If the phase-type distributed random variable is given by a Markov chain with intensity matrix over all transient states and the initial distribution , then the Laplace exponent (3.1) becomes

 ψ(s)=δs+12σ2s2+ρ(α(sI−T)−1t−1),

which can be extended to except at the negative of eigenvalues of .

Suppose is the set of the (possibly complex-valued) roots of the equality with negative real parts, and if these are assumed distinct, then the scale function can be written

 W(q)(x)=eΦ(q)xψ′(Φ(q))+∑i∈Iq1ψ′(−ξi,q)e−ξi,qx,x≥0. (3.16)

The set of phase-type Lévy processes is dense in the set of all Lévy processes, and hence the scale function of any spectrally negative Lévy process can be approximated by the scale function of the form (3.16). See [22] for numerical results.

#### 3.2.4. Meromorphic Lévy processes [30]

As a variant of the phase-type Lévy process, the meromorphic Lévy process [30] is a type of Lévy process whose Lévy measure admits a density of the form

 ν(dz)=∞∑j=1pjηje−ηj|z|dz,z<0,

for some . Examples include Lévy processes in the -family as we use for numerical results in Section 8 below. The equation has a countable number of negative real-valued roots that satisfy the interlacing condition:

 ⋯<−ηk<−ξk,q<⋯<−η2<−ξ2,q<−η1<−ξ1,q<0.

As discussed in [29], the scale function can be written as

 W(q)(x)=eΦ(q)xψ′(Φ(q))+∞∑i=11ψ′(−ξi,q)e−ξi,qx,x≥0. (3.17)

## 4. The (s,S)-policy

In this paper, we aim to prove that the -policy is optimal for some . For arbitrary , an -policy, , brings the level of the inventory process up to whenever it goes below .

This process can be defined recursively as follows. First, it moves like the original process until the first time it goes below :

 Ts,S1:=τ−s=inf{t>0:Xt

This is immediately pushed up to (and hence ) by adding , and then follows

 Ut=S+(Xt−XTs,S1),Ts,S1≤t

where is the first time after the pre-controlled process

 ~Us,St:=Us,St−+ΔXt,t≥0 (4.1)

goes below .

The process after can be constructed in the same way. The stopping time , for each , corresponds to the jump of ; the -measurable random variable

 us,Si:=S−~UTs,Si

is the corresponding jump size. Clearly, this strategy is admissible: it can be written

 Ls,St=∑i≥1:Ts,Si≤tus,Si,t≥0.

The process is a strong Markov process. To see this, the Markov property is clear because, from the construction, the distribution of only depends on and the increment , where the latter is independent of . This can be strengthened to the strong Markov property by the right-continuity of the path of (see, e.g., the proof of Exercise 3.2 of [31], which shows the strong Markov property of a reflected Lévy process).

In Figure 2, we show sample paths of the controlled process and its corresponding control process for , when the starting value is . Due to the negative jumps, the process as in (4.1) can jump to a level strictly below (and is then immediately pushed to ). In the figure, the red arrows show the corresponding impulse control: each time the process goes below , it pushes up to by adding . At the same time, the process increases by the same amount.

Our objective in this section is to compute the expected total costs under the -policy, denoted by

 vs,S(x):=Ex[∫∞0e−qtf(Us,St)dt+∑0≤t<∞e−qtg(ΔLs,St)1{ΔLs,St>0}],x∈R. (4.2)

Toward this end, we compute that of its “tilted version” (with respect to the unit proportional cost )

 ~vs,S(x) :=vs,S(x)+Cx,x∈R. (4.3)

As has been already seen in, e.g., [9, 12], the computation in the following arguments becomes simpler if we deal with (4.3) rather than (4.2). The reason will be clear in later arguments. In particular, for , the slope of is and hence that of is . Accordingly, we use instead of . To see this, note in the verification of optimality that we need to show the positivity of , and we have .

###### Proposition 4.1.

For all ,

 ~vs,S(S)=Φ(q)q¯¯¯¯Θ(q)(S−s)[¯¯¯¯Θ(q)(S−s)[Ψ(s;~f)−qΦ(q)(K+Cμq)]+G(s,S)],~vs,S(x)=⎧⎪⎨⎪⎩−¯¯¯¯Θ(q)(x−s)¯¯¯¯Θ(q)(S−s)G(s,S)+G(s,x)+~vs,S(S),x≥s,K+~vs,S(S),x

where we define

 Ψ(s;~f) :=∫∞0e−Φ(q)y~f(y+s)dy=∫∞se−Φ(q)(y−s)~f(y)dy,s∈R, (4.5) G(s,x) :=Φ(q)Ψ(s;~f)¯¯¯¯¯¯W(q)(x−s)+K−∫xsW(q)(x−y)~f(y)dy,x,s∈R. (4.6)

For the rest of the paper, we define , and analogously to (4.5). Analytical properties (as in, e.g., Lemma 5.1) of these functions are important in deriving the results of this paper. In particular, for the limiting case as we study in Section 7 below, the optimal threshold is given by the root of (see Lemma 5.1).

###### Remark 4.1.

By Assumption 2.1(1), the functions , , and are finite for any .

### 4.1. Proof of Proposition 4.1

Recall the down-crossing time as in (3.5). Notice from the construction that, -a.s., for and on . By these, (2.2), and the strong Markov property of , the expectation (4.2) must satisfy, for every ,

 vs,S(x)=Ex[∫τ−s0e−qtf(Xt)dt]+Ex[e−qτ−s(C(S−Xτ−s)+K)]+Ex[e−qτ−s]vs,S(S). (4.7)

Define

 k(s,x) :=Ex[∫τ−s0e−qtf(Xt)dt]−CEx[e−qτ−sXτ−s]+KEx[e−qτ−s]+Cx,x>s. (4.8)

Then, using (3.6) and (4.3), we can write (4.7) as

 (4.9)

where

 ~vs,S(S)=Φ(q)qk(s,S)¯¯¯¯Θ(q)(S−s),S>s. (4.10)

Here (4.10) holds by solving (4.9) for . Hence, once we identify the expression for , we can compute and consequently the whole function (4.3) as well.

The proof consists of evaluating the three expectations in (4.8). The first expectation can be obtained directly by (3.14). We have for ,

 Ex[∫τ−s0e−qtf(Xt)dt]=W(q)(x−s)Ψ(s;f)−∫xsW(q)(x−y)f(y)dy, (4.11)

which is well-defined by Remark 4.1.

Here, by the following lemma, we can write (4.5) and the integral interchangeably for and .

###### Lemma 4.1.

For , we have

 Ψ(s;f) =Ψ(s;~f)−CqΦ(q)[1Φ(q)+s], ∫xsW(q)(x−y)f(y)dy =∫xsW(q)(x−y)~f(y)dy−C[sZ(q)(x−s)+¯¯¯¯Z(q)(x−s)−x],x∈R.
###### Proof.

See Appendix A.1. ∎

Using (4.11) and Lemma 4.1, we have, for ,

 Ex[∫τ−s0e−qtf(Xt)dt]=¯¯¯¯Θ(q)(x−s)[Ψ(s;~f)−CqΦ(q)(1Φ(q)+s)]+C[¯¯¯¯Z(q)(x−s)−(x−s)]+Φ(q)¯¯¯¯¯¯W(q)(x−s)[Ψ(s;~f)−CqΦ(q)2]−∫xsW(q)(x−y)~f(y)dy. (4.12)

Regarding the second expectation of (4.8), combining (3.6) and (3.11) gives the following:

 Ex[e−qτ−sXτ−s] =¯¯¯¯Z(q)(x−s)−(s−μq)qΦ(q)¯¯¯¯Θ(q)(x−s)+s−qΦ(q)2W(q)(x−s),x,s∈R. (4.13)

In (4.8), substituting (3.6) (with replaced with ), (4.12), and (4.13), we have the expression

 k(s,x) =¯¯¯¯Θ(q)(x−s)[Ψ(s;~f)−qΦ(q)(K+Cμq)]+G(s,x).

Substituting this in (4.10) shows the first equation of (4.4). The second equation is immediate by (4.9).

###### Remark 4.2.

The same technique can be used to compute the expected costs under the four parameter band policy for an extension of the problem with a two-sided control; see the note by Yamazaki [48].

## 5. Candidate policies

In the previous section, we computed for arbitrary as in (4.4). Here two different functions for and are pasted together at the point , and hence the continuity/smoothness of the function does not necessarily hold at the point . The principle of smooth/continuous fit chooses the parameters so that the function becomes continuous/smooth at . In our problem, as we need to identify two parameters , we use one additional condition described below.

In this section, we obtain the candidates of for the optimal policy. Toward this end, we choose such that (1) is continuous (resp. differentiable) at the lower threshold