# Exact Methods for Multistage Estimation of a Binomial Proportion

## Abstract

We first review existing sequential methods for estimating a binomial proportion. Afterward, we propose a new family of group sequential sampling schemes for estimating a binomial proportion with prescribed margin of error and confidence level. In particular, we establish the uniform controllability of coverage probability and the asymptotic optimality for such a family of sampling schemes. Our theoretical results establish the possibility that the parameters of this family of sampling schemes can be determined so that the prescribed level of confidence is guaranteed with little waste of samples. Analytic bounds for the cumulative distribution functions and expectations of sample numbers are derived. Moreover, we discuss the inherent connection of various sampling schemes. Numerical issues are addressed for improving the accuracy and efficiency of computation. Computational experiments are conducted for comparing sampling schemes. Illustrative examples are given for applications in clinical trials.

## 1 Introduction

Estimating a binomial proportion is a problem of ubiquitous significance in many areas of engineering and sciences. For economical reasons and other concerns, it is important to use as fewer as possible samples to guarantee the required reliability of estimation. To achieve this goal, sequential sampling schemes can be very useful. In a sequential sampling scheme, the total number of observations is not fixed in advance. The sampling process is continued stage by stage until a pre-specified stopping rule is satisfied. The stopping rule is evaluated with accumulated observations. In many applications, for administrative feasibility, the sampling experiment is performed in a group fashion. Similar to group sequential tests [5, Section 8], [27], an estimation method based on taking samples by groups and evaluating them sequentially is referred to as a group sequential estimation method. It should be noted that group sequential estimation methods are general enough to include fixed-sample-size and fully sequential procedures as special cases. Particularly, a fixed-sample-size method can be viewed as a group sequential procedure of only one stage. If the increment between the sample sizes of consecutive stages is equal to , then the group sequential method is actually a fully sequential method.

It is a common contention that statistical inference, as a unique science to quantify the uncertainties of inferential statements, should avoid errors in the quantification of uncertainties, while minimizing the sampling cost. That is, a statistical inferential method is expected to be exact and efficient. The conventional notion of exactness is that no approximation is involved, except the roundoff error due to finite word length of computers. Existing sequential methods for estimating a binomial proportion are dominantly of asymptotic nature (see, e.g., [6, 24, 25, 28, 33] and the references therein). Undoubtedly, asymptotic techniques provide approximate solutions and important insights for the relevant problems. However, any asymptotic method inevitably introduces unknown error in the resultant approximate solution due to the necessary use of a finite number of samples. In the direction of non-asymptotic sequential estimation, the primary goal is to ensure that the true coverage probability is above the pre-specified confidence level for any value of the associated parameter, while the required sample size is as low as possible. In this direction, Mendo and Hernando [31] developed an inverse binomial sampling scheme for estimating a binomial proportion with relative precision. Tanaka [34] developed a rigorous method for constructing fixed-width sequential confidence intervals for a binomial proportion. Although no approximation is involved, Tanaka’s method is very conservative due to the bounding techniques employed in the derivation of sequential confidence intervals. Franzén [21] studied the construction of fixed-width sequential confidence intervals for a binomial proportion. However, no effective method for defining stopping rules is proposed in [21]. In his later paper [22], Franzén proposed to construct fixed-width confidence intervals based on sequential probability ratio tests (SPRTs) invented by Wald [35]. His method can generate fixed-sample-size confidence intervals based on SPRTs. Unfortunately, he made a fundamental flaw by mistaking that if the width of the fixed-sample-size confidence interval decreases to be smaller than the pre-specified length as the number of samples is increasing, then the fixed-sample-size confidence interval at the termination of sampling process is the desired fixed-width sequential confidence interval guaranteeing the prescribed confidence level. More recently, Jesse Frey published a paper [23] in The American Statistician (TAS) on the classical problem of sequentially estimating a binomial proportion with prescribed margin of error and confidence level. Before Frey submitted his original manuscript to TAS in July 2009, a general framework of multistage parameter estimation had been established by Chen [7, 9, 11, 13, 14], which provides exact methods for estimating parameters of common distributions with various error criterion. This framework is also proposed in [15]. The approach of Frey [23] is similar to that of Chen [7, 9, 11, 13, 14] for the specific problem of estimating a binomial proportion with prescribed margin of error and confidence level.

In this paper, our primary interests are in the exact sequential methods for the estimation of a binomial proportion with prescribed margin of error and confidence level. We first introduce the exact approach established in [7, 9, 11, 13, 14]. In particular, we introduce the inclusion principle proposed in [14] and its applications to the construction of concrete stopping rules. We investigate the connection among various stopping rules. Afterward, we propose a new family of stopping rules which are extremely simple and accommodate some existing stopping rules as special cases. We provide rigorous justification for the feasibility and asymptotic optimality of such stopping rules. We prove that the prescribed confidence level can be guaranteed uniformly for all values of a binomial proportion by choosing appropriate parametric values for the stopping rule. We show that as the margin of error tends to zero, the sample size tends to the attainable minimum as if the binomial proportion were exactly known. We derive analytic bounds for distributions and expectations of sample numbers. In addition, we address some critical computational issues and propose methods to improve the accuracy and efficiency of numerical calculation. We conduct extensive numerical experiment to study the performance of various stopping rules. We determine parametric values for the proposed stopping rules to achieve unprecedentedly efficiency while guaranteeing prescribed confidence levels. We attempt to make our proposed method as user-friendly as possible so that it can be immediately applicable even for layer persons.

The remainder of the paper is organized as follows. In Section 2, we introduce the exact approach proposed in [7, 9, 11, 13, 14]. In Section 3, we discuss the general principle of constructing stopping rules. In Section 4, we propose a new family of sampling schemes and investigate their feasibility, optimality and analytic bounds of the distribution and expectation of sample numbers. In Section 5, we compare various computational methods. In particular, we illustrate why the natural method of evaluating coverage probability based on gridding parameter space is neither rigorous nor efficient. In Section 6, we present numerical results for various sampling schemes. In Section 7, we illustrate the applications of our group sequential method in clinical trials. Section 8 is the conclusion. The proofs of theorems are given in appendices. Throughout this paper, we shall use the following notations. The empty set is denoted by . The set of positive integers is denoted by . The ceiling function is denoted by . The notation denotes the probability of the event associated with parameter . The expectation of a random variable is denoted by . The standard normal distribution is denoted by . For , the notation denotes the critical value such that . For , in the case that are i.i.d. samples of , we denote the sample mean by , which is also called the relative frequency when is a Bernoulli random variable. The other notations will be made clear as we proceed.

## 2 How Can It Be Exact?

In many areas of scientific investigation, the outcome of an experiment is of dichotomy nature and can be modeled as a Bernoulli random variable , defined in probability space , such that

where is referred to as a binomial proportion. In general, there is no analytical method for evaluating the binomial proportion . A frequently-used approach is to estimate based on i.i.d. samples of . To reduce the sampling cost, it is appropriate to estimate by a multistage sampling procedure. More formally, let and , with , be the pre-specified margin of error and confidence level respectively. The objective is to construct a sequential estimator for based on a multistage sampling scheme such that

(1) |

for any . Throughout this paper, the probability is referred to as the coverage probability. Accordingly, the probability is referred to as the complementary coverage probability. Clearly, a complete construction of a multistage estimation scheme needs to determine the number of stages, the sample sizes for all stages, the stopping rule, and the estimator for . Throughout this paper, we let denote the number of stages and let denote the number of samples at the -th stages. That is, the sampling process consists of stages with sample sizes . For , define and . The stopping rule is to be defined in terms of . Of course, the index of stage at the termination of the sampling process, denoted by , is a random number. Accordingly, the number of samples at the termination of the experiment, denoted by , is a random number which equals . Since for each , is a maximum-likelihood and minimum-variance unbiased estimator of , the sequential estimator for is taken as

(2) |

In the above discussion, we have outlined the general characteristics of a multistage sampling scheme for estimating a binomial proportion. It remains to determine the number of stages, the sample sizes for all stages, and the stopping rule so that the resultant estimator satisfies (1) for any .

Actually, the problem of sequential estimation of a binomial proportion has been treated by Chen [7, 9, 11, 13, 14] in a general framework of multistage parameter estimation. The techniques of [7, 9, 11, 13, 14] are sufficient to offer exact solutions for a wide range of sequential estimation problems, including the estimation of a binomial proportion as a special case. The central idea of the approach in [7, 9, 11, 13, 14] is the control of coverage probability by a single parameter , referred to as the coverage tuning parameter, and the adaptive rigorous checking of coverage guarantee by virtue of bounds of coverage probabilities. It is recognized in [7, 9, 11, 13, 14] that, due to the discontinuity of the coverage probability on parameter space, the conventional method of evaluating the coverage probability for a finite number of parameter values is neither rigorous not computationally efficient for checking the coverage probability guarantee.

As mentioned in the introduction, Frey published an article [23] in TAS on the sequential estimation of a binomial proportion with prescribed margin of error and confidence level. For clarity of presentation, the comparison of the works of Chen and Frey is given in Section 5.4. In the remainder of this section, we shall only introduce the idea and techniques of [7, 9, 11, 13, 14], which had been precedentially developed by Chen before Frey submitted his original manuscript to TAS in July 2009. We will introduce the approach of [7, 9, 11, 13, 14] with a focus on the special problem of estimating a binomial proportion with prescribed margin of error and confidence level.

### 2.1 Four Components Suffice

The exact methods of [7, 9, 11, 13, 14] for multistage parameter estimation have four main components as follows:

(I) Stopping rules parameterized by the coverage tuning parameter such that the associated coverage probabilities can be made arbitrarily close to by choosing to be a sufficiently small number.

(II) Recursively computable lower and upper bounds for the complementary coverage probability for a given and an interval of parameter values.

(III) Adapted Branch and Bound Algorithm.

(IV) Bisection coverage tuning.

Without looking at the technical details, one can see that these four components are sufficient for constructing a sequential estimator so that the prescribed confidence level is guaranteed. The reason is as follows: As lower and upper bounds for the complementary coverage probability are available, the global optimization technique, Branch and Bound (B&B) Algorithm [29], can be used to compute exactly the maximum of complementary coverage probability on the whole parameter space. Thus, it is possible to check rigorously whether the coverage probability associated with a given is no less than the pre-specified confidence level. Since the coverage probability can be controlled by , it is possible to determine as large as possible to guarantee the desired confidence level by a bisection search. This process is referred to as bisection coverage tuning in [7, 9, 11, 13, 14]. Since a critical subroutine needed for bisection coverage tuning is to check whether the coverage probability is no less than the pre-specified confidence level, it is not necessary to compute exactly the maximum of the complementary coverage probability. Therefore, Chen revised the standard B&B algorithm to reduce the computational complexity and called the improved algorithm as the Adapted B&B Algorithm. The idea is to adaptively partition the parameter space as many subintervals. If for all subintervals, the upper bounds of the complementary coverage probability are no greater than , then declare that the coverage probability is guaranteed. If there exists a subinterval for which the lower bound of the complementary coverage probability is greater than , then declare that the coverage probability is not guaranteed. Continue partitioning the parameter space if no decision can be made. The four components are illustrated in the sequel under the headings of stopping rules, interval bounding, adapted branch and bound, and bisection coverage tuning.

### 2.2 Stopping Rules

The first component for the exact sequential estimation of a binomial proportion is the stopping rule for constructing a sequential estimator such that the coverage probability can be controlled by the coverage tuning parameter . For convenience of describing some concrete stopping rules, define

and

where and are integers such that . Assume that . For the purpose of controlling the coverage probability by the coverage tuning parameter, Chen has proposed four stopping rules as follows:

Stopping Rule A: Continue sampling until for some .

Stopping Rule B: Continue sampling until for some .

Stopping Rule C: Continue sampling until and for some .

Stopping Rule D: Continue sampling until for some .

Stopping Rule A was first proposed in [7, Theorem 7] and restated in [9, Theorem 16]. Stopping Rule B was first proposed in [11, Theorem 1] and represented as the third stopping rule in [10, Section 4.1.1]. Stopping Rule C originated from [13, Theorem 1] and was restated as the first stopping rule in [10, Section 4.1.1]. Stopping Rule D was described in the remarks following Theorem 7 of [8]. All these stopping rules can be derived from the general principles proposed in [14, Section 3] and [15, Section 2.4].

Given that a stopping rule can be expressed in terms of and for , it is possible to find a bivariate function on , taking values from , such that the stopping rule can be stated as: Continue sampling until for some . It can be checked that such representation applies to Stopping Rules A, B, C, and D. For example, Stopping Rule B can be expressed in this way by virtue of function such that

The motivation of introducing function is to parameterize the stopping rule in terms of design parameters. The function determines the form of the stopping rule and consequently, the sample sizes for all stages can be chosen as functions of design parameters. Specifically, let

(3) | |||

(4) |

To avoid unnecessary checking of the stopping criterion and thus reduce administrative cost, there should be a possibility that the sampling process is terminated at the first stage. Hence, the minimum sample size should be chosen to ensure that . This implies that the sample size for the first stage can be taken as . On the other hand, since the sampling process must be terminated at or before the -th stage, the maximum sample size should be chosen to guarantee that . This implies that the sample size for the last stage can be taken as . If the number of stages is given, then the sample sizes for stages in between and can be chosen as integers between and . Specially, if the group sizes are expected to be approximately equal, then the sample sizes can be taken as

(5) |

Since the stopping rule is associated with the coverage tuning parameter , it follows that the number of stages and the sample sizes can be expressed as functions of . In this sense, it can be said that the stopping rule is parameterized by the coverage tuning parameter . The above method of parameterizing stopping rules has been used in [7, 9, 11, 13] and proposed in [10, Section 2.1, page 9].

### 2.3 Interval Bounding

The second component for the exact sequential estimation of a binomial proportion is the method of bounding the complementary coverage probability for in an interval contained by interval . Applying Theorem 8 of [9] to the special case of a Bernoulli distribution immediately yields

(6) |

for all . The bounds of (6) can be shown as follows: Note that for . As a consequence of the monotonicity of and with respect to , where is a real number independent of , the lower and upper bounds of for can be given as and respectively.

In page 15, equation (1) of [9], Chen proposed to apply the recursive method of Schultz [32, Section 2] to compute the lower and upper bounds of given by (6). It should be pointed out that such lower and upper bounds of can also be computed by the recursive path-counting method of Franzén [21, page 49].

### 2.4 Adapted Branch and Bound

The third component for the exact sequential estimation of a binomial proportion is the Adapted B&B Algorithm, which was proposed in [9, Section 2.8], for quick determination of whether the coverage probability is no less than for any value of the associated parameter. Such a task of checking the coverage probability is also referred to as checking the coverage probability guarantee. Given that lower and upper bounds of the complementary coverage probability on an interval of parameter values can be obtained by the interval bounding techniques, this task can be accomplished by applying the B&B Algorithm [29] to compute exactly the maximum of the complementary coverage probability on the parameter space. However, in our applications, it suffices to determine whether the maximum of the complementary coverage probability with respect to is greater than the confidence parameter . For fast checking whether the maximal complementary coverage probability exceeds , Chen proposed to reduce the computational complexity by revising the standard B&B Algorithm as the Adapted B&B Algorithm in [9, Section 2.8]. To describe this algorithm, let denote the parameter space . For an interval , let denote the maximum of the complementary coverage probability with respect to . Let and be respectively the lower and upper bounds of , which can be obtained by the interval bounding techniques introduced in Section 2.3. Let be a pre-specified tolerance, which is much smaller than . The Adapted B&B Algorithm of [9] is represented with a slight modification as follows.

. |

. |

. |

. |

. |

. . |

. |

. |

It should be noted that for a sampling scheme of symmetrical stopping boundary, the initial interval may be taken as for the sake of efficiency. In Section 5.1, we will illustrate why the Adapted B&B Algorithm is superior than the direct evaluation based on gridding parameter space. As will be seen in Section 5.2, the objective of the Adapted B&B Algorithm can also be accomplished by the Adaptive Maximum Checking Algorithm due to Chen [10, Section 3.3 ] and rediscovered by Frey in the second revision of his manuscript submitted to TAS in April 2010 [23, Appendix]. An explanation is given in Section 5.3 for the advantage of working with the complementary coverage probability.

### 2.5 Bisection Coverage Tuning

The fourth component for the exact sequential estimation of a binomial proportion is Bisection Coverage Tuning. Based on the adaptive rigorous checking of coverage probability, Chen proposed in [7, Section 2.7] and [9, Section 2.6] to apply a bisection search method to determine maximal such that the coverage probability is no less than for any value of the associated parameter. Moreover, Chen has developed asymptotic results in [9, page 21, Theorem 18] for determining the initial interval of needed for the bisection search. Specifically, if the complementary coverage probability associated with tends to as , then the initial interval of can be taken as , where is the largest integer such that the complementary coverage probability associated with is no greater than for all . By virtue of a bisection search, it is possible to obtain such that the complementary coverage probability associated with is guaranteed to be no greater than for all .

## 3 Principle of Constructing Stopping Rules

In this section, we shall illustrate the inherent connection between various stopping rules. It will be demonstrated that a lot of stopping rules can be derived by virtue of the inclusion principle proposed by Chen [14, Section 3].

### 3.1 Inclusion Principle

The problem of estimating a binomial proportion can be considered as a special case of parameter estimation for a random variable parameterized by , where the objective is to construct a sequential estimator for such that for any . Assume that the sampling process consists of stages with sample sizes . For , define an estimator for in terms of samples of . Let be a sequence of confidence intervals such that for any , is defined in terms of and that the coverage probability can be made arbitrarily close to by choosing to be a sufficiently small number. In Theorem 2 of [14], Chen proposed the following general stopping rule:

(7) |

At the termination of the sampling process, a sequential estimator for is taken as , where is the index of stage at the termination of sampling process.

Clearly, the general stopping rule (7) can be restated as follows:

Continue sampling until the confidence interval is included by interval for some .

The sequence of confidence intervals are parameterized by for purpose of controlling the coverage probability . Due to the inclusion relationship , such a general methodology of using a sequence of confidence intervals to construct a stopping rule for controlling the coverage probability is referred to as the inclusion principle. It is asserted by Theorem 2 of [14] that

(8) |

provided that for and . This demonstrates that if the number of stages is bounded with respective to , then the coverage probability associated with the stopping rule derived from the inclusion principle can be controlled by . Actually, before explicitly proposing the inclusion principle in [14], Chen had extensively applied the inclusion principle in [7, 9, 11, 13] to construct stopping rules for estimating parameters of various distributions such as binomial, Poisson, geometric, hypergeometric, normal distributions, etc. A more general version of the inclusion principle is proposed in [15, Section 2.4]. For simplicity of the stopping rule, Chen had made effort to eliminate the computation of confidence limits.

In the context of estimating a binomial proportion , the inclusion principle immediately leads to the following general stopping rule:

Continue sampling until for some . | (9) |

Consequently, the sequential estimator for is taken as according to (2). It should be pointed out that the stopping rule (9) had been rediscovered by Frey in Section 2, the 1st paragraph of [23]. The four stopping rules considered in his paper follow immediately from applying various confidence intervals to the general stopping rule (9).

In the sequel, we will illustrate how to apply (9) to the derivation of Stopping Rules A, B, C, D introduced in Section 2.2 and other specific stopping rules.

### 3.2 Stopping Rule from Wald Intervals

By virtue of Wald’s method of interval estimation for a binomial proportion , a sequence of confidence intervals for can be constructed such that

and that for and . Note that, for , the event is the same as the event . So, applying this sequence of confidence intervals to (9) results in the stopping rule “continue sampling until for some ”. Since for any , there exists a unique number such that , this stopping rule is equivalent to “Continue sampling until for some .” This stopping rule is actually the same as Stopping Rule D, since for .

### 3.3 Stopping Rule from Revised Wald Intervals

Define for , where is a positive number. Inspired by Wald’s method of interval estimation for , a sequence of confidence intervals can be constructed such that

and that for and . This sequence of confidence intervals was applied by Frey [23] to the general stopping rule (9). As a matter of fact, such idea of revising Wald interval by replacing the relative frequency involved in the confidence limits with had been proposed by H. Chen [4, Section 4].

### 3.4 Stopping Rule from Wilson’s Confidence Intervals

Making use of the interval estimation method of Wilson [36], one can obtain a sequence of confidence intervals for such that

and that for and . It should be pointed out that the sequence of Wilson’s confidence intervals has been applied by Frey [23, Section 2, page 243] to the general stopping rule (9) for estimating a binomial proportion.

Since a stopping rule directly involves the sequence of Wilson’s confidence intervals is cumbersome, it is desirable to eliminate the computation of Wilson’s confidence intervals in the stopping rule. For this purpose, we need to use the following result.

###### Theorem 1

Assume that and . Then, Wilson’s confidence intervals satisfy for .

### 3.5 Stopping Rule from Clopper-Pearson Confidence Intervals

Applying the interval estimation method of Clopper-Pearson [18], a sequence of confidence intervals for can be obtained such that for and , where the upper confidence limit satisfies the equation if ; and the lower confidence limit satisfies the equation if . The well known equation (10.8) in [20, page 173] implies that , with , is decreasing with respect to and that , with , is increasing with respect to . It follows that

and

for . Consequently,

for . This demonstrates that applying the sequence of Clopper-Pearson confidence intervals to the general stopping rule (9) gives Stopping Rule C.

It should be pointed out that Stopping Rule C was rediscovered by J. Frey as the third stopping rule in Section 2, page 243 of his paper [23].

### 3.6 Stopping Rule from Fishman’s Confidence Intervals

By the interval estimation method of Fishman [19], a sequence of confidence intervals for can be obtained such that

Under the assumption that and , by similar techniques as the proof of Theorem 7 of [8], it can be shown that for . Therefore, applying the sequence of confidence intervals of Fishman to the general stopping rule (9) gives Stopping Rule A.

It should be noted that Fishman’s confidence intervals are actually derived from the Chernoff bounds of the tailed probabilities of the sample mean of Bernoulli random variable. Hence, Stopping Rule A is also referred to as the stopping rule from Chernoff bounds in this paper.

### 3.7 Stopping Rule from Confidence Intervals of Chen et. al.

Using the interval estimation method of Chen et. al. [17], a sequence of confidence intervals for can be obtained such that

and that for and . Under the assumption that and , by similar techniques as the proof of Theorem 1 of [12], it can be shown that for . This implies that applying the sequence of confidence intervals of Chen et. al. to the general stopping rule (9) leads to Stopping Rule B.

## 4 Double-Parabolic Sequential Estimation

From Sections 2.2, 3.2 and 3.7, it can be seen that, by introducing a new parameter and letting take values and respectively, Stopping Rules B and D can be accommodated as special cases of the following general stopping rule:

Continue the sampling process until

(11) |

for some , where .

Moreover, as can be seen from (10), the stopping rule derived from applying Wilson’s confidence intervals to (9) can also be viewed as a special case of such general stopping rule with .

From the stopping condition (11), it can be seen that the stopping boundary is associated with the double-parabolic function such that and correspond to the sample mean and sample size respectively. For and , stopping boundaries with various are shown by Figure 1.

For fixed and , the parameters and affect the shape of the stoping boundary in a way as follows. As increases, the span of stopping boundary is increasing in the axis of sample mean. By decreasing , the stopping boundary can be dragged toward the direction of increasing sample size. Hence, the parameter is referred to as the dilation coefficient. The parameter is referred to as the coverage tuning parameter. Since the stopping boundary consists of two parabolas, this approach of estimating a binomial proportion is refereed to as the double-parabolic sequential estimation method.

### 4.1 Parametrization of the Sampling Scheme

In this section, we shall parameterize the double-parabolic sequential sampling scheme by the method described in Section 2.2. From the stopping condition (11), the stopping rule can be restated as: Continue sampling until for some , where the function is defined by

(12) |

Clearly, the function associated with the double-parabolic sequential sampling scheme depends on the design parameters and . Applying the function defined by (12) to (3) yields

(13) |

Since is usually small in practical applications, we restrict to satisfy . As a consequence of and the fact that for any , it must be true that for any . It follows from (13) that , which implies that the minimum sample size can be taken as

(14) |

On the other hand, applying the function defined by (12) to (4) gives

(15) |

Since for any , it follows from (15) that , which implies that maximum sample size can be taken as

(16) |

Therefore, the sample sizes can be chosen as functions of and which satisfy the following constraint: