Coherence and elicitability

Coherence and elicitability

Johanna F. Ziegel The author would like to thank Paul Embrechts, Tilmann Gneiting and Fabio Bellini for discussions.
Address correspondence to Johanna F. Ziegel, University of Bern, Department of Mathematics and Statistics, Institute of Mathematical Statistics and Actuarial Science, Sidlerstrasse 5, 3012 Bern, Switzerland,

The risk of a financial position is usually summarized by a risk measure. As this risk measure has to be estimated from historical data, it is important to be able to verify and compare competing estimation procedures. In statistical decision theory, risk measures for which such verification and comparison is possible, are called elicitable. It is known that quantile based risk measures such as value at risk are elicitable. In this paper, Gneiting’s (2011) result of the non-elicitability of expected shortfall is extended to all law-invariant spectral risk measures unless they reduce to minus the expected value. Hence, it is unclear how to perform forecast verification or comparison. However, the class of elicitable law-invariant coherent risk measures does not reduce to minus the expected value. We show that it consists of certain expectiles.

Keywords: Coherent risk measures; Decision theory; Elicitability; Expected shortfall; Expectiles; Law-invariant risk measures; Spectral risk measures

1 Introduction

Value at Risk (VaR) is the most common risk measure used in banking and finance. The VaR at level is given by

where the financial position is a real-valued random variable, and is its cumulative distribution function. In this paper, a positive value of denotes a profit. The sign convention we have chosen for implies that extreme losses correspond to levels close to zero, and for , the risk will be non-negative. Since the influential paper of ArtznerDelbaenETAL1999 introduced coherent risk measures, VaR has frequently been criticized as a risk measure because it fails to be subadditive, and hence it is not coherent; see for example Acerbi2002. Other authors have pointed out the lack of VaR at level to account for the size of losses beyond the level (DanielssoEmbrechtsETAL2001). Median shortfall at level , or equivalently, VaR at level (KouPengETAL2013), does account for the size of losses beyond level .

The Basel2012 has been investigating the points in favor of and against a change of the regulatory risk measure from VaR to the coherent risk measure expected shortfall (ES), also known as average or conditional value at risk, which is defined by

From the perspective of coherent risk measures, ES is a better alternative to VaR. It remedies both problems mentioned above: It is a coherent risk measure, and it is sensitive to the sizes of the potential losses beyond the threshold . Other popular coherent risk measures are the so-called spectral risk measures, which generalize ES (Acerbi2002).

However, despite their theoretical appeal, there are also major drawbacks to using spectral risk measures in risk management, which should not be neglected. ContDeguestETAL2010 show that there is a fundamental theoretical conflict between subadditivity and robustness of risk measurement procedures for spectral risk measures; see also the related discussion in KouPengETAL2013. ContDeguestETAL2010 state

We hope to have convinced the reader that there is more to risk measurement than the choice of a ‘risk measure’: statistical robustness, and not only ‘coherence’, should be a concern for regulators and end-users when choosing or designing risk measurement procedures. The design of robust risk estimation procedures requires the explicit inclusion of the statistical estimation step in the analysis of the risk measurement procedure.

The next steps beyond estimation are backtesting and forecast verification. Backtesting refers to validating a given estimation procedure for a risk measure on historical data. In this paper, following the ideas of Gneiting2011, we consider risk measures from a forecasting perspective. With our knowledge of today, we are trying to give the best possible point estimate of the risk measure for tomorrow, or ten days ahead, or for any other time point in the future. There are numerous choices concerning models, methods and parameters that have to be made to come up with predictions. Hence, for a number of competing forecast or estimation procedures we would like to decide which one performs best. If we restrict our attention to law-invariant coherent risk measures as introduced by Kusuoka2001 we can view them as functionals on some set of probability distributions on . From the viewpoint of statistical decision theory not all functionals allow for meaningful point forecasts; see Gneiting2011. Functionals for which meaningful point forecasts and forecast performance comparisons are possible are called elicitable; see Section 2 for details. One important example of elicitable functionals are quantiles, hence VaR is elicitable.

Gneiting2011 has shown that ES is not elicitable, which may be a partial explanation for the difficulties with robust estimation and backtesting. This raises the natural question whether there is a different option. Is there any (interesting) law-invariant coherent risk measure that is also an elicitable functional? We show that the only law-invariant spectral risk measure that is also elicitable is minus the expected value:

see Corollary 4.3. However, there are law-invariant coherent risk measures that are elicitable. They are expectiles which were first introduced by NeweyPowell1987. The elicitability of expectiles is a simple corollary of their definition. They have been considered as a risk measure by KuanYehETAL2009. Proposition 4.4 shows that they are coherent risk measures. Very recently, and independently of our work, a proof of this result also appears in BelliniKlarETAL2013. Proposition 4.4 also identifies the minimal generating set of the Kusuoka representation as defined in PichlerShapiro2012. Expectiles are the only elicitable law-invariant coherent risk measures; see Section 4.3.

In the literature, there are procedures for evaluating ES forecasts and that allow for tests; see for example McNeilFrey2000; Christoff2003. However, these methods do not allow for a direct comparison and ranking of the predictive performance of competing forecasting methods (Gneiting2011).

The non-elicitability of spectral risk measures, and in particular of ES, is the reason that there is no analogue to the quantile regression method (Koenker2005) for these functionals, and no M-estimators can be constructed. ChunShapiroETAL2012 construct a mixed quantile estimator for ES as an approximation to an M-estimator. The recent contribution of RockafellRoysetETAL2013 takes this approach further and proposes a framework for ‘generalized’ regression that is suitable for ES. However, the problem with forecast comparison remains.

The paper is organized as follows. In Section 2 we introduce the notion of elicitability and describe its importance in point forecasting. A brief introduction to law-invariant coherent risk measures is given in Section 3. Section 4 contains the main results of the paper, showing in particular that law-invariant spectral risk measures are not elicitable, and elaborating the prominent role of expectiles as the only elicitable law-invariant coherent risk measures. We conclude the paper with a discussion; see Section 5.

2 Elicitability

Let be a class of probability measures on with the Borel sigma algebra. We consider a functional

where denotes the power set of . Often, but not always, is single valued, for example if we consider the expectation functional on the class of all probability measures with finite mean. However, quantile functionals may be set-valued. In the case of single valued functionals we will confound the one-point set with its unique element.

In this paper we are interested in the statistical properties of functionals that are law-invariant coherent risk measures; see Section 3. The following Definitions 2.1 and 2.2 are central in the context of point forecasting; see Gneiting2011 for a discussion of their historical background. Let be a real-valued random variable, which models the future observation of interest.

Definition 2.1.

A scoring function is consistent for the functional relative to the class , if


for all , all , and all . Here, has distribution . It is strictly consistent if it is consistent and equality in (1) implies that .

Given a consistent scoring function for a functional , an optimal forecast for is given by

Competing forecast procedures for can be compared using the scoring function . Suppose that in forecast cases we have point forecasts , , and realizing observations . The index numbers the competing forecast procedures. We can rank the procedures by their average scores

The consistency of the scoring rule for the functional ensures that accurate forecasts of are rewarded. On the contrary, evaluating point forecasts with respect to ‘some’ scoring function, which is not consistent for , may lead to grossly misguided conclusions about the quality of the forecasts. A drastic example is provided in the simulation study of Gneiting2011. Summarized in rough terms, one can construct realistic examples where the performance of skilful statistical forecasts is ranked worse than an ignorant no-change forecast when evaluated by ‘some’ scoring function, such as the absolute error or the squared error, for example. Therefore, point forecasts for a functional have to be evaluated by means of a scoring function, which is consistent for .

Definition 2.2.

A functional is elicitable relative to the class , if there exists a scoring function which is strictly consistent for relative to .

Many interesting functionals are elicitable and a wealth of examples is given in Gneiting2011. The most prominent example concerning risk management may be VaR, which is essentially a quantile and as such elicitable. The scoring functions that are consistent for -quantiles have been characterized by Thomson1979; Saerens2000; see also Gneiting2011. Subject to some regularity and integrability conditions, they are given by


where is an increasing function and denotes the indicator function.

However, not all functionals are elicitable, the most striking example in the present context being ES. The following necessary condition is due to Osband1985; see also LambertPennockETAL2008. As this theorem is central to the results presented in this paper, we provide a proof.

Theorem 2.1 (Osband).

An elicitable functional has convex level sets in the following sense: If and , and for some , then and imply .


Let be a strictly consistent scoring function for and let , , , and be as required in the Theorem. Then we obtain for any that

hence . ∎

3 Coherent risk measures

Let be a standard probability space without atoms. A coherent risk measure is a map , which fulfils the following four properties. It is monotone, so implies ; it is subadditive, that is for all ; it is positively homogeneous, i.e. for , it holds that . Finally, it is translation invariant in the sense that for all , we have .

Coherent risk measures were introduced by ArtznerDelbaenETAL1999; see also Delbaen2002 and FollmerSchied2004. All common coherent risk measures used in applications, share the property of law-invariance. That is, if and have the same distribution on , then

Law-invariant risk measures were characterized by Kusuoka2001. His result was strengthened by JouiniSchachermETAL2006. We summarize the result that is relevant for this paper in the following theorem; compare JouiniSchachermETAL2006. Let denote the set of all probability measures on with the weak topology. For a cumulative distribution function we define its generalized inverse or quantile function by

Theorem 3.1 (Kusuoka).

Let be a law-invariant coherent risk measure. Then there exists a closed convex set such that


and .

For , we define


Up to the sign, the are exactly the spectral risk measures of Acerbi2002. The following alternative representation of will be useful in the following. It is a direct consequence of Fubini’s theorem.


where is standard uniformly distributed, and is given by

Following PichlerShapiro2012 we call the spectral function of . It is left-continuous, decreasing and . This implies in particular that the functional is finite for all . We call the function the integrated spectral function of . As is law-invariant, we will also write if has distribution .

4 Coherence and elicitability

4.1 Coherent functionals with convex level sets

If a functional is not elicitable with respect to a class of probability distributions , then it cannot be elicitable with respect to any larger class . In particular, if the functional does not have convex level sets in the sense of Theorem 2.1 for some class , it will fail to have convex level sets for any larger class containing . A simple class of probability distributions, that has proven useful to show the violations of the necessary condition for elicitability in Theorem 2.1 is the class of two-point distributions, that is


where is the Dirac measure at the point .

The following theorem summarizes the main results of this paper.

Theorem 4.1.

Let be the class of two-point distributions on ; see (5). Let be a closed set of probability measures on , that does not contain . If the functional

with defined at (3), has convex level sets, then for all , and there exists a such that all probability measures


are contained in . Furthermore, for all measures , there exists a such that




The lower bound in (8) and the integrated spectral functions of are illustrated in Figure 1. We would like to give a short summary of the proof of Theorem 4.1. The details are deferred to Section 4.4.

For a two-point distribution it is possible to calculate that for some . It follows from the properties of as a risk measure that . The assumption is necessary to guarantee that for some we have . Exploiting the convexity of level sets as given by Theorem 2.1, first we show that for all , and then we derive an explicit formula for in terms of and .


Weber2006 also studied law-invariant risk measures with convex level sets. His motivation came from considerations of dynamic consistency of risk measures, which he shows to be closely related to convexity of level sets. Weber2006 is more general than Theorem 4.1 in the sense that not only coherent risk measures are considered and that it provides a characterization instead of a necessary condition. However, his result requires regularity assumptions on the risk measure under consideration, which we do not impose in this work. See also the remarks in Section 4.2 and 4.3.

4.2 Spectral risk measures

The first corollary to Theorem 4.1 shows that none of the coherent risk measures considered in PichlerShapiro2012 are elicitable, unless they reduce to minus the expected value.

Corollary 4.2.

Suppose the functional in Theorem 4.1 has convex level sets and there is a finite set , such that

then and


By Theorem 4.1 the closed set is uncountable unless . If , then the lower bound in (8) is , which is the integrated spectral function of . The claim follows directly from Dana2005. ∎

Now, it follows easily that elicitable spectral risk measures are essentially the expected value.

Corollary 4.3.

Spectral risk measures, other than minus the expected value, are not elicitable relative to any class of probability distributions that contains the two-point distributions.


It is a direct consequence of Theorem 2.1 and Corollary 4.2 that any spectral risk measure, which is not minus the essential infimum, is not elicitable unless it is minus the expected value. Therefore, it only remains to show that is not an elicitable functional relative to the class of two-point distributions. Suppose the contrary, and let be a strictly consistent scoring function. If almost surely for some , then and we obtain for all . If has distribution with and , we obtain

With and letting , we obtain , a contradiction. ∎


While the proof of Corollary 4.3 shows that is not elicitable relative to the class of two-point distributions, it is easy to check that the interval is elicitable. Strictly consistent scoring functions are given at (2) with and any strictly increasing function .

Gneiting2011 shows that ES is not elicitable with respect to any class of probability measures that contains the measures with finite support, or the finite mixtures of absolutely continuous distributions with compact support. In both cases the proof is done by showing a violation of the necessary condition of convex level sets given in Theorem 2.1. We believe that it is possible to modify the proof of Theorem 4.1 using mixtures of absolutely continuous distributions with compact support instead of two-point distributions. However, the details remain to be worked out and are likely to be rather technical.


While the result that ES is not elicitable is due to Gneiting2011, the non-convexity of its level sets already appears in Weber2006.

4.3 Expectiles

Theorem 4.1 provides an upper and a lower bound on a potentially elicitable law-invariant coherent risk measures via the provided restrictions on the integrated spectral functions. This is illustrated in Figure 1, and details are given below.

Let be a closed set such that

By Dana2005 equation (8) of Theorem 4.1 implies that there exists a such that

The map is a spectral risk measure with spectral function for . The associated measure has density on and a point mass at . The integrated spectral function is illustrated as a dashed line in Figure 1. By Corollary 4.3, the spectral risk measure is not elicitable unless . In this case, it reduces to minus the expected value.

We define

where is given at (6) and


By Theorem 4.1 we immediately obtain . Invoking Dana2005 equation (7) yields , hence . In the remainder of this section we characterize the law-invariant coherent risk measure .

As introduced in NeweyPowell1987, the -expectile , , of a random variable with finite mean is the unique solution to the equation


BelliniKlarETAL2013 show that (up to the sign) expectiles are law-invariant coherent risk measures for . As mentioned in introduction, expectiles are elicitable. The scoring functions that are consistent for -expectiles were recently characterized by Gneiting2011. Subject to some regularity and integrability conditions, they are given by

where is a convex function with subgradient . The prominent role of expectiles as the only elicitable law-invariant coherent risk measures is underlined by the following proposition.

Proposition 4.4.

The law-invariant coherent risk measure defined at (9) is minus the -expectile for


Let a random variable with finite first moment, , and its -expectile with . We define

We will show that and that is minimal at . If is continuous the latter claim can alternatively be shown by methods of calculus. We show the claims directly in order to avoid case distinctions.

For , we obtain with defined at (6)

where we used AcerbiTasche2002 in the second step.

NeweyPowell1987 show that . Using (10) we obtain


Let such that . Then, using equation (11) and partial integration, we obtain

The last term in the above equation is always non-negative. It vanishes for , hence for all . The argument for is completely analogous. ∎


Expectiles as coherent risk measures also appear implicitly in Weber2006. He shows that the shortfall risk measure with loss function for is coherent. Such a shortfall risk measure is equal to the minus the -expectile with . However, Weber2006 did not draw the connection to the expectiles (NeweyPowell1987) in the statistical literature. Under the additional regularity assumptions (3.1) and (1) of Weber2006, Weber2006 characterizes all coherent risk measures with convex level sets as minus -expectiles with . Theorem 4.1 shows that these conditions are not necessary for the characterization in the coherent case.

Figure 1: Integrated spectral functions. In both panels, the dashed lines are the integrated spectral functions of , and the solid lines are those of and as examples. For comparison, the dotted line is the integrated spectral function of .

4.4 Proof of Theorem 4.1

For each , , we define

Using Fubini we obtain


For all , the set

is a subset of the unit simplex in because . The set is also closed, which can be seen using Helly’s theorem, the fact that is closed, and the representation of of at (12). Let be the lower boundary point of ; see Figure 2 for an illustration. As is closed, the supremum is attained and there is an such that . Note that . Suppose the contrary, then

which implies and for , hence because is decreasing and non-negative. This is a contradiction because

The function is increasing. Define


If , then for all , hence . This implies for all , and hence converges weakly to as . As is closed this is a contradiction to the assumption . Therefore, . We will conclude later that, actually, .

Figure 2: Illustration of the construction in the proof of Theorem 4.1.

Let . For , the distribution function of has generalized inverse , . This yields using (4)


Let , . Then . All with are given by

for , and ; cf. Figure 2. Note that . Convexity of the level sets of implies that for all , all and all , we have



We have , and , hence it follows that for all . Equation (14) also implies that for all , . Suppose the contrary. Then there is a such that for all , which implies for some . Now (14) yields , which is a contradiction because . Going back to the definition of at (13) we obtain in particular that for all . Therefore, .

For we obtain


which yields

Monotonicity of yields

hence we obtain


Equation (15) implies that



and hence


for all . The left-hand side is increasing in , whereas the right-hand side is decreasing in . Both sides are left-continuous. This implies that

for two constants . We have

and by (17)


which yields


The inequality is equivalent to . By the definition of and (18) we obtain that

where , fulfil ,