A multicriteria selection system based on player performance. Case study: The Spanish ACB Basketball League
In this paper, we describe an approach to rank sport players based on their efficiency. Although is extremely useful to analyze the performance of team games there is no unanimity on the use of a single index to perform such a ranking. We propose a method to summarize the huge amount of information collected at different aspects of a sport team which is almost daily publicly available. The tool will allow agents involved in a player’s negotiation to show the strengths (and weaknesses) of the player with respect to other players. The approach is based on applying a multicriteria outranking methodology using as alternatives the potential players and criteria different efficiency indices. A novel automatic parameter tuning approach is detailed that will allow coaches and sports managers to design templates and sports strategies that improve the efficiency of their teams. We report the results performed over the available information on the ACB Basketball League, and we show how it can be easily implemented and interpreted in practice by decision-makers non familiar with the mathematical side of the methodology.
Key words and phrases:Multicriteria Decision Aids, Ranking Strategies, Players Efficiency, Agents Negotiation.
2010 Mathematics Subject Classification:90B50,62F07,97M40
Operations Research is one of the most active fields to help decision makers with objective tools for choosing the “best” of their alternatives based on the evaluation of the adequate criteria. A wide family of tools are available in the literature to make the best decision, each of them depending of the type of provided information, characteristic of the decision, number of agents involved on it or the number of criteria and alternatives, etc. Needless to say that in the current digital age in which the amount of available data is exponentially increasing, it is specially important to provide quantitative tools to allow practitioners an efficient management of such an information. In particular, in Sport Science, each team or individual player updates, almost daily, its available information. Furthermore, making the best decision instead of a nearly good may induce a significative economic or social gain to the teams, player agents, or any of the several agents involved in the Sport Industry. Hence, using adequate tools to summarize and manage the information is one of the most important tasks in Sport Management. Multicriteria Analysis allows one to consider at the same time several criteria to chose among the available alternatives. If the number of alternatives is of reasonable size several tools are available from the so-called discrete multicriteria/multiattribute analysis, that allow one to rank, at least partially, the alternatives (see  and , among many others). Observe that, unless a preference weight is provided for each of the criteria that allows to unify the results of all of them into a single utility value, for a given alternative, the best result for a criterion may not coincide with the best choice for other criteria.
In particular, the goal of this paper is to adapt one the available ranking tools to measure the performance of players, based on the available criteria, as the annotated points, playing time or number of tries, and many others. Agents of basketball players highlighting the value of its players when negotiating with club presidents, would be helped with quantitative measures about the player on its current team (see ). Also, coaches and managers of sports teams design the lineup of players, and at the end of each season they evaluate the continuation of the players as well as the incorporation of new talents to complement and improve the team. Even for short-terms, the coaches select the most suitable players for a game or part of it. In order to make the decisions, among other factors, they are provided with a lot of information and the correct decision is expected by analyzing the available data. A deeper understanding of resources implies an adequate decision and then an improvement of the performance of the team (see  and Drust), since it allows one to detect the strengths and weaknesses of the players and the whole team.
One of the main differences between basketball teams and other sports comes from the availability of information. One can find data about each individual player or team for a game, from the number of scored points to the completed possessions. Hence, the detection of factors implying success in a basketball game is particularly hard since it depends on the accomplishment of numerous game actions (see ). Hence, in this framework, it is crucial to analyze the performance of teams and players by summarizing the available information. Thus, several desirable skills for players selection should be adequately selected to be summarized. However, although different indicators have been proposed in the literature, there is not an unified measure for the performance of a basketball player, the most popular being the TENDEX index , the ratio of performance-performance index rating used in competitions of European basketball (roughly, the difference between positive and negative action games), the Player Efficiency Rating the (PER) of Hollinger, or different valuation plus/minus rates . Note that an index to measure the performance of a player must take into account the characteristics of the player, its experience in the game, its profile (offensive or defensive), etc (see ), and then, an unified indicator is not possible for the whole set of players. Some discussions about the suitability of each of the proposed measures has been partly addressed in Berri and Bradbury, Lewin and Rosenbaum and Winston, among others.
Although the selection of the adequate or more representative indices is highly interesting in sport science, as stated in , computing an global efficiency index is difficult due to the variety of individual statistics. In , the authors recommen to limit the use of individual statistics for predicting the final ranking of their teams. We propose to turn this disadvantage into a strength by considering a different perspective to provide a preference ranking of the players based on using all the available information. More concretely, we use tools borrowed from Multicriteria Analysis and adapt them to help managers and/or coaches in the players’ selection task or for agents of player that wishing to highlight their clients’ strengths over other players. As far as we know, these tools have not been applied in this way to analyze sport players.
In particular, we adapt the PROMETHEE method, introduced by Brans . The PROMETHEE method have been successfully applied in several fields, as energy sustainability  or Education . It allows one to aggregate several criteria, whose results are known for each given player and season, into one or two indices, the so-called net flow preference indices. With this method we provide a support tool for practitioners which are not familiar which the quantitative techniques the approach, but that can be intuitively used. Hence, describe a method for selecting players which complement the usual techniques in this framework. One of the main highlights of our approach is that we propose a simple automatic strategy to determine the parameters needed for the application of the PROMETHEE methodology: the thresholds and the weights. This will allow agents and coaches which are not familiar with the quantitative tools to easily use and interpret the results derived from the application of the approach.
We apply the methodology to the case study of the 191 players who participated in the Spanish ACB Basketball League during the 2014-2015 season. We construct a ranking system by classifying the players into five different profiles (positional classification) and analyze two different scenarios for each of them. Different graphical representations of the results are shown which allow for a better understanding of the insights of our approach.
2. The outranking methodology
MultiCriteria Decision Aid (MCDA) allows one ranking a given finite set of alternatives based on multiple (probably) conflicting criteria. Several MCDA approaches have been proposed in the literature (see, for instance, ) and they have been applied to real-world problems in many different areas, as management or engineering, amongst others. The main problem and motivation, in the development of methods where different criteria are involved in the decision process, is that the best in one criterion may imply a worsering in other,. Hence, one is not able to determine which is the best or worst alternative. In general, only a partial order is possible with such an information. One of the choices in this framework is to provide a set of weights of relative importance for each criterion, and then, if the measures are of quantitative nature, one may compute an overall index to summarize the given criteria. However, this is in general not possible, and when it is, it is not easy to implement. One of the methods designed to rank a set of alternatives based on different criteria is the PROMETHEE method (Preference Ranking Organization Method for Enrichment Evaluations), that was introduced by Brans . There, two first versions of the method are provided, namely, PROMETHEE I and PROMETHEE II. Several extensions have appeared since then, from PROMETHEE III, for ranking based on interval, to PROMETHEE CLUSTER for nominal classification, in a series of paper mostly coauthored by Brans and Mareschal[6, 7, 8, amongst other]. Also, there has appeared some recent developments related to the PROMETHEE method as in the one by Soylu[soylu], in which a Tchebycheff agreggation technique is applied, the survey by Saaty and Ergu[saaty-ergu] where several multicriteria methods are compared in order to make a right decision or the combination of the SMAA and PROMETHEE to determine adequate weights and thresholds , to mention a few. The importance of the method in the Operations Research community is also reflected in that there is an available software, Visual PROMETHEE, that allows one to perform all the computations. Actually, the flexibility and applicability in many contexts of PROMETHEE methodologies have made it more and more popular in the recent years (see Brans and Mareschal and the references therein for a complete overview of the method).
To be self-contained we detail the general performance of PROMETHEE methods. Let us consider a finite set of alternatives and a set of criteria to be, without loss of generality, maximized to make a decision over the set of alternatives, . Every alternative is assumed to be evaluated under each of the criteria , with value , for , . The PROMETHEE paradigm is based on the principle that the decision maker may manifest some subjective information over the criteria that allows for evaluating the results under its own perspective. Each pair of alternative is then compared for each of the criteria, computing the absolute difference on their results:
Those differences are now evaluated using one of the six classical preference functions which are detailed in Table 1. A preference function for the -th criterion is a mapping that measures the intensity of agreement with the statement is better than according to criterion . A -value of will imply that the alternatives are indifferent while if the -value is , it will imply that one of the alternatives (the one with larger result) is highly preferred to the other. Intermediate values in determine the degree of preference between the two alternatives.
|Type I: Usual|
|Type II: U-shape|
|Type III: V-shape|
|Type IV: Level|
|Type V: V-shape with Indifference|
|Type VI: Gaussian||
Observe that each of the preference criteria depends on some thresholds that allows the decision maker to incorporate its preferences. In particular, the -parameter, usually called indifference threshold, indicates an upper bound (in terms of the difference of results for a pair of alternative under a criterion) for indifference between alternatives. On the other hand, the -parameter, so-called the preference threshold, is a lower bound to model whether one of the two alternatives is clearly preferable to a second one under the given criterion. The most popular type of preference function for quantitative information is Type V (V-shape with indifference criterion). This function is parameterized with two values: the indifference () and the preference () threshold. The correct specification of the set of thresholds from which two alternatives are considered equal or different is the other important decision faced by the decision maker with this methodology. Different alternatives for fixing the threshold values are possible. One of them is based on the decision-maker experience, who set the values using its own subjective opinion. Other, is to provide a mechanism to automatically compute them from using the given sample. In what follows we describe a methodology to specify the thresholds by using some descriptive indices from the particular instance under the study.
Let us consider one criterion, , which is assumed to be evaluated for all the alternatives. Let be the set of differences between all distinct alternatives for that criterion. We define the indifference and preference parameters as follows:
where denotes the quantile, for .
Note that this choice assures, among all the possible differences of the values of the criterion for all the alternatives (roughly, ), of them are classified as not indifferent nor prioritary, while and of the differences are considered as indifferent and preferential, respectively. This settings allows one to control that of combinations of two players are in the positive slope side of the Type V graph (see Table 1).
Once a preference criteria is specified for each criterion, , with their respective thresholds, for each of the criteria in the decision process, they are applied to the previously computed differences. Thus, the absolute differences are translated into preference values in . Then, with those values, for each pair of alternatives a preference index is computed as follows:
where represent the importance level of each criterion. It is commonly assumed that to represent proportionality of importance of the criteria in the decision making process.
The preference flows are computed as:
Observe that is the average level of preference of the alternative with the given decision maker specifications, with respect to the rest of the alternatives. On the other hand, is the average level of preference of the rest of the alternatives with respect to . With those values, the PROMETHEE I method identifies if an alternative is preferable to if and only if and (with at least one of the inequalities being strict). This first method allows us to partially rank the alternatives. Note that if one of the two conditions does not hold, the alternative are not comparable with this approach.
Finally, the PROMETHEE II method uses the so-called net flow index:
With such a flow, it is said that if preferable to if and only if .
Finally, we summarize the elements needed to applied the proposed approach for ranking players based on the efficiency. A particular choice of the input information will be described in our case study.
Alternatives: Players of a team during one or more seasons. In many sports is convenient to split players according to their positions, since the performance measure differ on their importance in the final ranking decision.
Criteria: For each of the alternatives (players), we must be able to measure two or more criteria which make impact on the efficiency of the player. An adequate choice of the criteria is crucial for a satisfying result, since they all put together conform the ranking system. They are usually chosen by experts (as agents or coaches) which has some evidence about which measures of efficiency are crucial in determining the a global efficiency measure, but also they must be in concordance with the available information.
Aggregation weights: Other aspect where the decision-maker may incorporate its expertise to the methodology is on the establishment of the importance weights of each of the considered criteria. A very simple choice is to consider that all criteria are equally important. However, it is very usual to consider that some criteria are more relevant than others, with respect to the impact in the global efficiency. This is the case when the decision maker’s goal is to rank a particular profile of alternatives, in which the same criteria are considered, but some of them are more important than other for the different play positions.
Shape of the preference functions: The decision maker should be able to choose one of the criteria in Table 1, to measure the degree of preference/indifference between players. The determination of the thresholds can be performed using the approach described in Section 2, just by fixing the values of and . Other options are also possible, as the use of the SMAA method described in .
Using the above information, provided by the decision maker, one can apply the described methodology to obtain a partial (PROMETHEE I) or total (PROMETHEE II) ranking system. In order to illustrate the proposed approach, in the next section, we describe its application for ranking basketball players of different typologies.
3. Case Study: Spanish ACB Basketball Players
In this section we apply the method described above to rank basketball players of different typologies based on several criteria. All the data used for the case study is available upon request for the interested readers.
The input data for this case study are the following:
Alternatives: We consider as alternatives each of the 191 players who participated in the Spanish ACB Basketball League during the 2014-2015 season. For the sake of data consistency and availability, we do not consider players that did not played at least 10 games and an average of at least 10 minutes. These requirements are usually assumed in the literature for analyzing the quality of basketball players (see Berri et al Berri et. al., Berri and Krautmann, Cooper et. al.). The data were obtained from the official database of the Spanish ACB Basketball League (http://www.acb.com). In order to compare and adequately weigh the criteria, the players were split based on their play positions into five categories: point guards (42), shooting guards (38), forwards (33), power-forwards (36) and centers (32).
Criteria: A set of 6 evaluation criteria have been considered in the case study for each player. We use the following indices as criteria for the proposed methodology, which, as far as we know, have not been previously used:
- PtsM: :
Ratio of points scored by the player with respect to the number of minutes played:
where and are, respectively, the points and minutes played by the player.
- DRM: :
Ratio of the player defensive rating with respect to the number of minutes played:
where , , and are, respectively, the defensive rebounds, steals, blocks and personal fouls made by the player.
- ORM: :
Ratio of the player offensive rating with respect to the number of minutes played:
where , , , , , , , , , and are, respectively, the 2-points field goals, 3-point field goals, free throws, offensive rebounds, assists, personal fouls received, field goal attempts, field goals (includes both 2-point field goals and 3-point field goals), free throw attempts, turnovers and blocks received by the player.
- EPts: :
Points efficiency measured as the ratio of points scored by the player with respect to the points that could have scored depending on the shots made and multiplied by 100:
where and are, respectively, the 2-point field goal attempts and 3-point field goal attempts made by the player.
- ASTM: :
Ratio of the number of assists and steals for every turnovers with respect to the number of minutes played:
- PCS %: :
Ratio between the number of possessions completed successfully (PCS, when a player makes a field goal, receives a personal foul or assists) and the number of completed possessions (PC, when a player attempts a field goal, receives a personal foul, assists or lose a ball) multiplied by 100:
Note that we split players by their game position since each profile has very particular characteristics, implying a significative difference in some of the measured criteria. In this way, we avoid bias between players due to the used criteria. For instance, point guard players usually get higher values in the ASTM index, while pivots obtain higher results in the DRM index, because its game position. This assertion was statistically checked and shown in Table 2, where we compute the average of each criterion for each player’s profile and we apply the ANOVA test to those values. The results indicates that means are statistically different for criteria DRM, ORM, EPts, ASTM and PCS %.
Aggregation weights: For illustrative purposes, we consider the following two different scenarios in our case study:
Scenario 1: All the criteria are identically weighted.
Scenario 2: Two criteria are overweighted for each of the players position clusters with the following strategy. From the difference between scored and received points by his team while it is on court (plus/minus, PM) weighted by minutes played divided by duration of the game, normally 40 minutes:
and considering that is positively affect to the player value for all players profiles, we will consider that the above criteria that are positively related to must be higher weight. In Table 4 are shown the correlations between all criteria and by player position (the correlations significantly different from zero to the level of significance of 5% are boldfaced). As can be observed, high values for and (for exterior players) and high values for and (for inside players) imply high values for . Thus, we will consider the following weights:
Points guards: the weights for y are set to , while the rest of the criteria are set to .
Centers: the weights for y are set to , while the rest of the criteria are set to .
Shape of the preference functions: The Type V (V-Shape with indifference Criterion) preference function was consider to compare players for all the criteria. Such a function was chosen because the quantitative nature of the measure and also because its flexibility. We apply the method described in Section 2, to determine the preference and indifference threshold, by considering and . This particular choice allows us, when comparing players, to get of the comparison resulting in high preference, and in indifferent. The remainder of the comparison result on preference degrees in , which gives some flexibility to the decision process and allows to distinguish between players. The thresholds obtained with this choice to all the set of players and criteria are shown in Table 3.
|Point guard||Shooting guard||Forward||Power-forward||Center|
The flows obtained are for each of the two scenarios and each of the two profiles of players are shown in Table 5 (Point Guards) and Table 6 (Centers). The PROMETHEE I graphs which can be built using the preference flows are drawn in Figures 1-4.
As can be observed from Figures 1 and 2, the most preferable point guard player under the two scenarios is Satoranski. This player obtained the maximum values for three of the six considered criteria (ORM, EPts and PSC%) and his results are among the best 25% for the the remainder criteria, justifying the top 1 ranking in this profile. On the other hand, Llull obtained a clear 2nd position under Scenario 1, while this position was obtained by V. Rossom under Scenario 2. This change of positions can be explained after a simple analysis of the absolute data obtained by each player:
where it can be observed that the criteria whose weights have been increased in Scenario 2 (EPts and ASTM) where those in which Rossom got better results that Llull. A similar consideration can be established for Markovic and Schreiner whose results in ASTM where the highest, increasing its preference under Scenario 2 with respect to those obtained under Scenario 1. Observe that this analysis which seems to be straightforward when comparing two players, can be cumbersome when analyzing the whole sample (42 players).
With respect to the Center players (Figures 3 and 4), Tomic was ranked in the first position under the two scenarios. Some remarkable considerations when comparing the rankings obtained with the two scenarios are the following:
Ayón considerably increases its ranking position under Scenario 2 w.r.t the one obtained under Scenario 1. It follows since the criteria whose weights were increased (DRM and ASTM) under Scenario 2, were those for which Ayón was among the top 5 players with best results.
T. Pleiss was ranked among the best players under Scenario 1, since it is the one with best result in EPts, decreases significatively its position under Scenario 2, mainly because the weight for EPts was 0.04 under this scenario.
Observe that in some cases the methodology PROMETHEE I does not give us a total order over the players. To obtain an strict ranking, we use the methodology PROMETHEE II, which aggregate the positive and negative preference flows into a single measure (net preference flow). Hence, although in some case only little differences, this approach allows us to strictly rank the players.
For instance, under Scenario 1, V. Rossom and S. Rodríguez (Point guard players) or Lima and Savané (Center players) are among those cases. PROMETHEE I is not able to determine which dominates (the positive flow is greater and the negative flow is smaller for a player) but PROMETHEE II states that V. Rossom and Lima obtained (slightly) larger net flow values than S. Rodríguez and Savané, respectively (see preference flows for center and guard players in Tables 5 and 6).
|Scenario 1||Scenario 2|
|T. Satoransky||0.7222||0.7343||0.0120||T. Satoransky||0.7246||0.7378||0.0132|
|V. Rossom||0.4176||0.4932||0.0756||S. Rodríguez||0.4288||0.4647||0.0359|
|S. Rodríguez||0.4069||0.4425||0.0356||M. Huertas||0.4281||0.4629||0.0348|
|San Miguel||0.2476||0.3242||0.0767||M. James||0.2742||0.3518||0.0777|
|T. Bellas||0.2336||0.3115||0.0779||V. Sada||0.1886||0.3638||0.1753|
|A. Oliver||0.2275||0.3172||0.0896||San Miguel||0.1861||0.2859||0.0998|
|R. Neto||0.1522||0.2652||0.1130||Q. Colom||0.1613||0.2696||0.1084|
|R. Luz||0.1056||0.2573||0.1517||R. Neto||0.1335||0.2680||0.1345|
|C. Cabezas||0.0912||0.2342||0.1429||D. Perkins||0.1108||0.3212||0.2104|
|V. Sada||0.0819||0.2795||0.1976||D. White||0.0970||0.2590||0.1620|
|R. López||-0.0456||0.1771||0.2227||C. Cabezas||-0.0087||0.2057||0.2144|
|A. Hernández||-0.1259||0.1710||0.2970||P. Pozas||-0.1626||0.2455||0.4081|
|D. Pérez||-0.3194||0.0841||0.4035||D. Pérez||-0.3531||0.0771||0.4302|
|R. Grimau||-0.3683||0.0814||0.4497||R. Grimau||-0.4150||0.0785||0.4935|
|J. Mayo||-0.6521||0.0364||0.6885||A. Rodríguez||-0.5272||0.0889||0.6161|
|A. Rodríguez||-0.7129||0.0411||0.7541||J. Mayo||-0.6491||0.0320||0.6811|
|R. Huertas||-0.8457||0.0020||0.8477||R. Huertas||-0.8907||0.0044||0.8951|
|Scenario 1||Scenario 2|
|A. Tomic||0.5818||0.6095||0.0277||A. Tomic||0.6721||0.6855||0.0134|
|F. Vázquez||0.3039||0.3843||0.0804||W. Tavares||0.2707||0.3932||0.1225|
|W. Tavares||0.1986||0.3307||0.1321||C. Iverson||0.1333||0.2577||0.1244|
|M. Begic||0.1128||0.2689||0.1561||W. Hernángomez||0.1205||0.2584||0.1379|
|C. Iverson||0.0967||0.2464||0.1497||F. Vázquez||0.0828||0.2192||0.1364|
|H. Rizvic||-0.0986||0.1847||0.2834||L. Williams||-0.0481||0.1792||0.2274|
|J. Triguero||-0.1352||0.1751||0.3103||H. Rizvic||-0.1048||0.1224||0.2272|
|K. Tillie||-0.2395||0.1567||0.3962||M. Diagne||-0.2539||0.0538||0.3077|
|G. Bogris||-0.2961||0.0930||0.3892||N. Jawai||-0.3060||0.0656||0.3716|
In this paper we present a flexible quantitative tool that allows us to aggregate some of the well-known indices to measure the basketball players’ skills, to provide either a partial (but richer) or total ranking on the players. The methodology is based on the use of the multicriteria method PROMETHEE. The PROMETHEE methodologies have the advantage that they do not consider the single player information, but the role of the player in the whole context, using the differences of the results between each two players instead of the single absolute value of the criterion for each player. Although this method is based on the interaction between the provided information and the experience provided by the decision maker, we propose an alternative approach to determine the values of some of the parameters which are needed for the use of the PROMETHEE methods.
One of the highlights of the method is that the results can be presented using a graph, and that can be easily interpreted by practitioners non familiar with the mathematical tool. That is, this work shows that performance data can be used by agents, managers or coaches, by adapting their needs and choices on what type of players they have or need in different seasons. That is, by using our approach, coaches are able to obtain very useful information to help them in the decision of signing a player up, or in case of players and agents, such an information allows them to contextualize the player’s performance, helping them to improve their efficiency or to get better contracts.
It is left for further research the incorporation of economic (budget constraints) or qualitative (like leadership or discipline) information of the players, either as one of the criterion or using a post-process optimization tools that, based on the PROMETHEE results allows us to adapt the “best” players to the team budget. Also, in , the combination of Stochastic Multicriteria Acceptability Analysis (SMAA) and the PROMETHEE method to determine the adequate weights for the decision. The application of such an approach will be the topic of a forthcoming paper.
The first and second authors were partially supported by project PP2016-PIP06 (Universidad de Granada) and the research group SEJ-534 (Junta de Andalucía). The first author was also partially supported by project MTM2016-74983-C2-1-R (MINECO, Spain).
-  Alamar, B.C. (2013). Sports analytics: A guide for coaches, managers, and other decision. Columbia Univ. Press.
-  Berri, D.J. and Bradbury, J.C. (2010). Working in the land of metricians. J. Sports Economics, 11(1), 29–47.
-  Berri, D.J., Brook, S.L. and Schmidt, M.B. (2007). Does One Simply Need to Score to Score?, Int. J. Sport Finance, 2, 142–148.
-  Berri, D.J. and Krautmann, A.C. (2006). Shrinking on the court: testing for the incentives effects of guaranteed pay, Economic Inquiry, 44(3), 536–546.
-  Brans, JP and Vincke, Ph (1985). A preference ranking organisation method: The PROMETHEE method for MCDM. Management Science, 31(6):647â656, 1985.d’Avenir, 183–213,Presses de L’Université Laval.
-  Brans, J.P. and Mareschal, B. (1992). Promethee-V: MCDM problems with segmentation constraints. INFOR, 30(2):85–96.
-  Brans, J.P. and Mareschal, B. (1994). The Promcalc and Gaia decision-support system for multicriteria decision aid. Decision Support Systems, 12(4-5):297–310.
-  Brans, J.P. and Mareschal, B. (1995). The PROMETHEE VI procedure. How to differentiate hard from soft multicriteria problems. Journal of Decision Systems, 4:213–223.
-  Brans, J.P. and Mareschal, B. (2016). PROMETHEE Methods. In S. Greco, J. Figueira, and M. Ehrgott, editors, Multiple Criteria Decision Analysis: State of the Art Surveys, pages 187â219. Springer, Berlin, 2016.
-  Cooper, W.W., Ruiz, J.L. and Sirvent, I. (2009). Selecting non-zero weights to evaluate effectiveness of basketball players with DEA, Eur. J. Oper. Res., 195, 563–574.
-  Corrente, S, Figueira, J.R., and Greco, S. (2014). The SMAA-PROMETHEE method. European Journal of Operational Research, 239(2):514â522,.
-  Decision Lab 2000 (1999). Getting Started Guide. Visual Decision Inc. Montreal, Canada. Replaced by Visual PROMETHEE. http://www.promethee-gaia.net.
-  Drust, B. (2010). Performance analysis research: Meeting the challenge. J. Sports Sciences, 28(9), 921–922.
-  A. Greco, J.R. Figueira, and M. Ehrgott (2016). Multiple Criteria Decision Analysis: State of the Art Surveys. Springer.
-  Gutiérrez, O. and Ruiz, J.L. (2013). Data envelopment analysis and cross-efficency evaluation in the management of sport teams: the assessment of game performance of players in the Spanish handball league. J. Sport Management, 27(3), 217–229.
-  Heeren, D. (1992). Basketball Abstract, New Jersey: Prentice Hall.
-  Jochemczyk, L. W., and Nowak, A. (2010). Constructing a network of shared agreement: A model of communication processes in negotiations. Group Decision and Negotiation, 19(6), 591-620.
-  Lewin, D. and Rosenbaum, D.T. (2007). The pot calling the kettle black. Are NBA statistical models more irrational than “irrational” decision-makers?. New England Symposium on Statistics in Sports. Harvard University Science Center.
-  Mertz, J., Hoover, L.D., Burke, J.M., Bellar, D., Jones, M.L., Leitzelar, B. and Judge, W. L. (2016). Ranking the greatest NBA players: a sport metrics analysis. International Journal of Performance Analysis in Sport, 13 (3), 737–759.
-  Montajabiha, M. (2016). An extended PROMETHE II multi-criteria group decision making technique based on intuitionistic fuzzy logic for sustainable energy planning. Group Decision and Negotiation, 25(2), 221-244.
-  Oliver, D. (2004). Basketball on paper. Rules and tools for performance analysis. Washington, D. C.: Brasseys, INC.
-  Puente, C., Del Coso, J., Salinero, J.J. and Abián-Vicén, J. (2015). Basketball performance indicators during the ACB regular season from 2003 to 2013. International Journal of Performance Analysis in Sport, 15 (3), 935–948.
-  B. Roy. Multicriteria Methodology for Decision Aiding. Nonconvex Optimization and its Appli- cations. Kluwer Academic Publishers, Dordrecht, 1996.
-  Saaty, T.L. (1980). The analytic hierarchy process: planning, priority setting, resource allocation. McGraw-Hill, US.
-  Winston, W.L. (2009). Mathletics. New Jersey: Princeton Univ. Press.
-  Ziv, G., Lidor, R. and Arnon. M. (2010). Predicting team rankings in basketball: the questionable use of on-court performance statistics. International Journal of Performance Analysis in Sport, 10 (2), 103–114.
-  Živković, Ž., Nikolić, D., Savić, M., Djordjević, P., and Mihajlović, I. (2017). Prioritizing Strategic Goals in Higher Education Organizations by Using a SWOTâPROMETHEE/GAIAâGDSS Model. Group Decision and Negotiation, 26(4), 829-846.