Editorial Board

A Refereed Monthly International Journal of Management

Assessment of Macroeconomic Performances and Human Development Levels of BRICS and MINT Countries Using CRITIC and COPRAS Methods

 

Talip Arsu

Assistant Professor,

University of Aksaray,

Vocational School of Social Sciences,

Department of Tourism and Hotel Management,

Türkiye

taliparsu@aksaray.edu.tr

ORCID: 0000-0002-2580-166X

 

 

 

Abstract

Generally utilizing the macroeconomic data of the countries, their economic superiority is assessed compared to others. However, the status of the economics of the country or how superior the related country from the others do not provide, by itself, sufficient information regarding the human development of the people in the country. Thus, besides the macroeconomic performance data of the countries, also the Human development level data are used in this study and so the assessment of the countries is aimed. In the evaluation conducted, CRITIC, among the objective weighting methods, is used and the criteria are weighted, then these weight are utilized and the countries are assessed according to the COPRAS method. In the assessment held according to three different scenarios, the countries assessed both by using the macroeconomic performance and human development level data together and also by using  the macroeconomic performance and human development level data separately (scenario 1- all criteria, scenario 2 – human development level criteria, scenario 3 – macroeconomic performance criteria). The BRICS and MINT which are at a trend of economic development are selected as sample to assess whether this development trend brings also the Human Development Index (HDI). In the first scenario where all the criteria are used and third scenario where just the macroeconomic performance indicators are used as criteria, according to the results of the CRITIC method, the most important criteria are defined in order as economic growth, unemployment rates and inflation rate. In the second scenario, based on the weights of the criteria, the most important criteria are found in order as mean years of schooling, the life expectancy at birth and GNI per capita. These weights determined are used within the COPRAS method and the most successful countries are found in order, according to the scenario 1 and scenario 3, as China, Russia and Indonesia. According to the scenario 2, the most successful countries are found in order as Russia, Turkey and Mexico.

Keywords: BRICS and MINT Countries, CRITIC, COPRAS, Macroeconomic Performance, Human Development Index

Introduction

In recent years, country groups named differently at various regions of the world are driving the global economy. Among such groups, the European Union states are the ones drawing most attention both economically and politically. In addition, the economies that are rapidly growing in recent years, that can draw the interest of most of the foreign investors and have rapidly increasing powers in influencing the globe are started to be grouped differently (BRIC; N-11; CIVETS; MINT; MIST; EAGLEs; 3G; SAMI). Among these, BRIC countries (Brazil, Russia, India, China) draw attention as the group that creates high growth opportunity due to cheap labor and thus low production costs, that can draw directly in the foreign capital investments, that increases its export, foreign currency reserves and national income, that is effected less from the economic crises faced (Ağır & Yıldırım, 2015: 40). The concept of BRIC countries is used first in the “Building Better Global Economic BRICs” report prepared by Jim O’Neill from the International Investment Bank Goldman Sachs. In this report, the BRIC countries are referred by using a resemblance to the word brick. According to the report, it is stipulated that the block created from bricks shall replace G7 in time and until the year 2050, BRIC countries shall be the most important actors of world economics. BRIC countries, holding their first summit in the year 2009, via inviting South Africa in the summit in 2011 became BRICS countries (Akgemci, 2011: 3). Due to the threats and opportunities represented by the BRICS countries in economic, social and political aspects, it is believed that they have the capacity to “change the world” (Bornmann et al., 2015: 1507).

It can be seen that O’Neil qualifies Mexico, Indonesia, Nigeria and Turkey which he did not include in the group when he created the BRICS term but in fact assessed in the same regard, as “climbing economic giants” as of the beginning of 2014 and defines them as a new group competing the BRICS with the acronym “MINT” (Yalçınkaya & Temelli, 2014: 203). There are some common properties playing a role in taking MINT countries as a separate group. First of all, compared to the aging and narrowing populations of many developed countries and China, they have relatively large, young and growing populations. Secondly, Indonesia positioned close to China, Turkey neighboring the European Union, Mexico at the doorsteps of America and Nigeria having the potential to serve as the economic center of Africa are at fine locations to benefit from the large markets geographically close to them.  Among the four MINT countries, only Nigeria is not a member of G20 group, however it has great natural resource riches especially oil and gas (Asongu et al., 2018: 27). Furthermore when  BRICS and MINT countries combined, according to the World Bank 2018 data, they form 50.4% of the world population and 27.5% of the total GPD (World Bank, 2018). These data shows the contributions of the BRICS and MINT countries to the development of the world. On the other hand, the question of whether the macroeconomic development trend in BRICS and MINT countries brings the citizens of these countries any human development or not comes into minds.

Human development is to focus on improving the lives the humans lead instead of assuming that the economic growth rate shall automatically create wellness for everyone. Income increase is seen as a tool for development rather than a goal within itself. The three determinants of the human development are having a long, healthy and creative life; being knowledgeable, and accessing the resources required for a good standard of life (UNDP, 2020). Therefore, the Human Development Index (HDI) published regularly by the United Nations Development Program each year, includes indicators related to the life expectancy and educational opportunities besides the income. For example, despite the GNI (Gross National Income) of Greece is same as Turkey’s, in HDI Turkey is at the 59^th position whereas Greece is at 32^nd. This supports that individual income or economic development is not solely sufficient for human development. However, the income of most of the countries at the higher ranks of HDI being high proves that one of the most fundamental indicators of human development is income. In addition, the income rests inadequate to solely reflect the macroeconomic performance of the countries. Financially, besides the income as an indicator of the economic stability, unemployment, inflation rate, external trade debt, growth rate, budget balance, debt load, real interest rate in the market and many similar indicators can also be used. As the result of analyses conducted with all or part of these indicators, the macroeconomic performance of the countries can be monitored. The assessment of the macroeconomic performance of the countries is hard in several aspects. First of all, assessment of all these indicators together is not possible technically as they are not represented in a common unit and the importance of the change in each of them shall not be same. Secondly, some of these indicators are associated with each other economically. They move sometimes in the same and sometimes in opposite directions. In other words, there can be controversy between macroeconomic goals (Güran & Tosun, 2005: 90). In order to overcome especially the conflict in the selection of the indicators, some calculations are used in the measurement of the Macroeconomic performance consisting of different indicators. Misery Index (sum of the inflation rate and the unemployment rate), Calmfors Index (difference between the unemployment rate and the normalized trade balance), and Magic Diamond (GDP growth rate, inflation rate, unemployment rate and normalized trade balance) can be given as examples of these calculations. Each of these calculations, despite being informative, has some deficiencies. Misery Index and Calmfors Index have just two indicators and both indicators are equally weighted. The Magic Diamond has four indicators however this method also operates under equal weight principles. The main deficiency of these calculations is the arbitrary weighting chart they share (Lovell et al., 1995: 508). In this study, in order to overcome these disadvantages, the indicators to optimally represent the macroeconomic performance of the countries are selected taking the literature and specialist views into account.

The fundamental goal of the study conducted via using macroeconomic performance indicators selected via taking the expert views and literature into account and the HDI indicators is determined as to assess the BRICS and MINT countries with COPRAS and CRITIC methods. In this way it is examined whether the economically developed countries provide human development to their citizens. In addition, instead of giving equal weights both to the macroeconomic indicators and to the human development indicators, the objective weighting method CRITIC is used and the arbitrary weighting is avoided.

Literature Review

Since the years the BRICS and MINT countries are referenced together, they had been the subject of many academic studies. Yalçınkaya & Temelli (2014) in economic growth rate and current transactions balance relationship,  Başar et al. (2016) in the effect of terrorism on the touristic activities, Asongu et al. (2018) in direct foreign investors, Bozma et al. (2018) in economic growth rate and energy consumption relationship, Asongu & Odhiambo (2018) in determinants of economic growth, Gryczka (2018) in comparison of the economic indicators, Belke & Demir (2019) in the relation of public spending and economic growth, and Shao et al. (2019) in the relation of direct foreign investors and the environmental pollution, had combined or compared BRICS and MINT countries. However, in the examined literate, no study was found that examines the macroeconomic performance and the human development of the BRICS and MINT countries.

MCDM methods, as in many fields, have been used also in many studies where human development levels and macroeconomic performance are examined. In some of the studies related to the human development, only HDI indicators are used as criteria, whereas in other HDI indicators are used together with different criteria. In studies related to the macroeconomic performance the basic economic indicators are used according to the preferences of each author. In general, when selecting criteria, the data accessibility and expert views had played a role. The studies where MCDM methods are used and where the human development levels or macroeconomic performance of the countries/regions are assessed are given on the Table 1.

Table 1.MCDM Studies where human development and macroeconomic performance are Assessed

	Authors	Method	Country/Area
Human Development	Despotis (2005)	DEA	All Countries
	Despotis (2005a)	DEA	27 Asia-Pacific Countries
	Lozano & Gutiérrez (2008)	DEA	All Countries
	Hatefi & Torabi (2010)	MCDEA	Asia-Pacific Economic Cooperation (APEC) Countries
	Soltanpanah et al. (2010)	Entropy, AHP, TOPSIS, SAW	All Countries
	Vierstraete (2012)	DEA	146 Countries
	Blancard & Hoarau (2013)	DEA	122 Developing Countries
	Reig-Martinez (2013)	DEA	42 countries in the European Economic Area, North Africa and the Middle East
	Bilbao-Terol et al. (2014)	TOPSIS	105 Countries
	Paksoy (2015)	VIKOR	31 European Countries
	Sieng & Yussof (2015)	Fuzzy TOPSIS	6 Developed Countries, 6 Developing ASEAN Countries
	Balcerzak & Pietrzak (2017)	TOPSIS	29 European Countries
	Eren et al. (2017)	DEA	All Countries
	Eren & Kaynak (2017)	Grey Relation Analysis (GRA)	28 European Countries
	Orakçı & Özdemir (2017)	Entropy, CRITIC, GRA, MOORA	30 European Countries
	Şahin & Öztel (2017)	COPRAS	BRICS Countries and Turkey
	Erpolat Taşabat & Başer (2017)	TOPSIS, WSA, AHP	33 European Countries
	Başar & Turanlı (2018)	MOORA	49 Countries with High Human Development Level
	do Carvalhal Monteiro et al. (2018)	ELECTRE TRI	All Countries
Macroeconomic Performance	Fare et al. (1994)	DEA	17 OECD Countries
	Lovell et al.  (1995)	DEA	19 OECD Countries
	Lin et al.  (2011)	DEA, AHP	31 Provinces of China
	Mohamad & Said (2011)	DEA	57 Organization of the Islamic Conference (OIC) Member Countries
	Poveda (2011)	DEA	23 Cities of Colombia
	Podvezko (2011)	SAW, COPRAS	4 European Countries
	Chattopadhyay & Bose (2015)	TOPSIS, Entropy	48 Countries from All Regions
	Eyüboğlu (2016)	AHP, TOPSIS	10 Developing Countries
	Önder & Boz (2017)	GRA	36 Mediterranean Countries
	Skare & Rabar (2017)	DEA	30 OECD Countries
	Ela & Kurt (2019)	TOPSIS	8 Sub-Saharan African countries
	Özbek & Demirkol (2019)	AHP, ARAS, COPRAS, GRA	29 European Countries

As shown on Table 1, MCDM methods have been used in many studies where human development or macroeconomic performance are handled. However in general, in order to weight the criteria Entropy, AHP; and to assess the alternatives TOPIS, DEA methods are frequently preferred. Whereas in this study, in order to weight the criteria, CRITIC method is used which is stipulated by Diakoulaki et al. (1995) and used in many decision problems like climate selection (Vujicic et al., 2017), risk assessment (Ayrım & Can, 2017), third party logistics service provider selection (Keshavarz Ghorabaee et al., 2017), assessment of construction equipment (Keshavarz Ghorabaee et al., 2018), assessment of financial performance (Şenol & Ulutaş, 2018), bank performance assessment (Akbulut, 2019), cargo firm assessment (Ulutaş & Karaköy, 2019), corporate sustainability performance analysis (Yalçın & Karakaş, 2019), assessment of venture capital investment partnerships (Apan & Öztel, 2020), personnel selection process (Ayçin, 2020), assessment of R&D performance of the countries (Orhan & Aytekin, 2020), and 5G industry assessment (Peng et al., 2020). For the evaluation of the alternatives, COPRAS method is preferred which is developed in 1996 by Zavadskas and Kaklauskas, and used solely or together with different MCDM methods like equipment selection for structure consolidation (Kaklauskas et al., 2006), risk analysis (Yazdani et al., 2011), technical institutions performance assessment (Das et al., 2012), maintenance strategy selection (Fouladgar et al., 2012), market segment assessment (Aghdaie et al., 2013), social media platform selection (Tavana et al., 2013), supplier selection (Keshavarz Ghorabaee et al., 2014), green supplier selection (Liou et al., 2016), storage location selection (Özbek & Erol, 2016), material selection (Mousavi-Nasab & Sotoudeh-Anvari, 2017), cloud storeage service provider assessment (Çakır & Karabıyık, 2017), database management system selection (Ayçin, 2018), and financial performance assessment (Ayçin & Çakın, 2019).

In some of the studies where human development levels are examined with MCDM methods, economic data are used however no study is found in the literature where macroeconomic performance and human development levels are examined together. In addition, other than the study where Deepa et al. (2019) recommended a new criteria weighting method based on CRITIC and assessed agricultural lands with COPRAS, no studies are seen that use CRITIC and COPRAS together. In this regard, this study have provided authentic results regarding both the used data set and the methods preferred, and had filled a gap in the literature.

Method

Purpose of the Research, Sample, Data Collection Method and Data

The fundamental goal of the research is determined as to assess the BRICS and MINT countries according to the human development and macroeconomic performance. BRICS is composed of Brazil, Russian Federation, India, China and South Africa, and MINT is composed of Mexico, Indonesia, Nigeria and Turkey. The common characteristics of these countries are that they are developing countries and have the potential to have a say in world economics. In accordance with the goals of the research, in order to assess whether the countries with economic consolidation potential like BRICS and MINT has the same potential in human development, these countries are taken as sample.  In order to assess the BRICS and MINT countries according to the human development and macroeconomic performance, the criteria given on the Table 2 are determined considering the expert views and the literature. While assessing the countries according to human development levels, the HDI indicators, selected as criteria in this study, are used frequently. However, as there are no generally acknowledges indicator groups for the assessment of macroeconomic performance, the most frequently used indicators are selected as criteria for this study.

Table 2. The Criteria Used In the Study

HDI	x1- Life Expectancy at Birth (Year)	Soltanpanah et al. (2010); Bilbao-Terol vd. (2014); Safari & Ebrahimi, (2014); Orakçı & Özdemir (2017); Krylovas vd. (2019); Omrani vd. (2020)
	x2- Mean Years of Schooling (Year)
	x3- Expected Years of Schooling (Year)
	x4- GNI Per Capita ($)
Macroeconomic Performance	x5- Economic Growth Rate (%)	Eyüboğlu (2016); Karakış & Göktolga (2016); Özbek & Demirkol (2019);
	x6- Unemployment Rate (%)	Podvezko (2011); Fare et al. (2014); Eyüboğlu (2016); Karakış & Göktolga (2016); Özbek & Demirkol (2019); Skare & Rabar (2017)
	x7- Inflation Rate (%)	Lovell et al. (1995); Eyüboğlu (2016); Karakış & Göktolga (2016); Skare & Rabar (2017); Özbek & Demirkol (2019);
	x8- Foreign Trade Balance (Export/Import)	Lovell et al. (1995); Özbek & Demirkol (2019);  Podvezko (2011); Skare & Rabar (2017)
	x9- GDP Per Capita ($)	Lovell et al. (1995); Lin et al. (2009); Podvezko (2011); Fare et al. (2014); Karakış & Göktolga (2016); Skare & Rabar (2017)

The data of the criteria selected for the assessment of the human development and macroeconomic performance of the BRICS and MINT countries are acquired from the World Bank and United Nations Human Development Reports (HDR) (The World Bank, 2020; HDR, 2019). The definitive statistical information on the selected criteria are given on the Table 3.

Table 3. Definitive Statistics of the Criteria

Criteria	Min	Max	Mean	Std. Dev.
x1- Life Expectancy at Birth	54,3	77,4	70,7	7,45
x2- Mean Years of Schooling	6,5	12,0	8,3	1,75
x3- Expected Years of Schooling	9,7	16,4	13,7	2,01
x4- GNI Per Capita	5086	25036	14743	7043
x5- Economic Growth Rate	0,8	6,8	3,3	2,28
x6- Unemployment Rate	3,3	26,9	8,94	7,43
x7- Inflation Rate	2,1	16,3	6,06	4,82
x8- Foreign Trade Balance (Export/Import)	0,83	1,47	1,02	0,18
x9- GDP Per Capita	2010	11288	7045	3579

Despite the BRICS countries are referenced together with the MINT countries, both their human development data and macroeconomic performance data show great differences. The countries are assessed both with a criteria group where HDI indicators and macroeconomic performance indicators are present together (scenario 1) and with criteria groups where HDI indicators and macroeconomic performance indicators are taken separately (scenario 2 and scenario 3).  In the analysis conducted by using the criteria selected for the assessment of the BRICS and MINT countries, the CRITIC and COPRAS methods are used. Firstly using CRITIC method the criteria weights are determined and then these weight are used in the COPRAS method and the countries are sorted according to three different scenarios.

CRITIC

In decision making problems the criteria are seen as information sources. The weight of importance of the criteria reflects the amount of information on each of them on the decision matrix. In order to gather information from the criterion some methods (MACBETH, SWARA etc.) utilize expert opinions. In CRITIC method, the achieved weights do not contain expert views so they can be qualified as “objective weights”. The weights acquired by this method contain the contrast intensity of each criterion and the conflicts between the criteria. The contrast intensity of the scales are evaluated with the standard deviation and the conflict among them is measured with the correlation coefficient (Ghorabaee et al., 2018).

CRITIC method consists of different number of steps at different resources however it is in general defined in five steps (Diakoulaki et al., 1995: 764-765; Akbulut, 2019: 254-255; Ayçin, 2020: 4-5; Ghorabaee et al., 2017: 69; Xu et al., 2020: 69).

Step 1: Establishment of the Decision Matrix

As in all MCDM methods, CRITIC method starts with formation of the decision matrix. As shown in the Equation (1), the decision matrix named as X is created as formed from n criteria and m alternatives.

(1)

In the Equation (1) the  values, represent the values taken by the i^th alternative according to the j^th assessment criteria. (j is the assessment criterion number  and i is the decision alternative number ;).

Step 2: Establishment of the Normalized Decision Matrix

The values of the criteria may not be represented by the same unit so that the criteria values should be reduced to the range of [0, 1]. Therefore, in the second step of the CRITIC method, the benefit direction (maximization) and cost direction (minimization) criteria are normalized via using Equation (2) and Equation (3).

	(2)
	(3)

While calculating the Equation (2) and Equation (3),  represents the minimum value at the related criteria and  represents the maximum value.

Step 3: Establishment of the Correlation Coefficient Matrix

CRITIC method is dependent on relations between the criteria couples so that in the third step the Equation (4) is used to calculate the linear correlation coefficients ( ) between criteria couples. Then the linear correlation coefficients calculated are used to create a correlation coefficient matrix.

(4)

Step 4: Calculation of the Total Information Measurement ( )

CRITIC method aims to acquire information from contrast intensity and conflicts. Higher the H_j value called the total information that is acquired from both sources, larger the information amount transmitted by the corresponding criterion shall become. In order to calculate the said total information the Equation (5) is used.

(5)

To calculate the H_j value indicating the total information standard deviation information are used. In order to calculate the standard deviation values the Equation (6) is used.

(6)

Step 5: Determination of the Criteria Weights ( )

In order to calculate the final criteria weights ( ) the H_j values of the criteria are used. Increase in the H_j value increases the relative importance for the decision making process. In order to calculate w_j value the Equation (7) is used.

(7)

 

 

COPRAS

The decision analysis is related to the situations where the decision maker should make choice among the various alternatives generally considering the conflicting criteria. Therefore, Zavadskas and Kaklauskas had developed an alternative assessment method called COPRAS (Complex Proportional Assessment) in 1996 (Aghdaie et al., 2013: 222). COPRAS method focuses on the importance and benefit degrees of the current alternatives under the presence of conflicting criteria. It takes in to account the performance of the alternatives according to the different criteria and related criteria weights. This method considers both the good and bad solutions and choses the best alternative (Chatterjee et al., 2011: 852).

COPRAS method, different from the other MCDM methods, stipulates how much each alternative is better or worse than the others as percentage while comparing alternatives. Among the advantages of the COPRAS method, solution steps being short and easy, not requiring a special computer software can be counted (Ayçin, 2019: 64). While COPRAS method consist of different number of steps considering different solution paths, it is generally calculated in six steps (Kaklauskas et al., 2005: 363-364; Das et al., 2012: 237; Özdağoğlu, 2013: 235-237):

Step 1: Establishment of the Decision Matrix

The decision matrixes are formed similarly in most of the MCDM methods. The decision matrix formed of n criteria and m alternatives and called as D in CRITIC method is formed in the same way for COPRAS as shown in the Equation (1).

Step 2: Normalization of the Decision Matrix

In this step, in order to render non-dimensional the different units of the criteria, the criteria values on the X matrix are subjected to normalization transaction. To perform the normalization calculation the Equation (8) is used.

(8)

Step 3: Establishment of the Weighted Normalized Decision Matrix

In order to weight the normalized decision matrix, the Equation (9) is used. The weights can be used as equal and also can be calculated by MACBETH, SWARA methods that are based on expert opinion or by Entropi, CRITIC methods that are not based on expert opinion.

(9)

Later on the acquired values are used to form the weighted normalized decision matrix called D’ as shown on the Equation (10).

(10)

Step 4: Sum of the Weighted Normalized Indexes

The sum of the weighted normalized values are calculated for both benefit direction (maximization) and cost direction (minimization) criteria in order. The sum of the benefit direction criteria weighted normalized values is called , and the sum of the cost direction criteria weighted normalized values is called . While calculating  Equation (11), and while calculating  Equation (12) are utilized.

	(11)
	(12)

For the benefit direction criteria the higher values indicate as better situation while for cost direction criteria a lower value indicates a better situation.

Step 5: Calculation of Relative Importance Levels of the Alternatives

In order to calculate the relative importance level of the alternatives represented by  the Equation (13) is utilized.

(13)

Q_i reflecting the relative importance of an alternative shows the satisfaction level the said alternative reaches. The alternative with the highest relative importance level (Q_max) among all alternatives is qualified as the best alternative.

Step 6: Calculation of the Performance Index ( ) of the Alternatives

, reflects the maximum relative importance value. The (P_i), the performance index calculated for each alternative via using Q_max is used to achieve the complete ranking of the candidate alternatives.

(14)

Alternative with the P_i named as the performance index equal to 100 is assessed as the best alternative and the rest of the alternatives are sorted from bigger to smaller values and the order of the alternatives is achieved.

Findings

In order to determine the weights of nine criteria determined considering the purpose of the research, the CRITIC method is used. Later on these weights are used in the COPRAS method and the countries are sorted. The solution is conducted according to the solution steps of the CRITIC and COPRAS methods. First of all, the decision matrix is formed by using the Equation (1) which is the most fundamental stage of all MCDM methods. The decision matrix used for both the CRITIC and COPRAS methods is shown on the Table 4.

 

Table 4. Decision Matrix

Criteria Aspect	Max	Max	Max	Max	Max	Min	Min	Max	Max
Criteria/Countries	x1	x2	x3	x4	x5	x6	x7	x8	x9
Brazil	75,7	15,4	7,8	14068	1,3	12,3	3,7	1,0671	9001,2
Russia	72,4	15,5	12	25036	2,3	4,8	2,9	1,4791	11288
India	69,4	12,3	6,5	6829	6,8	5,3	4,9	0,8352	2010
China	76,7	13,9	7,9	16127	6,6	4,3	2,1	1,0403	9770,8
South Africa	63,9	13,7	10,2	11756	0,8	26,9	4,5	1,0106	6374
Mexico	75	14,3	8,6	17628	2,1	3,3	4,9	0,9547	9673,4
Indonesia	71,5	12,9	8	11256	5,2	4,5	3,2	0,9692	3893,6
Nigeria	54,3	9,7	6,5	5086	1,9	8,2	12,1	0,9479	2028,2
Turkey	77,4	16,4	7,7	24905	2,8	10,9	16,3	0,9609	9370,2

In order to reduce the criteria values conflicting to each other and mostly indicated in different units to the range of [0, 1], normalization transaction is applied. In order to realize the normalization of the benefit direction (maximization) criteria the Equation (2) and for the normalization of the cost direction (minimization) criteria the Equation (3) are utilized. The normalized decision matrix formed by the achieved normalized criteria is given on the Table 5.

Table 5. CRITIC Normalized Decision Matrix

Criteria Aspect	Max	Max	Max	Max	Max	Min	Min	Max	Max
Criteria/Countries	x1	x2	x3	x4	x5	x6	x7	x8	x9
Brazil	0,926	0,851	0,236	0,450	0,083	0,619	0,887	0,360	0,754
Russia	0,784	0,866	1,000	1,000	0,250	0,936	0,944	1,000	1,000
India	0,654	0,388	0,000	0,087	1,000	0,915	0,803	0,000	0,000
China	0,970	0,627	0,255	0,553	0,967	0,958	1,000	0,319	0,836
South Africa	0,416	0,597	0,673	0,334	0,000	0,000	0,831	0,272	0,470
Mexico	0,896	0,687	0,382	0,629	0,217	1,000	0,803	0,186	0,826
Indonesia	0,745	0,478	0,273	0,309	0,733	0,949	0,923	0,208	0,203
Nigeria	0,000	0,000	0,000	0,000	0,183	0,792	0,296	0,175	0,002
Turkey	1,000	1,000	0,218	0,993	0,333	0,678	0,000	0,195	0,793

As the CRITIC method is a method beads on binary correlations between the criteria, in the next step the correlation coefficients between the criteria are calculated by using the Equation (4) and shown on the Table 6.

Table 6. Matrix for Correlation Coefficient between Criteria

Criteria Aspect	Max	Max	Max	Max	Max	Min	Min	Max	Max
Criteria/Countries	x1	x2	x3	x4	x5	x6	x7	x8	x9
x1	1	0,835	0,166	0,688	0,268	0,311	0,231	0,177	0,708
x2	0,835	1	0,503	0,877	-0,203	-0,091	0,049	0,443	0,851
x3	0,166	0,503	1	0,602	-0,411	-0,242	0,388	0,840	0,612
x4	0,688	0,877	0,602	1	-0,204	0,119	-0,110	0,617	0,886
x5	0,268	-0,203	-0,411	-0,204	1	0,569	0,261	-0,302	-0,281
x6	0,311	-0,091	-0,242	0,119	0,569	1	0,121	0,075	0,047
x7	0,231	0,049	0,388	-0,110	0,261	0,121	1	0,320	0,163
x8	0,177	0,443	0,840	0,617	-0,302	0,075	0,320	1	0,626
x9	0,708	0,851	0,612	0,886	-0,281	0,047	0,163	0,626	1

In the next stage, while calculating the H_j indicating the total information measurement, the correlation values between the criteria given on the Table 5 and the standard deviation values of the criteria are used. The standard deviations of the criteria are calculated by the Equation (6), and the Equation (5) is used for the calculation of the total information measurement H_j.  Then H_j value is used to achieve final criteria weights by utilizing the Equation (7). The total information measurement H_j and the criteria weights are shown on the Table 7.

Table 7. Total Information Measurements ( ) and Criteria Weights (

	x1	x2	x3	x4	x5	x6	x7	x8	x9
	1,4899	1,4212	1,7710	1,5976	3,1575	2,2329	2,2325	1,4639	1,6934
	0,0873	0,0833	0,1038	0,0936	0,1851	0,1309	0,1309	0,0858	0,0993

According to the criteria weights achieved in Table 6, the most important criterion is determined as the “x₅- Economic Growth Rate (0.1851)”. This criterion is followed, in order, by the “x₇-Unemployment Ratio (0.1309)”, “x₆-Inflation Rate (0.1309)” and “x₂- Mean Years of Schooling (0.1038)” criteria. Thus, when the weights reached as the result of the CRITIC method, the criteria regarding the macroeconomic performance had shown more importance.

At the subsequent stage, the criteria weights achieved by the CRITIC method are used in the solution steps of the COPRAS method. COPRAS method also, like the CRITIC method, starts with the formation of the decision matrix. As the same decision matrix is used for both COPRAS and CRITIC methods, the decision matrix is as shown on Table 3. However as the normalization steps of COPRAS method and CRITIC method are different, in this stage the normalization transaction of the COPRAS method is conducted. In order to render dimensionless the criteria in different units, the Equation (8) is used that is given in the second step of the COPRAS method. The achieved normalized values are given on the Normalized decision matrix given on the Table 8.

Table 8. COPRAS Normalized Decision Matrix

Criteria Aspect	Max	Max	Max	Max	Max	Min	Min	Max	Max
Criteria/Countries	x1	x2	x3	x4	x5	x6	x7	x8	x9
Brazil	0,119	0,124	0,104	0,106	0,044	0,153	0,068	0,115	0,142
Russia	0,114	0,125	0,160	0,189	0,077	0,060	0,053	0,160	0,178
India	0,109	0,099	0,086	0,051	0,228	0,066	0,090	0,090	0,032
China	0,121	0,112	0,105	0,122	0,221	0,053	0,038	0,112	0,154
South Africa	0,100	0,110	0,136	0,089	0,027	0,334	0,082	0,109	0,101
Mexico	0,118	0,115	0,114	0,133	0,070	0,041	0,090	0,103	0,153
Indonesia	0,112	0,104	0,106	0,085	0,174	0,056	0,059	0,105	0,061
Nigeria	0,085	0,078	0,086	0,038	0,064	0,102	0,222	0,102	0,032
Turkey	0,122	0,132	0,102	0,188	0,094	0,135	0,299	0,104	0,148

In the third step of the COPRAS method, the normalized criteria are weighted using the weight values achieved the CRITIC method. Primarily, the Equation (9) is used to weight each criterion value, and then the weighted normalized decision matrix given in the Equation (10) is formed. The weighted normalized decision matrix is shown on the Table 9.

Table 9. Weighted Normalized Decision Matrix

Criteria Aspect	Max	Max	Max	Max	Max	Min	Min	Max	Max
Criteria/Countries	x1	x2	x3	x4	x5	x6	x7	x8	x9
Brazil	0,010	0,010	0,011	0,010	0,008	0,020	0,009	0,010	0,014
Russia	0,010	0,010	0,017	0,018	0,014	0,008	0,007	0,014	0,018
India	0,010	0,008	0,009	0,005	0,042	0,009	0,012	0,008	0,003
China	0,011	0,009	0,011	0,011	0,041	0,007	0,005	0,010	0,015
South Africa	0,009	0,009	0,014	0,008	0,005	0,044	0,011	0,009	0,010
Mexico	0,010	0,010	0,012	0,012	0,013	0,005	0,012	0,009	0,015
Indonesia	0,010	0,009	0,011	0,008	0,032	0,007	0,008	0,009	0,006
Nigeria	0,007	0,007	0,009	0,004	0,012	0,013	0,029	0,009	0,003
Turkey	0,011	0,011	0,011	0,018	0,017	0,018	0,039	0,009	0,015

On the next step, the sums of the benefit direction and cost direction criteria’s weighted normalized values are calculated. In order to reach the sum of the weighted normalized values of seven benefit direction criteria the Equation (11) and to reach the sum of the weighted normalized values of two cost direction criteria the Equation (12) are utilized. The sums of both the benefit direction and the cost direction weighted normalized values are shown on the Table 10.

Table 10. Sum of the Weighted Normalized Indexes

Countries
Brazil	0,07347	0,02887
Russia	0,10023	0,01475
India	0,08469	0,02036
China	0,10807	0,01202
South Africa	0,06465	0,05452
Mexico	0,08123	0,01711
Indonesia	0,08483	0,01499
Nigeria	0,05028	0,04233
Turkey	0,09080	0,05679

The last stage of the COPRAS method is the calculation of the relative importance levels and the performance indexes. Primarily, using the sum of the weighted normalized indexes, by the help of the Equation (13) the relative importance levels are calculated. Later on, the relative importance levels are used and by the help of the Equation (14) the performance indexes are calculated. Finally, the countries are sorted in order according to the performance indexes achieved. The achieved relative importance levels, performance indexes and the order of the countries are given on the Table 11.

Table 11. Relative Importance Levels ( ), Performance Indexes ( ) and Order of the Alternatives

Countries			Rank
Brazil	0,09509	59,44414	7
Russia	0,14252	89,09728	2
India	0,11534	72,10275	5
China	0,15997	100	1
South Africa	0,07610	47,57046	8
Mexico	0,11771	73,58328	4
Indonesia	0,12647	79,06127	3
Nigeria	0,06502	40,64791	9
Turkey	0,10179	63,62961	6

As the result of the assessment conducted by using CRITIC weighted COPRAS method, it is determined that the China is the best country according to the human development and macroeconomic performance. China is followed in order by Russia, Indonesia and Mexico. The countries with the worst performance are found out to be Brazil, South Africa and Nigeria.

In the study the ranking is found according to the model where all the criteria (four HDI indicators and five macroeconomic performance indicators) are used. These rankings are named as the “Scenario 1”. In addition, the CRITIC based COPRAS solutions are performed according only to HDI indicators or only to macroeconomic performance indicators. The rankings achieved according to the HDI indicators are named as “Scenario 2” and the rankings achieved according to the macroeconomic performance indicators are named as “Scenario 3”. The rankings achieved for all three scenarios are given on Table 12.

Table 12. Rankings According to the Scenarios

Countries	Scenario 1	Scenario 2	Scenario 3
Brazil	7	6	6
Russia	2	1	2
India	5	8	4
China	1	4	1
South Africa	8	5	8
Mexico	4	3	5
Indonesia	3	7	3
Nigeria	9	9	9
Turkey	6	2	7

In the “World Bank” and “United Nations Human Development Report” from which the research data are gathered, there are rankings of all countries according to different indicators. These lists are composed for about 200 countries (some countries have 2018 data). In order to compare the rankings achieved within the three scenarios as the result of the study with the real lists, the rankings of the countries according to the HDI and macroeconomic indicators are given on the Table 13. As the HDI list is calculated with four indicators, the separate rankings for each indicator are not shown. As there is no index using the indicators of the macroeconomic performance, the rankings of these indicators are given separately. Also instead of the places of these countries among the 200 countries, the nine countries are sorted among themselves via considering their places at that list. For example, Russia being 49^th according to HDI is at the first place and Turkey being 59^th is at the second place.

Table 13. Lists According to the Real Data

Countries	HDI Rank	Growth Rate Rank	Unemployment Rate Rank	Inflation Rate Rank	Foreign Trade Balance Rank	GDP Per Capita Rate
Brazil	4	8	8	4	2	5
Russia	1	5	4	2	1	1
India	8	1	5	6	9	9
China	5	2	2	1	3	2
South Africa	7	9	9	5	4	6
Mexico	3	6	1	6	7	3
Indonesia	6	3	3	3	5	7
Nigeria	9	7	6	8	8	8
Turkey	2	4	7	9	6	4

The relations between the rankings achieved from the three scenarios in this study and the real lists of the indicators are examined with the spearman rank correlation. Spearman rank correlation is used frequently in the literature to examine the relations between the different rankings achieved with the MCDM methods (Peng, 2015; Kou et al., 2012). On Table 14, the spearman rank correlation values between the rankings achieved for the three scenarios by using CRITIC and COPRAS method and the real rankings of the used indicators are given.

Table 14: Spearman’s Rank Correlation Values between Rankings

Scenarios	HDI Rank	Growth Rate Rank	Unemployment Rate Rank	Inflation Rate Rank	Foreign Trade Balance Rate	GDP Per Capita Rate
Scenario 1	0,483	0,667*	0,783*	0,695*	0,367	0,600
Scenario 2	0,917*	-0,017	-0,200	-0,209	-0,533	0,900*
Scenario 3	0,367	0,683*	0,700*	0,778*	0,400	0,483

The rankings of the Scenario 1 where all the criteria resides and the Scenario 3 where the macroeconomic performance indicators reside have significant relation with the rankings of growth rate, unemployment rate and inflation rate. Also, there is a significant relation between the ranking of the Scenario 2 where the HDI indicators reside and the real HDI ranking and GDP per capita ranking.

Conclusion

For long years, the countries are assessed in many aspects in order to follow their own development levels and also to indicate the differences between them and other countries. In order to conduct these assessments, the indexes used frequently and containing different indicators compared to each other are started to be used by many economic and social institutions (World Bank, OECD, United Nations etc.) in the recent years. Of course, the indicators within these indexes vary according to the goals of the related institution. Besides, in many academic studies, different indicators are used to assess the countries. Many of the assessments used similar indicator combinations. The difference of this study from the other studies is to assess the human development and macroeconomic performance indicators jointly and severally.  While conducting this assessment, the CRITIC method that is an objective weighting method is used. In this way, the problems that may arise due to subjective judgments (human factor) are tried to be avoided by using an objective weighting method.

In the first stage of the study, the criteria weights are determined by the CRITIC method. According to the criteria weights of the scenario 1 where all indicators are included, the most important criterion is found out to be the “economic growth”. This criterion is followed by the inflation and unemployment rate criteria. Thus, in the assessment where all the criteria are included, the most important criteria consist of macroeconomic performance indicators. According to the assessment made only with the macroeconomic performance indicators (scenario 3), the criteria weight follow the same ranking. In the assessment made by only the HDI indictors, the most important criterion is found out to be the “Mean Years of Schooling” and this criterion is followed by the “life expectancy at birth” and “GNI per capita” criteria.

In the second stage of the study, the assessment of the countries is realized with the COPRAS method. The weights achieved in the CRITIC method are used and the country rankings for three scenarios are assessed. For the scenario 1 consisting of both HDI and macroeconomic performance indicators and the scenario 3 consisting of only the macroeconomic performance indicators, the most successful countries are, in order, China, Russia and Indonesia. According to the scenario 2 consisting of only HDI indicators, the most successful countries are, in order, Russia, Turkey and Mexico. When the results are examined, as can be understood from the weights achieved by the CRITIC method, macroeconomic performance indicators play an import role for the rankings of the countries. Therefore, in the assessments made according to the scenario 1 consisting of both HDI and macroeconomic performance indicators and the scenario 3 consisting of only the macroeconomic performance indicators, the countries had similar rankings. In addition, as can be understood according to the rankings of the countries according to the real data on Table 12, the ranking revealed according to the macroeconomic performance indicators and the real HDI rankings do not show similarities. For example, while India is at the first place according to the growth ratio, it is at 8^th rank according to the HDI.

The relation between the country rankings acquired as the result of the study and the country rankings revealed according to the real data is also important to provide an idea about the reliability of the study results. For this purpose, in order to test the relation between rankings achieved for each scenario in the result of the study and the real rankings of the countries, the Spearman’s rank correlations are observed. The relations between the rankings achieved for the scenario 1 and scenario 3 and the real data rankings show similarities. The rankings achieved for both scenarios are in significant relations with the rankings of growth rate, unemployment ratio and the inflation rate. The rankings achieved for the scenario 2 where only HDI indicators are used are in significant relations with the real HDI ranking and the GDP per capita ranking. The real HDI ranking to be in a significant relation with the ranking of the scenario 2 is an expected condition, whereas the GDP per capita ranking being in a significant relationship with the scenario 2 is in quality to prove the effect of the economic well-being of the people on the human development.

When the study results are examined, the countries should try to improve their critical indicators to show a better performance and climb the rankings. Especially, the countries can take measures to improve the macroeconomic performance indicators (economic growth, unemployment ratio, inflation rate) that come forth as the most important criteria as the result of the CRITIC method. In fact, providing the economic growth rate that is the most important one of the criteria, shall contribute directly to the improvement of other indicators in the study. In order to provide economic growth, especially the production sector can be supported and additional employment can be created, thus unemployment can be reduced and also as the domestic production shall decrease the general level of the prices, the related country can be relived off the inflation pressure. Furthermore, as the growth of the economy shall direct the countries to spend more on health and education, the most fundamental indicators of the humandevelopment level, the life expectancy at birth and the education opportunities can be improved.

In the conducted study, a model based on CRITIC and COPRAS that may assess the countries regarding the macroeconomic performance and human development is established. In future studies, this model can be utilized in assessment of different countries. The methods used in this study are objective methods that do not require expert opinions. In the future studies, the MCDM methods where expert opinions are used can be utilized and the countries can be assessed according to the same criteria. Also again in the future studies, different criteria combinations from the criteria used in this study can be adopted and different results can be achieved.

References

Aghdaie, M. H., Zolfani, S. H., & Zavadskas, E. K. (2013). Market segment evaluation and selection based on application of fuzzy AHP and COPRAS-G methods. Journal of Business Economics and Management, 14(1), 213-233. https://doi.org/10.3846/16111699.2012.721392

Ağır, H., & Yıldırım, S. (2015). Türkiye ile BRICS ekonomilerinin makroekonomik performans karşılaştırması: Betimsel bir analiz. KSÜ Sosyal Bilimler Dergisi, 12(2), 39-66.

Akbulut, O. Y. (2019). CRITIC ve EDAS yöntemleri ile İş Bankası'nın 2009-2018 yılları arasındaki performansının analizi. Ekonomi Politika ve Finans Araştırmaları Dergisi, 4(2), 249-263. https://doi.org/10.30784/epfad.594762

Akgemci, E. (2011). BRICS ülkeleri neyi temsil ediyor? Ankara Üniversitesi Avrupa Toplulukları Araştırma ve Uygulama Merkezi E-Bülten, 3(26), 3.

Apan, M., & Öztel, A. (2020). Girişim sermayesi yatırım ortaklıklarının CRITIC-PROMETHEE bütünleşik karar verme yöntemi ile finansal performans değerlendirmesi: Borsa İstanbul’da bir uygulama. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi, 63, 54-73

Asongu, S., & Odhiambo, N. M. (2018). Drivers of growth in fast emerging economies: a dynamic instrumental quantile approach to real output and its rates of growth in Brics and Mint countries, 2001-2011. Applied Econometrics and International Development, 18(1), 5-22. http://dx.doi.org/10.2139/ssrn.3166310

Asongu, S., Akpan, U. S., & Isihak, S. R. (2018). Determinants of foreign direct investment in fast-growing economies: evidence from the BRICS and MINT countries. Financial Innovation, 4(1), 26. https://doi.org/10.1186/s40854-018-0114-0

Ayçin, E. (2018). Using The SWARA and COPRAS Methods in The Selection of Database Management System. Dijital Çağda İşletmecilik Dergisi, 1(2), 51-58.

Ayçin, E. (2019). Çok Kriterli Karar Verme: Bilgisayar Uygulamalı Çözümler. Nobel Yayınevi: Ankara.

Ayçin, E., & Çakın, E. (2019). KOBİ'lerin Finansal Performansının MACBETH-COPRAS Bütünleşik Yaklaşımıyla Değerlendirilmesi. Journal of Yasar University, 14(55), 251-265.

Ayçin, E., (2020). Personel Seçim Sürecinde CRITIC ve MAIRCA Yöntemlerinin Kullanılması, İşletme, 1(1), 1-12.

Ayrım, Y., & Can, G. F. (2017). Risk Değerlendirmesinde CRITIC Metodu İle Sektörlerin Karşılaştırması. Journal of Turkish Operations Management, 1(1), 67-78.

Balcerzak, A. P., & Pietrzak, M. B. (2017). Human development and quality of institutions in highly developed countries. In Financial Environment and Business Development Bilgin, M.H., Danis, H., Demir, E., Can, U. (Eds.) (pp. 231-241). Springer, Cham.

Başar, Ö. D. & Turanlı, M., (2018). Ranking of high human development countries with respect to variables relating with female living standards by using MOORA method. International Journal of Gender and Women’s Studies, 6(1), 111-120. https://doi.org/10.15640/ijgws.v6n1p10

Başar, S., Künü, S., & Hopoğlu, S. (2016). Terörün BRICS ve MINT Ülkelerindeki Turizm Faaliyetleri Üzerine Etkisi. Finans Politik & Ekonomik Yorumlar, 53(618), 43-50.

Belke, M., & Demir, H. İ. (2019). BRICS–MINT Ülkelerinde Kamu Harcamaları ve Ekonomik Büyüme İlişkisi: 1990–2017 Dönemi. ICONSR 2019, 20.

Bilbao-Terol, A., Arenas-Parra, M., Cañal-Fernández, V., & Antomil-Ibias, J. (2014). Using TOPSIS for assessing the sustainability of government bond funds. Omega, 49, 1-17. https://doi.org/10.1016/j.omega.2014.04.005

Blancard, S., & Hoarau, J. F. (2013). A new sustainable human development indicator for small island developing states: A reappraisal from data envelopment analysis. Economic Modelling, 30, 623-635. http://dx.doi.org/10.1016/j.econmod.2012.10.016

Bornmann, L., Wagner, C., & Leydesdorff, L. (2015). BRICS countries and scientific excellence: A bibliometric analysis of most frequently cited papers. Journal of the Association for Information Science and Technology, 66(7), 1507-1513. https://doi.org/10.1002/asi.23333

Bozma, G., Aydın, R., & Kolçak, M. (2018). BRICS ve MINT Ülkelerinde Ekonomik Büyüme ve Enerji Tüketimi İlişkisi. Igdir University Journal of Social Sciences, (15), 323-338.

Chattopadhyay, S., & Bose, S. (2015). Global macroeconomic performance: a comparative study based on composite scores. Journal of Reviews on Global Economics, 4, 51-68.

Çakır, E., & Karabıyık, B. K. (2017). Bütünleşik SWARA-COPRAS yöntemi kullanarak bulut depolama hizmet sağlayıcılarının değerlendirilmesi. Bilişim Teknolojileri Dergisi, 10(4), 417-434. https://doi.org/10.17671/gazibtd.296094

Das, M. C., Sarkar, B., & Ray, S. (2012). A framework to measure relative performance of Indian technical institutions using integrated fuzzy AHP and COPRAS methodology. Socio-Economic Planning Sciences, 46(3), 230-241. https://doi.org/10.1016/j.seps.2011.12.001

Deepa, N., Ganesan, K., Srinivasan, K., & Chang, C. Y. (2019). Realizing sustainable development via modified integrated weighting MCDM model for ranking agrarian dataset. Sustainability, 11(21), 6060. https://doi.org/10.3390/su11216060

Despotis, D. K. (2005). Measuring human development via data envelopment analysis: the case of Asia and the Pacific. Omega, 33(5), 385-390. https://doi.org/10.1016/j.omega.2004.07.002

Despotis, D. K. (2005a). A reassessment of the human development index via data envelopment analysis. Journal of the operational research society, 56(8), 969-980. https://doi.org/10.1057/palgrave.jors.2601927

Diakoulaki, D., Mavrotas, G., & Papayannakis, L. (1995). Determining objective weights in multiple criteria problems: The critic method. Computers & Operations Research, 22(7), 763-770. https://doi.org/10.1016/0305-0548(94)00059-H

do Carvalhal Monteiro, R. L., Pereira, V., & Costa, H. G. (2018). A multicriteria approach to the human development index classification. Social Indicators Research, 136(2), 417-438. https://doi.org/10.1007/s11205-017-1556-x

Ela M. & Soysal Kurt H.. (2019). Comparison of macroeconomic performances of Sub-Saharan African countries with TOPSIS method. Gümüşhane Üniversitesi Sosyal Bilimler Enstitüsü Elektronik Dergisi, 10(3), 547-555.

Erbolat Taşabat, S. & Başer, B. (2017). The Analysis of Human Development Levels Via Multi-Criteria Decision Making Methods With Different Weighting Techniques: The Case of European Union Members, Candidates and Potential Candidates. International Journal of Recent Advances in Multidisciplinary Research, 4(11), 3022-3029.

Eren, M., & Kaynak, S. (2017). An evaluation of EU member states according to human development and global competitiveness dimensions using the multi-period grey relational analysis (MP-GRA) technique. Grey Systems: Theory and Application, 7(1), 60-70. https://doi.org/10.1108/GS-08-2016-0023

Eren, M., Eren, M., & Başar, S. (2017). Measuring of human development through the output-oriented super efficiency VRS DEA model without inputs. Serbian Journal of Management, 12(2), 255-270. https://doi.org/10.5937/sjm12-11184

Eyüboglu, K. (2016). Gelismekte Olan Ülkelerin Makro Performanslarinin AHP ve TOPSIS Yöntemleri ile Karsilastirilmasi. Cankiri Karatekin Universitesi Iktisadi ve Idari Bilimler Fakultesi Dergisi, 6(1), 131-146. https://doi.org/10.18074/cnuiibf.278

Färe, R., Grosskopf, S., Norris, M., & Zhang, Z. (1994). Productivity growth, technical progress, and efficiency change in industrialized countries. The American economic review, 84(1), 66-83.

Fouladgar, M. M., Yazdani-Chamzini, A., Lashgari, A., Zavadskas, E. K., & Turskis, Z. (2012). Maintenance strategy selection using AHP and COPRAS under fuzzy environment. International journal of strategic property management, 16(1), 85-104. https://doi.org/10.3846/1648715X.2012.666657

Gryczka, M. (2018). MINT countries as possible rising stars in the global economy–benchmarking with BRICS countries. Acta Scientiarum Polonorum. Oeconomia, 17(3), 23-31. https://doi.org/10.22630/ASPE.2018.17.3.34

Güran, M., & Tosun, M. (2005). Türkiye Ekonomisinin Makro Ekonomik Performansı: 1951-2003 Dönemi İçin Parametrik Olmayan Bir Ölçüm. Ankara Üniversitesi SBF Dergisi, 60(4), 89-115.

Hatefi, S. M., & Torabi, S. A. (2010). A common weight MCDA–DEA approach to construct composite indicators. Ecological Economics, 70(1), 114-120. https://doi.org/10.1016/j.ecolecon.2010.08.014

Kaklauskas, A., Zavadskas, E. K., Raslanas, S., Ginevicius, R., Komka, A., & Malinauskas, P. (2006). Selection of low-e windows in retrofit of public buildings by applying multiple criteria method COPRAS: A Lithuanian case. Energy and buildings, 38(5), 454-462. https://doi.org/10.1016/j.enbuild.2005.08.005

Karakış, E., & Göktolga, Z. G. (2016). Orta Asya Türk Cumhuriyetlerinin Ekonomik Performanslarının Analitik Hiyerarşi Süreci ve VIKOR Metodu ile Karşılaştırılması In International Conference On Eurasian Economies (pp. 786-793).

Keshavarz Ghorabaee, M. K., Amiri, M., Sadaghiani, J. S., & Goodarzi, G. H. (2014). Multiple criteria group decision-making for supplier selection based on COPRAS method with interval type-2 fuzzy sets. The International Journal of Advanced Manufacturing Technology, 75(5-8), 1115-1130. https://doi.org/10.1007/s00170-014-6142-7

Keshavarz Ghorabaee, M., Amiri, M., Kazimieras Zavadskas, E., & Antuchevičienė, J. (2017). Assessment of third-party logistics providers using a CRITIC–WASPAS approach with interval type-2 fuzzy sets. Transport, 32(1), 66-78. https://doi.org/10.3846/16484142.2017.1282381

Keshavarz Ghorabaee, M., Amiri, M., Zavadskas, E. K., & Antucheviciene, J. (2018). A new hybrid fuzzy MCDM approach for evaluation of construction equipment with sustainability considerations. Archives of Civil and Mechanical Engineering, 18, 32-49. https://doi.org/10.1016/j.acme.2017.04.011

Krylovas, A., Dadelienė, R., Kosareva, N., & Dadelo, S. (2019). Comparative evaluation and ranking of the European countries based on the interdependence between human development and internal security indicators. Mathematics, 7(3), 293. https://doi.org/10.3390/math7030293

Lin, M. I., Lee, Y. D., & Ho, T. N. (2011). Applying integrated DEA/AHP to evaluate the economic performance of local governments in China. European Journal of Operational Research, 209(2), 129-140. https://doi.org/10.1016/j.ejor.2010.08.006

Liou, J. J., Tamošaitienė, J., Zavadskas, E. K., & Tzeng, G. H. (2016). New hybrid COPRAS-G MADM Model for improving and selecting suppliers in green supply chain management. International Journal of Production Research, 54(1), 114-134. https://doi.org/10.1080/00207543.2015.1010747

Lovell, C. K., Pastor, J. T., & Turner, J. A. (1995). Measuring macroeconomic performance in the OECD: A comparison of European and non-European countries. European journal of operational research, 87(3), 507-518. https://doi.org/10.1016/0377-2217(95)00226-X

Lozano, S., & Gutiérrez, E. (2008). Data envelopment analysis of the human development index. International Journal of Society Systems Science, 1(2), 132-150. https://doi.org/10.1504/IJSSS.2008.021916

Mohamad, N. H., & Said, F. B. (2011). Assessing Macroeconomic Performance of OIC Member Countries Using Data Envelopment Analysis, DEA. Journal of Economic Cooperation & Development, 32(4), 21-50.

Mousavi-Nasab, S. H., & Sotoudeh-Anvari, A. (2017). A comprehensive MCDM-based approach using TOPSIS, COPRAS and DEA as an auxiliary tool for material selection problems. Materials & Design, 121, 237-253. https://doi.org/10.1016/j.matdes.2017.02.041

Omrani, H., Alizadeh, A., & Amini, M. (2020). A new approach based on BWM and MULTIMOORA methods for calculating semi-human development index: An application for provinces of Iran. Socio-Economic Planning Sciences, 70, 1-11. https://doi.org/10.1016/j.seps.2019.02.004

Orakçı, E., & Özdemir, A. (2017). Telafi Edici Çok Kriterli Karar Verme Yöntemleri ile Türkiye ve AB Ülkelerinin İnsani Gelişmişlik Düzeylerinin Belirlenmesi. Journal of Economics & Administrative Sciences/Afyon Kocatepe Üniversitesi Iktisadi ve Idari Bilimler Fakültesi Dergisi, 19(1), 61-74. https://doi.org/10.5578/jeas.49652

Orhan M. & Aytekin M. (2020), Türkiye ile AB’ye Son Katılan Ülkelerin Ar-Ge Performanslarının CRITIC Ağırlıklı MAUT ve SAW Yöntemiyle Kıyaslanması. Business & Management Studies: An International Journal, 8(1): 754-778. http://dx.doi.org/10.15295/bmij.v8i1.1355

Önder, E., & Boz, C. (2017). Comparing Macroeconomic Performance of the Union for the Mediterranean Countries Using Grey Relational Analysis and Multi-Dimensional Scaling. European Scientific Journal, 13, 285-299.

Özbek, A., & Demirkol, İ. (2019). Avrupa Birliği Ülkeleri İle Türkiye'nin Ekonomik Göstergelerinin Karşılaştırılması. Journal of Management & Economics, 26(1), 71-91. https://doi.org/10.18657/yonveek.418796

Özbek, A., & Erol, E. (2016). COPRAS ve MOORA yöntemlerinin depo yeri seçim problemine uygulanması. Ekonomi İşletme Siyaset ve Uluslararası İlişkiler Dergisi, 2(1), 23-42.

Paksoy, S. (2015). ÜLKE Göstergelerinin VIKOR Yöntemi ile Değerlendirilmesi. International Journal of Economic & Social Research, 11(2), 153-169.

Peng, X., Zhang, X., & Luo, Z. (2019). Pythagorean fuzzy MCDM method based on CoCoSo and CRITIC with score function for 5G industry evaluation. Artificial Intelligence Review, 53, 3813–3847. https://doi.org/10.1007/s10462-019-09780-x

Podvezko, V. (2011). The comparative analysis of MCDA methods SAW and COPRAS. Engineering Economics, 22(2), 134-146. http://dx.doi.org/10.5755/j01.ee.22.2.310

Poveda, A. C. (2011). Economic development and growth in Colombia: An empirical analysis with super-efficiency DEA and panel data models. Socio-Economic Planning Sciences, 45(4), 154-164. https://doi.org/10.1016/j.seps.2011.07.003

Reig-Martínez, E. (2013). Social and economic wellbeing in Europe and the Mediterranean Basin: Building an enlarged human development indicator. Social Indicators Research, 111(2), 527-547. https://doi.org/10.1007/s11205-012-0018-8

Safari, H., & Ebrahimi, E. (2014). Using modified similarity multiple criteria decision making technique to rank countries in terms of human development index. Journal of Industrial Engineering and Management (JIEM), 7(1), 254-275. https://doi.org/10.3926/jiem.837

Shao, Q., Wang, X., Zhou, Q., & Balogh, L. (2019). Pollution haven hypothesis revisited: a comparison of the BRICS and MINT countries based on VECM approach. Journal of Cleaner Production, 227, 724-738. https://doi.org/10.1016/j.jclepro.2019.04.206

Sieng, L. W., & Yussof, I. (2017). Comparative study of Malaysia human capital with selected ASEAN and Developed Countries: A fuzzy TOPSIS method. Geografia-Malaysian Journal of Society and Space, 11(6), 11-22.

Skare, M., & Rabar, D. (2017). Measuring sources of economic growth in OECD countries. Engineering Economics, 28(4), 386-400. https://doi.org/10.5755/j01.ee.28.4.18502

Soltanpanah, H., Farughi, H., & Golabi, M. (2010). Utilization and comparison of multi attribute decision techniques to rank countries upon human development rate. International Research Journal of Finance and Economics, 60, 175-188.

Şahin, C. & Öztel, A. (2017). Ülkelerin Yaşanabilirlik Düzeylerinin COPRAS Yöntemiyle Karşılaştırmalı Analizi: BRICS Ülkeleri ve Türkiye. USOBED Uluslararası Batı Karadeniz Sosyal ve Beşeri Bilimler Dergisi, 1(1), 75-84.

Şenol, Z., & Ulutaş, A. (2018). Muhasebe temelli performans ölçümleri ile piyasa temelli performans ölçümlerinin CRITIC ve ARAS yöntemleriyle değerlendirilmesi. Finans Politik & Ekonomik Yorumlar, 55(641), 83-102.

Tavana, M., Momeni, E., Rezaeiniya, N., Mirhedayatian, S. M., & Rezaeiniya, H. (2013). A novel hybrid social media platform selection model using fuzzy ANP and COPRAS-G. Expert Systems with Applications, 40(14), 5694-5702. https://doi.org/10.1016/j.eswa.2013.05.015

Ulutaş, A., & Karaköy, Ç. (2019). CRITIC ve ROV yöntemleri ile bir kargo firmasının 2011-2017 yılları sırasındaki performansının analiz edilmesi. MANAS Sosyal Araştırmalar Dergisi, 8(1), 223-230.

UNDP (2020). Unıted Nations Development Programme Human Development Reports. 02.06.2020 tarihinde http://hdr.undp.org/en/content/human-development-index-hdi sayfasından erişilmiştir.

Vierstraete, V. (2012). Efficiency in human development: A data envelopment analysis. The European Journal of Comparative Economics, 9(3), 425.

Vujičić, M. D., Papić, M. Z., & Blagojević, M. D. (2017). Comparative analysis of objective techniques for criteria weighing in two MCDM methods on example of an air conditioner selection. Tehnika, 72(3), 422-429. https://doi.org/10.5937/tehnika1703422V

World Bank (2018). The World Bank indicators. 12.06.2020 tarihinde https://data.worldbank.org/indicator sayfasından erişilmiştir.

Yalçın, N., & Karakaş, E. Kurumsal Sürdürülebilirlik Performans Analizinde CRITIC-EDAS Yaklaşımı. Çukurova Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 34(4), 147-162.

Yalçınkaya, Ö. G. Ö., & Temelli, Ö. G. F. (2014). Ekonomik Büyüme ile Cari İşlemler Dengesi Arasındaki İlişki: BRICS ve MINT (1992-2013). Süleyman Demirel Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 19(4), 201-224.

Yazdani, M., Alidoosti, A., & Zavadskas, E. K. (2011). Risk analysis of critical infrastructures using fuzzy COPRAS. Economic research-Ekonomska istraživanja, 24(4), 27-40. https://doi.org/10.1080/1331677X.2011.11517478