Good practice for single-valued neutrosophic ratings in expert panels: scale design, aggregation and the elicitation of indeterminacy
DOI:
https://doi.org/10.5281/zenodo.22712798Palavras-chave:
single-valued neutrosophic numbers, linguistic rating scales, indeterminacy elicitation, aggregation operators, expert panels, educational evaluationResumo
Single-valued neutrosophic numbers record truth, indeterminacy and falsity as separate components and are widely used to aggregate expert ratings in educational evaluation. This article derives guidance for such applications from exact results, a reanalysis of two published studies in which the author took part, and a simulation. With a fixed label-to-triple table, the score of the mean triple stays close to the mean rating: it preserves dominance and can order two panels differently only when their means differ by at most an exactly computed gap, 0.025 for twelve raters on an 11-level scale. On a published panel of 12 teachers rating 12 teamwork factors it reproduced the mean-rating order. Two design choices deserve attention: mapping the top label exactly to (1, 0, 0) makes the equal-weight averaging operator tie every item with one top rating, 9 factors on that panel, and contrasts between groups of factors are best reported with resampling ranges. The framework's distinctive contribution appears when indeterminacy is elicited from raters. In a simulation with 432 conditions, a score using reported hesitation selected the decision maker's best item more often than the mean rating when hesitation tracked item ambiguity and ambiguity was to be penalised, by 20.3 percentage points in a reference simulated condition, and less often when no penalty was wanted or hesitation ran against ambiguity. In published neutrosophic Z-number assessments, the group comparison held under each of five prespecified summaries. Seven recommendations for practice follow.
Received: 1 August 2026 | Peer review completed: 10 September 2026 | Accepted: 12 September 2026
Referências
[1] L. Zadeh, "Fuzzy sets," Information and Control, vol. 8, no. 3, pp. 338-353, 1965, doi: 10.1016/S0019-9958(65)90241-X.
[2] K. T. Atanassov, "Intuitionistic fuzzy sets," Fuzzy Sets and Systems, vol. 20, no. 1, pp. 87-96, 1986, doi: 10.1016/S0165-0114(86)80034-3.
[3] F. Smarandache, "Neutrosophic set - a generalization of the intuitionistic fuzzy set," in 2006 IEEE International Conference on Granular Computing, IEEE, 2006, pp. 38-42, doi: 10.1109/GRC.2006.1635754.
[4] J. Ye, "A multicriteria decision-making method using aggregation operators for simplified neutrosophic sets," Journal of Intelligent & Fuzzy Systems, vol. 26, no. 5, pp. 2459-2466, 2014, doi: 10.3233/IFS-130916.
[5] J.-J. Peng, J.-Q. Wang, J. Wang, H.-Y. Zhang, and X.-H. Chen, "Simplified neutrosophic sets and their applications in multi-criteria group decision-making problems," International Journal of Systems Science, vol. 47, no. 10, pp. 2342-2358, 2016, doi: 10.1080/00207721.2014.994050.
[6] P. Biswas, S. Pramanik, and B. C. Giri, "TOPSIS method for multi-attribute group decision-making under single-valued neutrosophic environment," Neural Computing and Applications, vol. 27, no. 3, pp. 727-737, 2016, doi: 10.1007/s00521-015-1891-2.
[7] D. Stanujkic et al., "A single valued neutrosophic extension of the simple WISP method," Informatica, pp. 635-651, 2022, doi: 10.15388/22-infor483.
[8] J. A. Urrutia Guevara, E. E. Pino Andrade, N. V. Quevedo Arnaiz, and D. M. Macazana Fernández, "Neutrosophic analysis of group dynamics and teamwork," Neutrosophic Sets and Systems, vol. 62, pp. 217-226, 2023, doi: 10.5281/zenodo.10436938.
[9] A. J. Berrospi Zapata, J. Pérez Soncco, R. E. Alcantara Poma, L. G. Lurquin Valdivieso, J. R. Ramírez Del Águila, and D. M. Macazana Fernández, "Evaluating English oral production in university students through Google Meet using neutrosophic Z-numbers," Neutrosophic Sets and Systems, vol. 84, pp. 71-84, 2025, doi: 10.5281/zenodo.15640322.
[10] D. Blackwell, "Equivalent comparisons of experiments," The Annals of Mathematical Statistics, vol. 24, no. 2, pp. 265-272, 1953, doi: 10.1214/aoms/1177729032.
[11] S. S. Stevens, "On the theory of scales of measurement," Science, vol. 103, no. 2684, pp. 677-680, 1946, doi: 10.1126/science.103.2684.677.
[12] T. M. Liddell and J. K. Kruschke, "Analyzing ordinal data with metric models: What could possibly go wrong?," Journal of Experimental Social Psychology, vol. 79, pp. 328-348, 2018, doi: 10.1016/j.jesp.2018.08.009.
[13] P.-C. Bürkner and M. Vuorre, "Ordinal regression models in psychology: A tutorial," Advances in Methods and Practices in Psychological Science, vol. 2, no. 1, pp. 77-101, 2019, doi: 10.1177/2515245918823199.
[14] A. Robitzsch, "Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation methods," Frontiers in Education, vol. 5, 2020, doi: 10.3389/feduc.2020.589965.
[15] R. Şahin, "Multi-criteria neutrosophic decision making method based on score and accuracy functions under neutrosophic environment," arXiv:1412.5202, 2014.
[16] Nancy and H. Garg, "An improved score function for ranking neutrosophic sets and its application to decision-making process," International Journal for Uncertainty Quantification, vol. 6, no. 5, pp. 377-385, 2016, doi: 10.1615/Int.J.UncertaintyQuantification.2016018441.
[17] Z. Xu, "Intuitionistic fuzzy aggregation operators," IEEE Transactions on Fuzzy Systems, vol. 15, no. 6, pp. 1179-1187, 2007, doi: 10.1109/TFUZZ.2006.890678.
[18] K. Kumar and S.-M. Chen, "Multiattribute decision making based on the improved intuitionistic fuzzy Einstein weighted averaging operator of intuitionistic fuzzy values," Information Sciences, vol. 568, pp. 369-383, 2021, doi: 10.1016/j.ins.2021.03.020.
[19] Y. He, H. Chen, L. Zhou, J. Liu, and Z. Tao, "Intuitionistic fuzzy geometric interaction averaging operators and their application to multi-criteria decision making," Information Sciences, vol. 259, pp. 142-159, 2014, doi: 10.1016/j.ins.2013.08.018.
[20] H. Garg, "Novel neutrality operation-based Pythagorean fuzzy geometric aggregation operators for multiple attribute group decision analysis," International Journal of Intelligent Systems, vol. 34, no. 10, pp. 2459-2489, 2019, doi: 10.1002/int.22157.
[21] D. Ellsberg, "Risk, ambiguity, and the Savage axioms," The Quarterly Journal of Economics, vol. 75, no. 4, p. 643, 1961, doi: 10.2307/1884324.
[22] I. Gilboa and D. Schmeidler, "Maxmin expected utility with non-unique prior," Journal of Mathematical Economics, vol. 18, no. 2, pp. 141-153, 1989, doi: 10.1016/0304-4068(89)90018-9.
[23] P. Klibanoff, M. Marinacci, and S. Mukerji, "A smooth model of decision making under ambiguity," Econometrica, vol. 73, no. 6, pp. 1849-1892, 2005, doi: 10.1111/j.1468-0262.2005.00640.x.
[24] E. Triantaphyllou and A. Sánchez, "A sensitivity analysis approach for some deterministic multi-criteria decision-making methods," Decision Sciences, vol. 28, no. 1, pp. 151-194, 1997, doi: 10.1111/j.1540-5915.1997.tb01306.x.
[25] R. Lahdelma, J. Hokkanen, and P. Salminen, "SMAA - Stochastic multiobjective acceptability analysis," European Journal of Operational Research, vol. 106, no. 1, pp. 137-143, 1998, doi: 10.1016/S0377-2217(97)00163-X.
[26] T. P. Morris, I. R. White, and M. J. Crowther, "Using simulation studies to evaluate statistical methods," Statistics in Medicine, vol. 38, no. 11, pp. 2074-2102, 2019, doi: 10.1002/sim.8086.
[27] J. S. Chai et al., "New similarity measures for single-valued neutrosophic sets with applications in pattern recognition and medical diagnosis problems," Complex & Intelligent Systems, vol. 7, no. 2, pp. 703-723, 2021, doi: 10.1007/s40747-020-00220-w.
[28] L. Martinez and F. Herrera, "A 2-tuple fuzzy linguistic representation model for computing with words," IEEE Transactions on Fuzzy Systems, vol. 8, no. 6, pp. 746-752, 2000, doi: 10.1109/91.890332.
[29] S. Du, J. Ye, R. Yong, and F. Zhang, "Some aggregation operators of neutrosophic Z-numbers and their multicriteria decision making method," Complex & Intelligent Systems, vol. 7, no. 1, pp. 429-438, 2021, doi: 10.1007/s40747-020-00204-w.
[30] G. Beliakov, A. Pradera, and T. Calvo, Aggregation Functions: A Guide for Practitioners (Studies in Fuzziness and Soft Computing). Springer, 2007, doi: 10.1007/978-3-540-73721-6.
[31] M. Grabisch, J.-L. Marichal, R. Mesiar, and E. Pap, "Aggregation functions: Means," Information Sciences, vol. 181, no. 1, pp. 1-22, 2011, doi: 10.1016/j.ins.2010.08.043.
[32] S. A. Wind and M. E. Peterson, "A systematic review of methods for evaluating rating quality in language assessment," Language Testing, vol. 35, no. 2, pp. 161-192, 2018, doi: 10.1177/0265532216686999.
[33] M. Uto and M. Ueno, "A generalized many-facet Rasch model and its Bayesian estimation using Hamiltonian Monte Carlo," Behaviormetrika, vol. 47, no. 2, pp. 469-496, 2020, doi: 10.1007/s41237-020-00115-7.
[34] M. Uto, "A Bayesian many-facet Rasch model with Markov modeling for rater severity drift," Behavior Research Methods, vol. 55, no. 7, pp. 3910-3928, 2022, doi: 10.3758/s13428-022-01997-z.
[35] H. Levy, "Stochastic dominance and expected utility: Survey and analysis," Management Science, vol. 38, no. 4, pp. 555-593, 1992, doi: 10.1287/mnsc.38.4.555.
[36] E. Hüllermeier and W. Waegeman, "Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods," Machine Learning, vol. 110, no. 3, pp. 457-506, 2021, doi: 10.1007/s10994-021-05946-3.
[37] R. T. Clemen and R. L. Winkler, "Combining probability distributions from experts in risk analysis," Risk Analysis, vol. 19, no. 2, pp. 187-203, 1999, doi: 10.1111/j.1539-6924.1999.tb00399.x.
[38] A. O'Hagan, "Expert knowledge elicitation: Subjective but scientific," The American Statistician, vol. 73, no. sup1, pp. 69-81, 2019, doi: 10.1080/00031305.2018.1518265.
[39] M. G. Kendall, "A new measure of rank correlation," Biometrika, vol. 30, no. 1-2, pp. 81-93, 1938, doi: 10.1093/biomet/30.1-2.81.
[40] B. Efron, "Bootstrap methods: Another look at the jackknife," The Annals of Statistics, vol. 7, no. 1, 1979, doi: 10.1214/aos/1176344552.
[41] D. Dubois, "The role of fuzzy sets in decision sciences: Old techniques and new directions," Fuzzy Sets and Systems, vol. 184, no. 1, pp. 3-28, 2011, doi: 10.1016/j.fss.2011.06.003.
[42] L. A. Zadeh, "A note on Z-numbers," Information Sciences, vol. 181, no. 14, pp. 2923-2932, 2011, doi: 10.1016/j.ins.2011.02.022.
[43] H. B. Mann and D. R. Whitney, "On a test of whether one of two random variables is stochastically larger than the other," The Annals of Mathematical Statistics, vol. 18, no. 1, pp. 50-60, 1947, doi: 10.1214/aoms/1177730491.
[44] R. A. Espín Andrade, E. Fernández, and E. González, "Compensatory fuzzy logic: A frame for reasoning and modeling preference knowledge in intelligent systems," in Soft Computing for Business Intelligence (Studies in Computational Intelligence). Springer, 2014, pp. 3-23, doi: 10.1007/978-3-642-53737-0_1.
[45] A. Vargha and H. D. Delaney, "A critique and improvement of the CL common language effect size statistics of McGraw and Wong," Journal of Educational and Behavioral Statistics, vol. 25, no. 2, pp. 101-132, 2000, doi: 10.3102/10769986025002101.
Downloads
Publicado
Edição
Secção
Licença
Direitos de Autor (c) 2026 Dante Manuel Macazana Fernández
Este trabalho encontra-se publicado com a Licença Internacional Creative Commons Atribuição 4.0.
