Wide or incompatible? Separating the width and the non-overlap of expert intervals when monetising intangible quality costs
DOI:
https://doi.org/10.5281/zenodo.22755556Keywords:
quality costs, expert elicitation, interval aggregation, interval non-overlap, typed abstention, sensitivity analysisAbstract
Expert intervals can be uncertain in two observable ways: each interval can be wide, and intervals can fail to overlap. The interval mean used to monetise a panel's judgements does not distinguish them. We reanalyse a published estimate of intangible quality costs in the scrap reception of a steel mill, where seven experts gave intervals for the share of sales affected, and report a limiting result: keeping the two indicators apart changes almost nothing that is decided here. Code reproduces the published total (USD 579614.91) and its split across four intangible risks exactly; the weighted Eq. (14) would give USD 693898 with the printed weights and USD 582129 once they are normalised. Two intervals end at 0.15 and two start at 0.16, so four of 21 pairs are separated by an open gap of 0.01, the recording resolution. Under five assumed coverage readings, between-interval dispersion accounts for 0.18 to 0.65 of pooled variance; the data identify none of the readings. No interval summary can reorder the intangible costs, which share one rate, and under hypothetical budgets a typed rule abstains exactly where the averaged interval does, changing only the stated reason. In an exploratory simulation, typed routing beat both fixed follow-ups in none of 648 conditions without a mixture of panel types and in 35 of 1620 conditions overall, all with the true rate near the threshold; always requesting information had the lowest mean regret, and a bias shared by all experts removed every advantage.
References
[1] A. Schiffauerova and V. Thomson, "A review of research on cost of quality models and best practices," International Journal of Quality & Reliability Management, vol. 23, no. 6, pp. 647-669, 2006, doi: 10.1108/02656710610672470.
[2] M. Plewa, G. Kaiser, and E. Hartmann, "Is quality still free?," International Journal of Quality & Reliability Management, vol. 33, no. 9, pp. 1270-1285, 2016, doi: 10.1108/ijqrm-11-2014-0189.
[3] Y. Pérez Mayedo, Y. Velázquez Labrada, N. M. Álvarez Álvarez, L. R. Pérez Molina, M. Leyva Vázquez, and G. T. Shakila Devi, "Estimation of intangible quality costs using neutrosophic AHP-TOPSIS," Neutrosophic Sets and Systems, vol. 84, pp. 37-50, 2025, doi: 10.5281/zenodo.15640268.
[4] K. J. Kaplan, "On the ambivalence-indifference problem in attitude theory and measurement: A suggested modification of the semantic differential technique," Psychological Bulletin, vol. 77, no. 5, pp. 361-372, 1972, doi: 10.1037/h0032590.
[5] R. R. Yager, "Entropy and specificity in a mathematical theory of evidence," International Journal of General Systems, vol. 9, no. 4, pp. 249-260, 1983, doi: 10.1080/03081078308960825.
[6] J. Abellán and A. Masegosa, "Requirements for total uncertainty measures in Dempster–Shafer theory of evidence," International Journal of General Systems, vol. 37, no. 6, pp. 733-747, 2008, doi: 10.1080/03081070802082486.
[7] S. Ferson and L. R. Ginzburg, "Different methods are needed to propagate ignorance and variability," Reliability Engineering & System Safety, vol. 54, no. 2-3, pp. 133-144, 1996, doi: 10.1016/s0951-8320(96)00071-3.
[8] T. P. Morris, I. R. White, and M. J. Crowther, "Using simulation studies to evaluate statistical methods," Statistics in Medicine, vol. 38, no. 11, pp. 2074-2102, 2019, doi: 10.1002/sim.8086.
[9] S. K. Srivastava, "Towards estimating Cost of Quality in supply chains," Total Quality Management & Business Excellence, vol. 19, no. 3, pp. 193-208, 2008, doi: 10.1080/14783360701600605.
[10] H. Hussinki, T. King, J. Dumay, and E. Steinhöfel, "Accounting for intangibles: a critical review," Journal of Accounting Literature, vol. 47, no. 5, pp. 27-51, 2025, doi: 10.1108/jal-05-2022-0060.
[11] Q. Wu, J. Xuan, F. Zhou, Y. Mei, and J. Su, "New Quality Cost Framework (QCF) Based on the Hybrid Fuzzy MCDM Approach," Computational Intelligence and Neuroscience, vol. 2022, pp. 1-13, 2022, doi: 10.1155/2022/6416989.
[12] L. Pérez-Fernández, M. A. Sebastián, and C. González-Gaya, "Methodology to Optimize Quality Costs in Manufacturing Based on Multi-Criteria Analysis and Lean Strategies," Applied Sciences, vol. 12, no. 7, Art. no. 3295, 2022, doi: 10.3390/app12073295.
[13] A. O'Hagan et al., Uncertain Judgements: Eliciting Experts' Probabilities. Wiley, 2006, doi: 10.1002/0470033312.
[14] M. G. Morgan, "Use (and abuse) of expert elicitation in support of decision making for public policy," Proceedings of the National Academy of Sciences, vol. 111, no. 20, pp. 7176-7184, 2014, doi: 10.1073/pnas.1319946111.
[15] J. B. Soll and J. Klayman, "Overconfidence in Interval Estimates," Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 30, no. 2, pp. 299-314, 2004, doi: 10.1037/0278-7393.30.2.299.
[16] D. A. Moore and P. J. Healy, "The trouble with overconfidence," Psychological Review, vol. 115, no. 2, pp. 502-517, 2008, doi: 10.1037/0033-295x.115.2.502.
[17] A. Speirs-Bridge, F. Fidler, M. McBride, L. Flander, G. Cumming, and M. Burgman, "Reducing Overconfidence in the Interval Judgments of Experts," Risk Analysis, vol. 30, no. 3, pp. 512-523, 2010, doi: 10.1111/j.1539-6924.2009.01337.x.
[18] C. Genest and J. V. Zidek, "Combining Probability Distributions: A Critique and an Annotated Bibliography," Statistical Science, vol. 1, no. 1, 1986, doi: 10.1214/ss/1177013825.
[19] R. T. Clemen and R. L. Winkler, "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, vol. 19, no. 2, pp. 187-203, 1999, doi: 10.1111/j.1539-6924.1999.tb00399.x.
[20] A. R. Colson and R. M. Cooke, "Expert Elicitation: Using the Classical Model to Validate Experts’ Judgments," Review of Environmental Economics and Policy, vol. 12, no. 1, pp. 113-132, 2018, doi: 10.1093/reep/rex022.
[21] V. Hemming, M. A. Burgman, A. M. Hanea, M. F. McBride, and B. C. Wintle, "A practical guide to structured expert elicitation using the IDEA protocol," Methods in Ecology and Evolution, vol. 9, no. 1, pp. 169-180, 2018, doi: 10.1111/2041-210x.12857.
[22] I. J. Navarro, V. Yepes, and J. V. Martí, "Sustainability assessment of concrete bridge deck designs in coastal environments using neutrosophic criteria weights," Structure and Infrastructure Engineering, vol. 16, no. 7, pp. 949-967, 2020, doi: 10.1080/15732479.2019.1676791.
[23] P. Smets, "Analyzing the combination of conflicting belief functions," Information Fusion, vol. 8, no. 4, pp. 387-412, 2007, doi: 10.1016/j.inffus.2006.04.003.
[24] W. Liu, "Analyzing the degree of conflict among belief functions," Artificial Intelligence, vol. 170, no. 11, pp. 909-924, 2006, doi: 10.1016/j.artint.2006.05.002.
[25] R. R. Yager, "On the dempster-shafer framework and new combination rules," Information Sciences, vol. 41, no. 2, pp. 93-137, 1987, doi: 10.1016/0020-0255(87)90007-7.
[26] D. Dubois and H. Prade, "Representation and combination of uncertainty with belief functions and possibility measures," Computational Intelligence, vol. 4, no. 3, pp. 244-264, 1988, doi: 10.1111/j.1467-8640.1988.tb00279.x.
[27] S. Destercke, D. Dubois, and E. Chojnacki, "Possibilistic Information Fusion Using Maximal Coherent Subsets," IEEE Transactions on Fuzzy Systems, vol. 17, no. 1, pp. 79-92, 2009, doi: 10.1109/tfuzz.2008.2005731.
[28] D. Dubois, "On Ignorance and Contradiction Considered as Truth-Values," Logic Journal of IGPL, vol. 16, no. 2, pp. 195-216, 2008, doi: 10.1093/jigpal/jzn003.
[29] E. Hüllermeier and W. Waegeman, "Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods," Machine Learning, vol. 110, no. 3, pp. 457-506, 2021, doi: 10.1007/s10994-021-05946-3.
[30] T. L. Saaty, "A scaling method for priorities in hierarchical structures," Journal of Mathematical Psychology, vol. 15, no. 3, pp. 234-281, 1977, doi: 10.1016/0022-2496(77)90033-5.
[31] J. Aczél and T. Saaty, "Procedures for synthesizing ratio judgements," Journal of Mathematical Psychology, vol. 27, no. 1, pp. 93-102, 1983, doi: 10.1016/0022-2496(83)90028-7.
[32] E. Forman and K. Peniwati, "Aggregating individual judgments and priorities with the analytic hierarchy process," European Journal of Operational Research, vol. 108, no. 1, pp. 165-169, 1998, doi: 10.1016/s0377-2217(97)00244-0.
[33] W. Ossadnik, S. Schinke, and R. H. Kaspar, "Group Aggregation Techniques for Analytic Hierarchy Process and Analytic Network Process: A Comparative Analysis," Group Decision and Negotiation, vol. 25, no. 2, pp. 421-457, 2016, doi: 10.1007/s10726-015-9448-4.
[34] Y. Dong, G. Zhang, W.-C. Hong, and Y. Xu, "Consensus models for AHP group decision making under row geometric mean prioritization method," Decision Support Systems, vol. 49, no. 3, pp. 281-289, 2010, doi: 10.1016/j.dss.2010.03.003.
[35] T. L. Saaty and L. G. Vargas, "Uncertainty and rank order in the analytic hierarchy process," European Journal of Operational Research, vol. 32, no. 1, pp. 107-117, 1987, doi: 10.1016/0377-2217(87)90275-x.
[36] C.-L. Hwang and K. Yoon, Multiple Attribute Decision Making. Springer Berlin Heidelberg, 1981, doi: 10.1007/978-3-642-48318-9.
[37] M. Behzadian, S. Khanmohammadi Otaghsara, M. Yazdani, and J. Ignatius, "A state-of the-art survey of TOPSIS applications," Expert Systems with Applications, vol. 39, no. 17, pp. 13051-13069, 2012, doi: 10.1016/j.eswa.2012.05.056.
[38] P. Biswas, S. Pramanik, and B. C. Giri, "TOPSIS method for multi-attribute group decision-making under single-valued neutrosophic environment," Neural Computing and Applications, vol. 27, no. 3, pp. 727-737, 2016, doi: 10.1007/s00521-015-1891-2.
[39] I. Deli and Y. Şubaş, "A ranking method of single valued neutrosophic numbers and its applications to multi-attribute decision making problems," International Journal of Machine Learning and Cybernetics, vol. 8, no. 4, pp. 1309-1322, 2017, doi: 10.1007/s13042-016-0505-3.
[40] E. Triantaphyllou and A. Sánchez, "A Sensitivity Analysis Approach for Some Deterministic Multi‐Criteria Decision‐Making Methods*," Decision Sciences, vol. 28, no. 1, pp. 151-194, 1997, doi: 10.1111/j.1540-5915.1997.tb01306.x.
[41] J. Butler, J. Jia, and J. Dyer, "Simulation techniques for the sensitivity analysis of multi-criteria decision models," European Journal of Operational Research, vol. 103, no. 3, pp. 531-546, 1997, doi: 10.1016/s0377-2217(96)00307-4.
[42] R. Lahdelma, J. Hokkanen, and P. Salminen, "SMAA - Stochastic multiobjective acceptability analysis," European Journal of Operational Research, vol. 106, no. 1, pp. 137-143, 1998, doi: 10.1016/s0377-2217(97)00163-x.
[43] T. Tervonen and J. R. Figueira, "A survey on stochastic multicriteria acceptability analysis methods," Journal of Multi-Criteria Decision Analysis, vol. 15, no. 1-2, pp. 1-14, 2008, doi: 10.1002/mcda.407.
[44] M. G. Kendall and B. B. Smith, "The Problem of Rankings," The Annals of Mathematical Statistics, vol. 10, no. 3, pp. 275-287, 1939, doi: 10.1214/aoms/1177732186.
[45] A. F. Hayes and K. Krippendorff, "Answering the Call for a Standard Reliability Measure for Coding Data," Communication Methods and Measures, vol. 1, no. 1, pp. 77-89, 2007, doi: 10.1080/19312450709336664.
[46] B. Efron, "Bootstrap Methods: Another Look at the Jackknife," The Annals of Statistics, vol. 7, no. 1, 1979, doi: 10.1214/aos/1176344552.
[47] C. Chow, "On optimum recognition error and reject tradeoff," IEEE Transactions on Information Theory, vol. 16, no. 1, pp. 41-46, 1970, doi: 10.1109/tit.1970.1054406.
[48] Y. Yao, "Three-way decisions with probabilistic rough sets," Information Sciences, vol. 180, no. 3, pp. 341-353, 2010, doi: 10.1016/j.ins.2009.09.021.
[49] R. Howard, "Information Value Theory," IEEE Transactions on Systems Science and Cybernetics, vol. 2, no. 1, pp. 22-26, 1966, doi: 10.1109/tssc.1966.300074.
