Do we really need confidence intervals in the new statistics?

2019
This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as 'effect' sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each one is highly predictable from the other. CIs are supposed to be a measure of likelihood or uncertainty in the results, showing a range of possible effect sizes that could have been produced by random sampling variation alone. NNTD is supposed to be a measure of the robustness of the effect size to any variation, including that produced by missing data. Given that they are largely equivalent and interchangeable under the conditions tested here, the paper suggests that both are really measures of robustness. It concludes that NNTD is to be preferred because it requires many fewer assumptions, is more tolerant of missing data, is easier to explain, and directly addresses the key question of whether the underlying effect size is zero or not.
INTERNATIONAL JOURNAL OF SOCIAL RESEARCH METHODOLOGY
页码:281-291|卷号:22|期号:3
ISSN:1364-5579
收录类型
SSCI
发表日期
2019
学科领域
循证社会科学-综合
国家
英国
语种
英语
DOI
10.1080/13645579.2018.1525064
其他关键词
NULL-HYPOTHESIS; P-VALUES; TESTS; FALLACY
EISSN
1464-5300
资助机构
ESRCUK Research & Innovation (UKRI)Economic & Social Research Council (ESRC) [ES/J01172X/2, ES/N012046/1] Funding Source: UKRI
被引频次(WOS)
7
被引更新日期
2022-01
来源机构
Durham University Durham University
关键词
Confidence intervals significance tests new statistics number needed to disturb research trustworthiness analysing social science data