Confirmatory bias in peer review

2020
A reduction in reviewer's recommendation quality may be caused by a limitation of time or cognitive overload that comes from the level of redundancy, contradiction and inconsistency in the research. Some adaptive mechanisms by reviewers who deal with situations of information overload may be chunking single pieces of manuscript information into generic terms, unsystematic omission of research details, queuing of information processing, and prematurely stop the manuscript evaluation. Then, how would a reviewer optimize attention to positive and negative attributes of a manuscript before making a recommendation? How a reviewer's characteristics such as her prior belief about the manuscript quality and manuscript evaluation cost, affect her attention allocation and final recommendation? To answer these questions, we use a probabilistic model in which a reviewer chooses the optimal evaluation strategy by trading off the value and cost of review information about the manuscript quality. We find that a reviewer could exhibit a confirmatory behavior under which she pays more attention to the type of manuscript attributes that favor her prior belief about the manuscript quality. Then, confirmatory bias could be an optimal behavior of the reviewers that optimize attention to positive and negative manuscript attributes under information overload. We also show that reviewer's manuscript evaluation cost plays a key role in determining whether she may exhibit confirmatory bias. Moreover, when the reviewer's prior belief about the manuscript quality is low enough, the probability of obtaining a positive review signal decreases with reviewer's manuscript evaluation cost, for a sufficiently high cost.
SCIENTOMETRICS
页码:517-533|卷号:123|期号:1
ISSN:0138-9130
来源机构
University of Granada
收录类型
SSCI
发表日期
2020
学科领域
循证社会科学-综合
国家
西班牙
语种
英语
DOI
10.1007/s11192-020-03357-0
其他关键词
INFORMATION
EISSN
1588-2861
被引频次(WOS)
6
被引更新日期
2022-01
关键词
Peer review Confirmatory bias Optimal behavior Evaluation cost Reviewer recommendation