Amanda Sizo, Alexandra Oliveira, Adriano Lino, Álvaro Rocha, Luís Paulo Reis
Scientific journals often rely on informal methods to evaluate reviewers, such as editor ratings and author feedback. Reviewer self-assessment offers a promising, yet underexplored, approach to improving the peer-review process. This study examined the factors associated with reviewers’ self-assessments. We surveyed 642 reviewers and editors from three Information Systems (IS) conferences (January–February 2020), and 144 responses were analyzed using quantitative inferential statistics. Most respondents were male (72.2%) and based in Europe (59%). We found no significant association between self-assessment and conventional experience markers (reviewing and publishing experiences). In contrast, significant associations were observed between higher self-assessment and the perceived importance of feedback from editors (χ2 = 19.689, p ≈ 0.002), feedback from authors (χ2 = 25.168, p < 0.001), and formal training (χ2 = 14.64, p ≈ 0.047). Although our sample comes from IS settings, these mechanisms are process-based; therefore, these findings could be extended to a broader peer review ecosystem. Overall, organizational interventions, structured feedback from editors and authors, and formal training are more closely related to reviewers’ self-assessments than accumulated publishing or reviewing experience. © The Author(s) 2025.
LIACC - Artificial Intelligence and Computer Science Laboratory, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias S/N, Porto, 4200-465, Portugal; Polytechnic Institute of Porto School of Health, Polytechnic Institute of Porto, Porto, Portugal; Institute of Engineering and Geosciences, Federal University of Western Pará, Santarém, Brazil; Faculty of Economics and Business, Brawijaya University, Malang, Indonesia
