CoARA will not save science from the tyranny of administrative evaluation

The Coalition for Advancing Research Assessment (CoARA) agreement is a cornerstone in the ongoing efforts to reform research evaluation. CoARA advocates for administrative evaluations of research that rely on peer review, supported by responsible metrics, as beneficial for both science and society. Its principles can be critically examined through the lens of Philip Kitcher’s concept of well-ordered science in a democratic society. From Kitcher’s perspective, CoARA’s approach faces two significant challenges: definitions of quality and impact are determined by governments or evaluation institutions rather than emerging from broad public deliberation, and a select group of scientists is empowered to assess research based on these predefined criteria. This creates susceptibility to both the "tyranny of expertise" and the "tyranny of ignorance" that Kitcher cautions against. Achieving Kitcher’s ideal would require limiting administrative evaluations to essential tasks, such as researcher recruitment and project funding, while establishing procedures grounded in principles of fairness.