Institutional design and biases in evaluation reports by international organizations
By: Echkhard, S., Jankauskas, V. and Leuschner, E
.
Material type: 
BookPublisher: Public Administration Review Description: 84(3), May-Jun, 2024: p.560-573.Subject(s): Institutional design of evaluation systems, Evaluation bias, Bureaucratic influence, Positive assessments, Deep learning analysis, United Nations organizations, Evidence-based policymaking, Operative units, Central evaluation units, Report commissioning, Public sector evaluation, Organizational performance, Evaluation system design, Political perception, Evaluation outcomes, International organizations, Sentence-level analysis, Evaluation report content, Functional vs politicized tools, Institutional accountability
In:
Public Administration ReviewSummary: Governments spend hundreds of millions on evaluations to assess the performance of public organizations. In this article, we scrutinize whether variation in the institutional design of evaluation systems leads to biases in evaluation findings. Biases may emerge because influence over evaluation processes could enable the bureaucracy to present its work in a more positive way. We study evaluation reports published by nine international organizations (IOs) of the United Nations system. We use deep learning to measure the share of positive assessments at the sentence level per evaluation report as a proxy for the positivity of evaluation results. Analyzing 1082 evaluation reports, we find that reports commissioned by operative units, as compared to central evaluation units, systematically contain more positive assessments. Theoretically, this link between institutional design choices and evaluation outcomes may explain why policymakers perceive similar tools for evidence-based policymaking as functional in some organizations, and politicized in others.- Reproduced
https://onlinelibrary.wiley.com/doi/10.1111/puar.13705
| Item type | Current location | Call number | Vol info | Status | Date due | Barcode |
|---|---|---|---|---|---|---|
Articles
|
Indian Institute of Public Administration | 84(3), May-Jun, 2024: p.560-573 | Available | AR132375 |
Governments spend hundreds of millions on evaluations to assess the performance of public organizations. In this article, we scrutinize whether variation in the institutional design of evaluation systems leads to biases in evaluation findings. Biases may emerge because influence over evaluation processes could enable the bureaucracy to present its work in a more positive way. We study evaluation reports published by nine international organizations (IOs) of the United Nations system. We use deep learning to measure the share of positive assessments at the sentence level per evaluation report as a proxy for the positivity of evaluation results. Analyzing 1082 evaluation reports, we find that reports commissioned by operative units, as compared to central evaluation units, systematically contain more positive assessments. Theoretically, this link between institutional design choices and evaluation outcomes may explain why policymakers perceive similar tools for evidence-based policymaking as functional in some organizations, and politicized in others.- Reproduced
https://onlinelibrary.wiley.com/doi/10.1111/puar.13705
There are no comments for this item.