The assessment that the secretary would not hire.
A few years ago, after presenting a methodology and the results of a public policy evaluation at a meeting of the National Council of State Planning Secretaries (Conseplan), I received a comment from one of the attendees that was as complimentary as it was disconcerting. The presentation had been very good, she said. The methodology was rigorous and comprehensive, and the objective results raised important questions. Before my ego could even enjoy the praise, she concluded: "I would never commission work of that kind."
The explanation was straightforward. Such a careful evaluation would inevitably uncover problems – design flaws, execution difficulties, results below expectations, unforeseen effects. The opposition would take it upon themselves to transform each finding into an accusation against the government. This person, however, had no guarantee that they would be able to use the conclusions to improve policy. They would immediately assume the costs of revealing the problems, without controlling the conditions necessary to address them.
I was reminded of this episode while reading Simon Schwartzman's recent article, "In the King's Ears," published in O Estado de S. Paulo on July 10, 2026. Schwartzman draws attention to a difficulty known to those who work with public policy. Formulating good diagnoses and good proposals, based on evidence, is only part of the job. To produce change, they need to overcome political distrust, budgetary constraints, and the executive capacity of governments. Evidence can help revise beliefs and guide choices, but it does not remove the costs, interests, and conflicts involved in any change.
Everyone seems to agree that public policies should be evaluated. Evaluation is associated with good governance, transparency, the rational use of resources, and institutional learning. The awarding of the Nobel Prize in Economics to researchers who developed evaluation methodologies has further reinforced the presence and importance of evaluation for public policies.
But this near-universal agreement masks an uncomfortable question: why would a leader commission an independent and thorough evaluation if the political risks of the results fall on them, while the benefits are uncertain and their ability to utilize the recommendations is limited?
This is the first of three conversations about the paradoxes of public policy evaluation. In this one, we will focus on the political incentives for evaluation – or for avoiding evaluations that might reveal more than the government wants or is able to address. In the second, we will discuss the limits of the evaluations themselves: what they can measure, explain, and attribute to policies. In the third, we will address the gap between producing knowledge, learning from it, and effectively transforming a public policy.
The risk is certain; the change , uncertain.
It would be easy to attribute that secretary's statement to politicians' resistance to evaluation, transparency, or evidence. But that would be unfair: he wasn't questioning the quality of the evaluation, he was questioning the political rationale of commissioning it. His comment revealed a calculation that deserves to be taken seriously.
In fact, the political costs of an evaluation tend to be immediate, concentrated, and visible. The benefits, on the other hand, are uncertain, dispersed, and often delayed. A problem identified today may make headlines tomorrow. A successful correction may only produce results several years later, when another government is in power.
Blame also tends to be concentrated, while merit is diffuse. Public policies have a history. They accumulate decisions, rules, commitments, bureaucratic routines, and vested interests built up over many years. Yet, in the public eye, evaluation rarely distinguishes between inherited problems and those produced by the current administration. The government official who commissions and publicizes the evaluation study may end up being held responsible for everything it reveals.
There is yet another asymmetry. The manager controls the decision to commission the evaluation, but does not necessarily control the conditions for using its results. A recommendation may require additional resources, legislative support, coordination with other bodies, changes in bureaucracy, or confronting organized interests. Identifying a problem does not mean having the power to solve it.
Furthermore, knowledge creates obligations. Before evaluation, difficulties can be attributed to a lack of information, uncertainty, or the time required for results to appear. Once a problem is demonstrated, it becomes more difficult to ignore it. Evaluation not only expands available knowledge, but also increases the responsibility of those who have received that knowledge.
The secretary's dilemma was therefore real. The risk of evaluation was immediate and fell on the person who hired him. The benefits of the learning process would depend on other actors, uncertain circumstances, and often a political timeframe that extended beyond the term of office.
The risk was certain. The possibility of learning and changing was not.
Evidence alone does not govern.
Schwartzman's article examines the same problem from a different angle. While the secretary's episode calls into question the incentives for producing an evaluation, the studies discussed by Simon show what happens when diagnoses and proposals have already reached managers.
The results indicate that public managers can value evidence, change their beliefs, and recognize the quality of a proposal. However, the decision to act depends on other factors: cost, feasibility, political support, administrative capacity, and trust in the person presenting the recommendation.
One of the studies mentioned by Schwartzman showed that the content of the proposal was not enough to determine its acceptance. Mayors tended to welcome it when it came from politically aligned organizations and ignore it when it was presented by groups identified with the opposition. Institutions with a good reputation, but without a clear alignment, produced intermediate effects. In other words, the messenger could have more weight than the message itself.
Evidence, therefore, does not reach the government without identity. Its reception depends on who produces it, who presents it, and the trust – technical and political, more political than technical, I would say – attributed to these actors. Good data, rigorous methods, and consistent arguments are not, by themselves, sufficient to guide decisions.
This finding contradicts a widespread belief among evaluation specialists, researchers, and policymakers: that evidence speaks for itself and, once known, naturally leads to corrective action. That's not how governments work. And, strictly speaking, that's not how any organization works.
Public decisions are made in environments of uncertainty, conflict, and scarcity. Different groups compete for resources, priorities, and interpretations of problems. The same evidence can support different decisions, depending on the objectives pursued, the risks involved, and the position of the decision-maker. Even when there is agreement on the diagnosis, there may be disagreement on what to do, who should pay, or who will lose ground.
In the case of evaluations, the problem identified by Schwartzman becomes even more delicate. It's not just the quality of the evidence that matters, but also who produces it, how it is received, and who will have the legitimacy to interpret it publicly. A proposal can be ignored without major consequences. Evaluation, on the other hand, often begins by revealing that something did not work as planned. It may show that the most vulnerable beneficiaries were less reached, that resources arrived late, that the instruments were inadequate, that the objectives were incompatible, or that the results fell far short of official announcements. Before offering a solution, evaluation disrupts the narrative of success built around the policy.
By commissioning an independent assessment, therefore, the government not only assumes the risk of potentially inconvenient results, but also opens the door for those results to be appropriated and interpreted by actors over whom it has little or no control.
Therefore, the problem is not simply about convincing the ruler to listen to good recommendations. It is necessary to understand why he would agree to produce and disseminate evidence that could weaken his position before he even has the opportunity to use it.
Formulating the right proposals is difficult enough. Producing rigorous diagnoses of current policies can be even riskier.
Evaluating further is not enough.
The conclusion cannot be that governments should avoid rigorous evaluations or that their results need to be protected from public debate. Public policies mobilize societal resources, affect rights, and distribute opportunities. They must, therefore, be examined with independence and transparency.
But simply repeating that more evaluation is needed is not enough. Making evaluations mandatory does not eliminate the incentives that lead managers to restrict questions, select indicators, control the dissemination of results, or commission studies only when they believe they know the answers in advance.
In political environments where every identified problem is immediately transformed into proof of incompetence, evaluation can encourage more defense than learning. Evaluations continue to exist, but they often become bureaucratic requirements, instruments of legitimation, or technical rituals with little influence on the decisions that actually matter for the quality of politics.
The solution does not lie in reducing transparency, nor in imagining that simply making evaluation mandatory will produce learning. As long as acknowledging a problem is politically riskier than ignoring it, there will be incentives to restrict questions, choose convenient indicators, and reduce the scope of evaluations, which will tend to be more defensive, controlled, and less useful for correcting policies.
That secretary's speech may have been less cynical than realistic. He wasn't afraid of knowledge itself. He was afraid of bearing the costs of revealing problems he might not have the power to solve. The challenge, therefore, is not only to convince governments that assessment is important: it's to create conditions so that knowing about problems is no more risky than ignoring them.
But there is another question. Even when governments decide to evaluate, to what extent can evaluations measure, explain, and attribute the results of policies operating in complex realities? That will be our next conversation.
This text does not necessarily reflect the opinion of Unicamp.
Cover photo:
