Education, Information Credibility, and Control of Corruption

Cartoon by Winsor McCay (1930), archived by Alan Light on Flickr.

Here’s an interesting paper by Weitz-Shapiro and Winters (2014)  on the role of education in political control of corruption, which should be of interest to those working in the open government/transparency domains.

When are citizens most likely to hold politicians to account for wrongdoing? In a crowded information environment, political accountability can be achieved only if credible information is available and citizens are able to identify that information. In this paper, we argue that the ability to discern more from less credible information is increasing in citizen sophistication. Using data from an original survey experiment in Brazil, we show that all citizens react negatively to corruption allegations, but that highly educated respondents are more likely to punish credible accusations and to overlook less credible accusations. We then show, using municipal-level audit data, that voters are more likely to punish credible accusations of corruption in municipalities with high literacy rates. Our findings suggest a novel mechanism that may link increasing education with control of political corruption: educated citizens are better able to discern and therefore act on credible accusations.

And, from the conclusion, an important message on the credibility of institutions:

Our findings have interesting implications for our understanding of the relationship between education and political accountability. They suggest a new mechanism through which high educational attainment might decrease corruption—not through changes in preferences that may be associated with different education levels, but rather because more educated individuals are better able to discern more from less credible information and therefore are more likely to act on the former. These results should be heartening to governments, like Brazil’s, that have invested in the creation of reputable independent auditing and control units. As long as these agencies are able to maintain their reputation for high quality, we should expect their influence to grow as the population becomes increasingly educated.

Finally, I couldn’t help but notice that, indirectly, this paper is a good reminder of the validity of the principal-agent model of accountability. Even though it is now fashionable to criticize the model, despite its limitations, it is far from obsolete.

You can download the full paper here [PDF].

New Book on 25 Years of Participatory Budgeting

Screenshot 2014-06-09 17.17.40

A little while ago I mentioned the launch of the Portuguese version of the book organized by Nelson Dias, “Hope for Democracy: 25 Years of Participatory Budgeting Worldwide”.

The good news is that the English version is finally out. Here’s an excerpt from the introduction:

This book represents the effort  of more than forty authors and many other direct and indirect contributions that spread across different continents seek to provide an overview on the Participatory Budgeting (PB) in the World. They do so from different backgrounds. Some are researchers, others are consultants, and others are activists connected to several groups and social movements. The texts reflect this diversity of approaches and perspectives well, and we do not try to influence that.

(….)

The pages that follow are an invitation to a fascinating journey on the path of democratic innovation in very diverse cultural, political, social and administrative settings. From North America to Asia, Oceania to Europe, from Latin America to Africa, the reader will find many reasons to closely follow the proposals of the different authors.

The book  can be downloaded here [PDF]. I had the pleasure of being one of the book’s contributors, co-authoring an article with Rafael Sampaio on the use of ICT in PB processes: “Electronic Participatory Budgeting: False Dilemmas and True Complexities” [PDF].

While my perception may be biased, I believe this book will be a major contribution for researchers and practitioners in the field of participatory budgeting and citizen engagement in general. Congratulations to Nelson Dias and all the others who contributed their time and energy.

Social Accountability: What Does the Evidence Really Say?

So what does the evidence about citizen engagement say? Particularly in the development world it is common to say that the evidence is “mixed”. It is the type of answer that, even if correct in extremely general terms, does not really help those who are actually designing and implementing citizen engagement reforms.

This is why a new (GPSA-funded) work by Jonathan Fox, “Social Accountability: What does the Evidence Really Say” is a welcome contribution for those working with open government in general and citizen engagement in particular. Rather than a paper, this work is intended as a presentation that summarizes (and disentangles) some of the issues related to citizen engagement.

Before briefly discussing it, some definitional clarification. I am equating “social accountability” with the idea of citizen engagement given Jonathan’s very definition of  social accountability:

“Social accountability strategies try to improve public sector performance by bolstering both citizen engagement and government responsiveness”

In short, according to this definition, social accountability is defined, broadly, as “citizen participation” followed by government responsiveness, which encompasses practices as distinct as FOI law campaigns, participatory budgeting and referenda.

But what is new about Jonathan’s work? A lot, but here are three points that I find particularly important, based on a very personal interpretation of his work.

First, Jonathan makes an important distinction between what he defines as “tactical” and “strategic” social accountability interventions. The first type of interventions, which could also be called “naïve” interventions, are for instance those bounded in their approach (one tool-based) and those that assume that mere access to information (or data) is enough. Conversely, strategic approaches aim to deploy multiple tools and articulate society-side efforts with governmental reforms that promote responsiveness.

This distinction is important because, when examining the impact evaluation evidence, one finds that while the evidence is indeed mixed for tactical approaches, it is much more promising for strategic approaches. A blunt lesson to take from this is that when looking at the evidence, one should avoid comparing lousy initiatives with more substantive reform processes. Otherwise, it is no wonder that “the evidence is mixed.”

Second, this work makes an important re-reading of some of the literature that has found “mixed effects”, reminding us that when it comes to citizen engagement, the devil is in the details. For instance, in a number of studies that seem to say that participation does not work, when you look closer you will not be surprised that they do not work. And many times the problem is precisely the fact that there is no participation whatsoever. False negatives, as eloquently put by Jonathan.

Third, Jonathan highlights the need to bring together the “demand” (society) and “supply” (government) sides of governance. Many accountability interventions seem to assume that it is enough to work on one side or the other, and that an invisible hand will bring them together. Unfortunately, when it comes to social accountability it seems that some degree of “interventionism” is necessary in order to bridge that gap.

Of course, there is much more in Jonathan’s work than that, and it is a must read for those interested in the subject. You can download it here [PDF].

A Review of the Evidence on Open Budgeting

Brand new.

“A Review of the Evidence on Open Budgeting” is a recent report by the World Bank Institute’sCapacity Development and Results team. It explores key questions and existing evidence around the impact of open budgeting. Despite the growing body of literature, there remains limited substantiation for whether and how open budgeting contributes to reductions in poverty and improvements in the lives of the poor. This report pieces together the results chain presenting evidence for and against from the literature. It explores links between open budgeting and indicators of impact such as human development and public service delivery. The findings highlight the importance of measuring budget transparency, accountability, and participation. The findings show that the impact of institutional changes differ under varying conditions in specific contexts. The conclusions of the report point to the need for further investigation into impact and establishing effective measurement practices for monitoring related institutional change under varying conditions and different contexts.”

You can download the report here [PDF].

The Problem with Theory of Change

picture by skreened.com

If you are working in the fields of development or governance it’s highly likely that you’ve come across the term “theory of change” (ToC). At a conference a couple of weeks ago, while answering some questions, I mentioned that I preferred not to use the term. The comment didn’t go unnoticed by some witty observers on Twitter, and I was surprised by the number of people who came to me afterwards asking why I do not “like” theory of change.

I can see why some people are attracted to the term. First, “change” is a powerful word: it even helps win elections. And when it comes to governance issues, the need for change is almost a consensus. Second, the user of the word “theory” gives scientific verve to the conversation. However, the problem is precisely the appropriateness of its use if one thinks of the word in scientific terms. It seems that people are saying “theory” when they actually mean (at best) “hypothesis”.

We don’t have to go very far to find out what scientific theory actually is. Keeping to information that is just a click away, let’s take one of the definitions reproduced in Wikipedia’s entry for “theory”:

A scientific theory is a well-substantiated explanation of some aspect of the natural world, based on a body of facts that have been repeatedly confirmed through observation and experiment. Such fact-supported theories are not “guesses” but reliable accounts of the real world.

 Or “scientific theory”:

A theory is a good theory if it satisfies two requirements: It must accurately describe a large class of observations on the basis of a model that contains only a few arbitrary elements, and it must make definite predictions about the results of future observations.

 And here’s a rap video on the difference between theory and hypothesis:

Granted, the word “theory” is often used as a synonym of “hypothesis”, and even dictionaries do so. But the problem of this in the context of current usages of “theory of change” is that it masks the difference between what we know and do not know about something, often conveying a false sense of scientific rigor. And, particularly when it comes to issues such as development and governance, it is extremely important to have a clear distinction between well-substantiated explanations and every other color of hypotheses, assumptions, and guesses. In fact, in any field, it is a minimal requirement for the production of knowledge.

So here’s an interesting exercise. Search on the web for the use of “theory of change” combined with terms like “accountability” and “open government.” Find, for yourself, which ones are really “theories of change” or, rather, merely “hunches of change.”

Most likely, people will keep using theory of change indiscriminately until the next flavor of the moment comes up. In the meantime, beware.

***

Also read: Open Government, Feedback Loops and Semantic Extravaganza

Participation, Transparency and Accountability: Innovations in South Korea, Brazil, and the Philippines

A report by Brian Wampler for the Global Initiative for Fiscal Transparency (GIFT):

Citizen participation in budgetary and other fiscal processes has been expanding at international, national, and local levels over the past 15 years. The direct participation of citizens, it is hoped, will improve governance, limit misuse of public funds, and produce more informed, engaged citizens. At the national level, reformist governments now encourage the direct engagement of citizens during multiple moments of the policy cycle—from initial policy formulation to the oversight of policy implementation. Reformist governments hope to take advantage of increased citizen participation to increase their legitimacy, thus allowing them to change spending and policy priorities, increase state effectiveness by make public bureaucrats more responsive to citizens and elected officials, and, finally, ensure that the quality of public services improves. During the 1980s and 1990s, many subnational governments took advantage of policy decentralization to experiment with new institutional types. Direct citizen participation has been most robust at subnational levels due to the decreased costs and the greater direct impact of citizens on policymaking.

(….)

The main purpose of this report is to examine how three countries, South Korea, Brazil, and the Philippines, have made extensive efforts to create new institutions and policies that encourage the participation of citizens and CSOs in complex policy processes. South Korea developed an institutional arrangement based on policy experts, CSOs, and the Korean Development Institute. Brazil uses a model that relies extensively on the participation of citizens at multiple tiers of government. Finally, the Philippines use a mixed model that incorporates citizens and CSOs at national and subnational levels

(….)

Political reformers seeking to incorporate greater numbers of people into policymaking venues face a series of challenges. These include: (1) asymmetrical access to information as well as differing skills base to interpret information; (2) the difficultly of decision-making when groups grow in size; (3) a reduction in the importance of any single participant due to the greater number of participants; (4) political contestation over who has the right to participate; (5) who are the legitimate representatives of different groups; and (6) higher organizational costs (time, money, personnel). This report maps out how new participatory institutions and programs that are designed to help governments and their civil society allies draw citizens directly into decision-making processes.To explain the variation in the type of participatory experiences now used by different countries,we identify four factors that most strongly affect the types of participation-oriented reforms as well as the results. These four factors include: (a) presidential-level support for reform, (b) the configuration of civil society, (c) state capacity and (d) the geo-political direction of reform (topdown/center –periphery vs. bottom-up/periphery/center. It is the combination of these four factors that most strongly explains the type of institutions adopted in each of these countries.

Read the full report here [PDF]. 

When Citizen Engagement Saves Lives (and what we can learn from it)

When it comes to the relationship between participatory institutions and development outcomes, participatory budgeting stands out as one of the best examples out there. For instance, in a paper recently published in World Development,  Sonia Gonçalves finds that municipalities that adopted participatory budgeting in Brazil “favoured an allocation of public expenditures that closely matched the popular preferences and channeled a larger fraction of their total budget to key investments in sanitation and health services.”  As a consequence, the author also finds that this change in the allocation of public expenditures “is associated with a pronounced reduction in the infant mortality rates for municipalities which adopted participatory budgeting.”

Evolution of Expenditure Share in Health and Sanitation compared between adopters and non-adopters of PB (Goncalves 2013).

Evolution of  the share of expenditures in health and sanitation compared between adopters and non-adopters of participatory budgeting (Goncalves 2013).

Now, in an excellent new article published in Comparative Political Studies, the authors Michael Touchton and Brian Wampler come up with similar findings (abstract):

We evaluate the role of a new type of democratic institution, participatory budgeting (PB), for improving citizens’ well-being. Participatory institutions are said to enhance governance, citizens’ empowerment, and the quality of democracy, creating a virtuous cycle to improve the poor’s well-being. Drawing from an original database of Brazil’s largest cities over the last 20 years, we assess whether adopting PB programs influences several indicators of well-being inputs, processes, and outcomes. We find PB programs are strongly associated with increases in health care spending, increases in civil society organizations, and decreases in infant mortality rates. This connection strengthens dramatically as PB programs remain in place over longer time frames. Furthermore, PB’s connection to well-being strengthens in the hand of mayors from the nationally powerful, ideologically and electorally motivated Workers’ Party. Our argument directly addresses debates on democracy and well-being and has powerful implications for participation, governance, and economic development.

When put together, these findings provide compelling evidence for those who – often unfamiliar with the literature – question the effectiveness of participatory governance institutions. Surely, more research is needed, and different citizen engagement initiatives (and contexts) may lead to different results.

But these articles also bring another important takeaway for those working with development and public sector reform. And that is the need to consider the fact that participatory institutions (as most institutional reforms) may take time to produce desirable/noticeable effects. As noted by Touchton and Wampler:

 The relationships we describe between PB and health and sanitation spending, PB and CSOs, and PB and health care outcomes in this section are greater in magnitude and stronger in statistical significance for municipalities that have used PB for a longer period of time. Municipalities using PB for less than 4 years do exhibit lower infant mortality rates than municipalities that never adopted PB. However, there is no statistically significant difference in spending on health care and sanitation between municipalities using PB for less than 4 years and municipalities that never adopted the program. This demonstrates the benefits from adopting PB are not related to low-hanging fruit, but built over a great number of years. Our results imply PB is associated with long-term institutional and political change—not just short-term shifts in funding priorities .

If throughout the years participatory budgeting has produced  evidence of its effectiveness on a number of fronts (e.g. pro-poor spending), it is only 25 years after its first implementation in Brazil that we start to see systematic evidence of sound development outcomes such as reduction in infant mortality. In other words, rushing to draw conclusions at early stages of participatory governance interventions may result in misleading assessments. Even worse, it may lead to discontinuing efforts that are yet to bear fruit in the medium and longer terms.

10 Most Read Posts in 2013

Below is a selection of the 10 most read posts at DemocracySpot in 2013. Thanks to all of those who stopped by throughout the year, and happy 2014.

1. Does transparency lead to trust? Some evidence on the subject.

2. The Foundations of Motivation for Citizen Engagement

3. Open Government, Feedback Loops, and Semantic Extravaganza

4. Open Government and Democracy

5. What’s Wrong with e-Petitions and How to Fix them

6. Lawrence Lessig on Sortition and Citizen Participation

7. Unequal Participation: Open Government’s Unresolved Dilemma

8. The Effect of SMS on Participation: Evidence from Uganda

9. The Uncertain Relationship Between Open Data and Accountability

10. Lisbon Revisited: Notes on Participation

Why ‘I-Paid-A-Bribe’ Worked in India but Failed in China

source: China Daily

Interesting paper by Yuen Yuen Ang, Political Scientist at the University of Michigan:

Authoritarian states restrain online activism not only through repression and censorship, but also by indirectly weakening the ability of netizens to self-govern and constructively engage the state. I demonstrate this argument by comparing I-Paid-A-Bribe (IPAB) — a crowd-sourcing platform that collects anonymous reports of petty bribery — in India and China. Whereas IPAB originated and has thrived in India, a copycat effort in China fizzled out within months. Contrary to those who attribute China’s failed outcome to repression, I find that even before authorities shut down IPAB, the sites were already plagued by internal organizational problems that were comparatively absent in India. The study tempers expectations about the revolutionary effects of new media in mobilizing contention and checking corruption in the absence of a strong civil society.

And a brief video with Yuen Yuen

Also read

I Paid a Bribe. So What? 

Open Government and Democracy

Open Data and Citizen Engagement – Disentangling the Relationship

[This is a cross-post from Sunlight Foundation's  series OpenGov Conversations, an ongoing discourse featuring contributions from transparency and accountability researchers and practitioners around the world.] 

As asserted by Jeremy Bentham nearly two centuries ago, “[I]n the same proportion as it is desirable for the governed to know the conduct of their governors, is it also important for the governors to know the real wishes of the governed.” Although Bentham’s historical call may come across as obvious to some, it highlights one of the major shortcomings of the current open government movement: while a strong focus is given to mechanisms to let the governed know the conduct of their governors (i.e. transparency), less attention is given to the means by which the governed can express their wishes (i.e. citizen engagement).

But striking a balance between transparency and participation is particularly important if transparency is conceived as a means for accountability. To clarify, let us consider the role transparency (and data) plays in a simplified accountability cycle. As any accountability mechanism built on disclosure principles, it should require a minimal chain of events that can be summarized in the following manner: (1) Data is published; (2) The data published reaches its intended public; (3) Members of the public are able to process the data and react to it; and (4) Public officials respond to the public’s reaction or are sanctioned by the public through institutional means. This simplified path toward accountability highlights the limits of the disclosure of information. Even in the most simplified model of accountability, while essential, the disclosure of data accounts for no more than one-fourth of the accountability process. [Note 1 - see below]

But what are the conditions required to close the accountability cycle? First, once the data is disclosed (1), in order for it to reach its intended public (2), a minimal condition is the presence of info-mediators that can process open data in a minimally enabling environment (e.g. free and pluralistic media). Considering these factors are present, we are still only half way towards accountability. Nevertheless, the remaining steps (3 and 4) cannot be achieved in the absence of citizen engagement, notably electoral and participatory processes.

 

Beyond Elections

 

With regard to elections as a means for accountability, citizens may periodically choose to reward or sanction elected officials based on the information that they have received and processed. While this may seem a minor requisite for developed democracies like the US, the problem gains importance for a number of countries where open data platforms have launched but where elections are still a work in progress (in such cases, some research suggests that transparency may even backfire).

But, even if elections are in place, alone they might not suffice. The Brazilian case is illustrative and highlights the limits of representative systems as a means to create sustained interface between governments and citizens. Despite two decades of electoral democracy and unprecedented economic prosperity in the country, citizens suddenly went to the streets to demand an end to corruption, improvement in public services and… increased participation. Politicians, themselves, came to the quick realization that elections are not enough, as recently underlined by former Brazilian President Lula in an op ed at the New York Times “(….) people do not simply wish to vote every four years. They want daily interaction with governments both local and national, and to take part in defining public policies, offering opinions on the decisions that affect them each day.” If transparency and electoral democracy are not enough, citizen engagement remains as the missing link for open and inclusive governments.

 

Open Data And Citizen Engagement

 

Within an ecosystem that combines transparency and participation, examining the relationship between the two becomes essential. More specifically, a clearer understanding of the interaction between open data and participatory institutions remains a frontier to be explored. In the following paragraphs I put forward two issues, of many, that I believe should be considered when examining this interaction.

I) Behavior and causal chains

Evan Lieberman and his colleagues conducted an experiment in Kenya that provided parents with information about their children’s schools and how to improve their children’s learning. Nevertheless, to the disillusionment of many, despite efforts to provide parents with access to information, the intervention had no impact on parents’ behavior. Following this rather disappointing finding, the authors proceeded to articulating a causal chain that explores the link between access to information and behavioral change.

Information-Citizen Action Causal Chain

The Information-Citizen Action Causal Chain (Lieberman et al. 2013)

 

While the model put forward by the authors is not perfect, it is a great starting point and it does call attention to the dire need for a clear understanding of the ensemble of mechanisms and factors acting between access to data and citizen action.

II) Embeddedness in participatory arrangements

Another issue that might be worth examination relates to the extent to which open data is purposefully connected to participatory institutions or not. In this respect, much like the notion of targeted transparency, a possible hypothesis would be that open data is fully effective for accountability purposes only when the information produced becomes “embedded” in participatory processes.

This notion of “embeddedness” would call for hard thinking on how different participatory processes can most benefit from open data and its applications (e.g. visualizations, analysis). For example, the use of open data to inform a referendum process is potentially a very different type of use than within participatory budgeting process. Stemming from this reasoning, open data efforts should be increasingly customized to different existing participatory processes, hence increasing their embeddedness in these processes. This would be the case, for instance, when budget data visualization solutions are tailored to inform participatory budgeting meetings, thus creating a clear link between the consumption of that data and the decision-making process that follows.

Granted, information is per se an essential component of good participatory processes, and one can take a more or less intuitive view on which types of information are more suitable for one process or another. However, a more refined knowledge of how to maximize the impact of data in participatory processes is far from achieved and much more work is needed.

 

R&D For Data-Driven Participation

 

Coming up with clear hypotheses and testing them is essential if we are to move forward with the ecosystem that brings together open data, participation and accountability. Surely, many organizations working in the open government space are operating with limited resources, squeezing their budgets to keep their operational work going. In this sense, conducting experiments to test hypotheses may appear as a luxury that very few can afford.

Nevertheless, one of the opportunities provided by the use of technologies for civic behavior is that of potentially driving down the costs for experimentation. For instance, online and mobile experiments could play the role of tech-enabled (and affordable) randomized controlled trials, improving our understanding of how open data can be best used to spur collective action. Thinking of the ways in which technology can be used to conduct lowered costs experiments to shed light on behavioral and causal chains is still limited to a small number of people and organizations, and much work is needed on that front.

Yet, it is also important to acknowledge that experiments are not the only source of relevant knowledge. To stick with a simple example, in some cases even an online survey trying to figure out who is accessing data, what data they use, and how they use it may provide us with valuable knowledge about the interaction between open data and citizen action. In any case, however, it may be important that the actors working in that space agree upon a minimal framework that facilitates comparison and incremental learning: the field of technology for accountability desperately needs a more coordinated research agenda.

Citizen Data Platforms?

As more and more players engage in participatory initiatives, there is a significant amount of citizen-generated data being collected, which is important on its own. However, in a similar vein to government data, the potential of citizen data may be further unlocked if openly available to third parties who can learn from it and build upon it. In this respect, it might not be long before we realize the need to have adequate structures and platforms to host this wealth of data that – hopefully – will be increasingly generated around the world. This would entail that not only governments open up their data related to citizen engagement initiatives, but also that other actors working in that field – such as donors and NGOs – do the same. Such structures would also be the means by which lessons generated by experiments and other approaches are widely shared, bringing cumulative knowledge to the field.

However, as we think of future scenarios, we should not lose sight of current challenges and knowledge gaps when it comes to the relationship between citizen engagement and open data. Better disentangling the relationship between the two is the most immediate priority, and a long overdue topic in the open government conversation.

 

Notes

 

Note 1: This section of this post is based on arguments previously developed in the article, “The Uncertain Relationship between Open Data and Accountability”.

Note 2: And some evidence seems to confirm that hypothesis. For instance, in a field experiment in Kenya, villagers only responded to information about local spending in development projects when that information was coupled with specific guidance on how to participate in local decision-making processes).