Social Accountability: What Does the Evidence Really Say?

So what does the evidence about citizen engagement say? Particularly in the development world it is common to say that the evidence is “mixed”. It is the type of answer that, even if correct in extremely general terms, does not really help those who are actually designing and implementing citizen engagement reforms.

This is why a new (GPSA-funded) work by Jonathan Fox, “Social Accountability: What does the Evidence Really Say” is a welcome contribution for those working with open government in general and citizen engagement in particular. Rather than a paper, this work is intended as a presentation that summarizes (and disentangles) some of the issues related to citizen engagement.

Before briefly discussing it, some definitional clarification. I am equating “social accountability” with the idea of citizen engagement given Jonathan’s very definition of  social accountability:

“Social accountability strategies try to improve public sector performance by bolstering both citizen engagement and government responsiveness”

In short, according to this definition, social accountability is defined, broadly, as “citizen participation” followed by government responsiveness, which encompasses practices as distinct as FOI law campaigns, participatory budgeting and referenda.

But what is new about Jonathan’s work? A lot, but here are three points that I find particularly important, based on a very personal interpretation of his work.

First, Jonathan makes an important distinction between what he defines as “tactical” and “strategic” social accountability interventions. The first type of interventions, which could also be called “naïve” interventions, are for instance those bounded in their approach (one tool-based) and those that assume that mere access to information (or data) is enough. Conversely, strategic approaches aim to deploy multiple tools and articulate society-side efforts with governmental reforms that promote responsiveness.

This distinction is important because, when examining the impact evaluation evidence, one finds that while the evidence is indeed mixed for tactical approaches, it is much more promising for strategic approaches. A blunt lesson to take from this is that when looking at the evidence, one should avoid comparing lousy initiatives with more substantive reform processes. Otherwise, it is no wonder that “the evidence is mixed.”

Second, this work makes an important re-reading of some of the literature that has found “mixed effects”, reminding us that when it comes to citizen engagement, the devil is in the details. For instance, in a number of studies that seem to say that participation does not work, when you look closer you will not be surprised that they do not work. And many times the problem is precisely the fact that there is no participation whatsoever. False negatives, as eloquently put by Jonathan.

Third, Jonathan highlights the need to bring together the “demand” (society) and “supply” (government) sides of governance. Many accountability interventions seem to assume that it is enough to work on one side or the other, and that an invisible hand will bring them together. Unfortunately, when it comes to social accountability it seems that some degree of “interventionism” is necessary in order to bridge that gap.

Of course, there is much more in Jonathan’s work than that, and it is a must read for those interested in the subject. You can download it here [PDF].

When Citizen Engagement Saves Lives (and what we can learn from it)

When it comes to the relationship between participatory institutions and development outcomes, participatory budgeting stands out as one of the best examples out there. For instance, in a paper recently published in World Development,  Sonia Gonçalves finds that municipalities that adopted participatory budgeting in Brazil “favoured an allocation of public expenditures that closely matched the popular preferences and channeled a larger fraction of their total budget to key investments in sanitation and health services.”  As a consequence, the author also finds that this change in the allocation of public expenditures “is associated with a pronounced reduction in the infant mortality rates for municipalities which adopted participatory budgeting.”

Evolution of Expenditure Share in Health and Sanitation compared between adopters and non-adopters of PB (Goncalves 2013).

Evolution of  the share of expenditures in health and sanitation compared between adopters and non-adopters of participatory budgeting (Goncalves 2013).

Now, in an excellent new article published in Comparative Political Studies, the authors Michael Touchton and Brian Wampler come up with similar findings (abstract):

We evaluate the role of a new type of democratic institution, participatory budgeting (PB), for improving citizens’ well-being. Participatory institutions are said to enhance governance, citizens’ empowerment, and the quality of democracy, creating a virtuous cycle to improve the poor’s well-being. Drawing from an original database of Brazil’s largest cities over the last 20 years, we assess whether adopting PB programs influences several indicators of well-being inputs, processes, and outcomes. We find PB programs are strongly associated with increases in health care spending, increases in civil society organizations, and decreases in infant mortality rates. This connection strengthens dramatically as PB programs remain in place over longer time frames. Furthermore, PB’s connection to well-being strengthens in the hand of mayors from the nationally powerful, ideologically and electorally motivated Workers’ Party. Our argument directly addresses debates on democracy and well-being and has powerful implications for participation, governance, and economic development.

When put together, these findings provide compelling evidence for those who – often unfamiliar with the literature – question the effectiveness of participatory governance institutions. Surely, more research is needed, and different citizen engagement initiatives (and contexts) may lead to different results.

But these articles also bring another important takeaway for those working with development and public sector reform. And that is the need to consider the fact that participatory institutions (as most institutional reforms) may take time to produce desirable/noticeable effects. As noted by Touchton and Wampler:

 The relationships we describe between PB and health and sanitation spending, PB and CSOs, and PB and health care outcomes in this section are greater in magnitude and stronger in statistical significance for municipalities that have used PB for a longer period of time. Municipalities using PB for less than 4 years do exhibit lower infant mortality rates than municipalities that never adopted PB. However, there is no statistically significant difference in spending on health care and sanitation between municipalities using PB for less than 4 years and municipalities that never adopted the program. This demonstrates the benefits from adopting PB are not related to low-hanging fruit, but built over a great number of years. Our results imply PB is associated with long-term institutional and political change—not just short-term shifts in funding priorities .

If throughout the years participatory budgeting has produced  evidence of its effectiveness on a number of fronts (e.g. pro-poor spending), it is only 25 years after its first implementation in Brazil that we start to see systematic evidence of sound development outcomes such as reduction in infant mortality. In other words, rushing to draw conclusions at early stages of participatory governance interventions may result in misleading assessments. Even worse, it may lead to discontinuing efforts that are yet to bear fruit in the medium and longer terms.

10 Most Read Posts in 2013

Below is a selection of the 10 most read posts at DemocracySpot in 2013. Thanks to all of those who stopped by throughout the year, and happy 2014.

1. Does transparency lead to trust? Some evidence on the subject.

2. The Foundations of Motivation for Citizen Engagement

3. Open Government, Feedback Loops, and Semantic Extravaganza

4. Open Government and Democracy

5. What’s Wrong with e-Petitions and How to Fix them

6. Lawrence Lessig on Sortition and Citizen Participation

7. Unequal Participation: Open Government’s Unresolved Dilemma

8. The Effect of SMS on Participation: Evidence from Uganda

9. The Uncertain Relationship Between Open Data and Accountability

10. Lisbon Revisited: Notes on Participation

Open Budgets in Africa: Tokenistic?

Matt Andrews recently posted an interesting analysis in his blog. Measuring the difference in transparency between budget formulation and budget execution, Matt finds that “Most countries have a gap between the scores they get in transparency of budget preparation and transparency of budget execution. Indeed, 63% of the countries have more transparency in budget formulation than in budget execution.” And he concludes that “countries with higher OBI scores tend to have relatively bigger gaps than the others—so that I am led to believe that countries generally focus on improving transparency in formulation to get better scores (with efforts to make execution getting less attention).” He has also written a second post about it and the IBP folks have replied to him here.


Also read

Open Government and Democracy 

The Uncertain Relationship Between Open Data and Accountability

The Participatory Turn: Participatory Budgeting Comes to America


So here it is, finally, the much awaited PhD by Hollie Russon-Gilman (Ash Center – Harvard) on Participatory Budgeting in the United States.

Below is the abstract.

Participatory Budgeting (PB) has expanded to over 1,500 municipalities worldwide since
its inception in Porto Alege, Brazil in 1989 by the leftist Partido dos Trabalhadores
(Workers’ Party). While PB has been adopted throughout the world, it has yet to take
hold in the United States. This dissertation examines the introduction of PB to the United
States with the first project in Chicago in 2009, and proceeds with an in-depth case study
of the largest implementation of PB in the United States: Participatory Budgeting in New
York City. I assess the outputs of PB in the United States including deliberations,
governance, and participation.
I argue that PB produces better outcomes than the status quo budget process in New York
City, while also transforming how those who participate understand themselves as
citizens, constituents, Council members, civil society leaders and community
stakeholders. However, there are serious challenges to participation, including high costs
of engagement, process exhaustion, and perils of scalability. I devise a framework for
assessment called “citizenly politics,” focusing on: 1) designing participation 2)
deliberation 3) participation and 4) potential for institutionalization. I argue that while the
material results PB produces are relatively modest, including more innovative projects,
PB delivers more substantial non-material or existential results. Existential citizenly
rewards include: greater civic knowledge, strengthened relationships with elected
officials, and greater community inclusion. Overall, PB provides a viable and
informative democratic innovation for strengthening civic engagement within the United
States that can be streamlined and adopted to scale.

You can read the full dissertation here [PDF].

Like it?  You might also want to read this about who participates in NYC’s PB and this about the effects of PB on infant mortality in Brazil.

Taxation and Accountability: Experimental Evidence for Taxation’s Effect on Citizen Behavior

A paper by Lucy Martin (Yale)

In sub-Saharan Africa, low taxes co-exist with even lower government accountability, seen in high levels of corruption and low public goods provision. While there are existing theories of why taxation might be linked to better governance, many of the microfoundations of this effect remain unclear. I argue that taxation impacts governance by altering the expressive benefit citizens receive from sanctioning corrupt officials, making those who pay taxes more likely to hold leaders accountable. I provide new cross-national evidence that taxation and corruption are linked; I then formalize the theory and test the proposed mechanism using a set of laboratory-in-the-field experiments in Uganda. I find evidence that taxation activates a stronger fairness norm, leading citizens to demand more from leaders. This effect is strongest among adult, wage-earning men – exactly the group who has the most experience, historically, paying taxes in Uganda. I then propose additional tests, to be carried out in 2013, to strengthen and expand my findings.

And a tip for development professionals from the conclusion:

(…) aid professionals should seriously consider the role of formal taxation, as well as more informal community contributions, when designing development interventions. Adding some sort of community contribution to external aid programs could encourage give aid beneciaries more ownership over projects and, this paper suggests, make them more likely to hold local leaders accountable for how development funds are spent.

Read the full paper here [PDF].

Does transparency lead to trust? Some evidence on the subject.

As open government gains traction in the international agenda, it is increasingly common to come across statements that assume a causal relationship in which transparency leads to trust in government. But to what extent are claims that transparency leads to trust backed up by evidence?

Judging from some recent publications on the subject, such a relationship is not as straightforward as sadvocates would like. In fact, in a number of cases, the evidence points in another direction: that is, transparency may ultimately decrease trust.

Below is a brief overview of research that has been carried out on the subject:

Transparency has been trumpeted by many as the key to trust in government. The assumption is that if government organisations open up and show the public what decisions are made, how they are made and what the results are, people will automatically have more trust in government. But does transparency really lead to more trust? Or will it only provide critical citizens with more information to blame government again and again for small mistakes? Transparency and Trustexamines the effects of transparency on trust in a government organisation. By using an experimental method this study moves beyond normative or correlational research on transparency. In doing so, causal inferences regarding the relation between transparency and trust are allowed. Several objects of transparency and dimensions of information are being put to the test in three experiments. The experiments show that transparency is merely a ‘hygiene factor’: it does not contribute to higher levels of trust and it can even lead to lower levels of trust if people are disappointed with the degree to which government is transparent. This conclusion challenges current overly optimistic assumptions concerning the effect of transparency on trust.

Building on the notion of transparency as a strong democratic value and theories of procedural justice, this article reports an explorative experimental test whether transparency in decision making may lead to increased perceived legitimacy in terms of decision acceptance and trust. This is done in a context of difficult decisions of high importance for citizens – namely priority setting in public health care. An experiment was designed in which ordinary citizens were presented with a description of a case of priority setting between two groups with different health care needs. One group was given no information at all on the decision-making procedure, as an example of non-transparent decision making, and six groups were presented with different descriptions of the decision-making procedure, as examples of transparency in decision making. The transparent procedures were derived from three basic forms of democratic decision making: representation, direct participation and expert decision making. A second manipulation framed the decision-making procedure alternatively in positive or negative terms in order to capture media framing effects as well. According to the findings of the study, transparent decision-making procedures tend to weaken rather than strengthen general trust in health care – a finding that might reveal obstacles to attempts to strengthen the legitimacy of health care by employing transparent procedures. The results also show that while the form of decision making had no significant impact on perceived legitimacy, positive or negative framing of a decision-making procedure influences public perceptions of both the procedure and the decision outcome.

Of course, the impact of transparency on trust may vary according to the context:

 Transparency is considered a key value for trustworthy governments. However, the effect of transparency on citizens’ trust across national cultures is overlooked in current research. This article compares the effect of transparency on trust in government in the Netherlands and South Korea. The effect is investigated in two similar series of three experiments. The authors hypothesize that the effect of transparency differs because the countries have different cultural values regarding power distance and short- and long-term orientation. Results reveal similar patterns in both countries: transparency has a subdued and sometimes negative effect on trust in government. However, the negative effect in South Korea is much stronger. The difference in the magnitude of transparency’s effect suggests that national cultural values play a significant role in how people perceive and appreciate government transparency.

But some evidence goes even further, suggesting that transparency may have a demobilizing effect on citizens. And, if context matters, such a demobilizing effect might be particularly strong in the context of developing countries:

International organizations, policy experts, and nongovernmental organizations promote greater governmental transparency as a crucial reform to enhance accountability and curb corruption. Transparency is predicted to deter corruption in part by expanding the possibilities for public or societal accountability, that is, for citizens and citizens associations to monitor, scrutinize, and act to hold public office holders to account. Although the societal accountability mechanism linking transparency and good government is often implied, it builds on a number of assumptions seldom examined empirically. This article unpacks the assumptions of principal-agent theories of accountability and suggests that the logic of collective action can be used to understand why exposure of egregious and endemic corruption may instead demobilize the demos (i.e., resignation) rather than enhance accountability (i.e., indignation). We explore these theoretical contentions and examine how transparency affects three indicators of indignations versus resignation—institutional trust, political involvement, and political interest—given different levels of corruption. The empirical analyses confirm that an increase in transparency in highly corrupt countries tends to breed resignation rather than indignation.

Democratic theory often assumes that offering more information to voters will enhance electoral accountability. However, it is unclear whether corruption information translates into higher political participation and increased support for challengers. For example, information on corruption could lower the utility one gets from participating in elections at all. We provide experimental evidence that such information not only decreases incumbent support in local elections in Mexico, but also decreases voter turnout and challengers’ votes, as well as erodes partisan attachments. Our results suggest that while information clearly is necessary to improve accountability, corruption information is not necessarily suficient, since voters may respond to it by withdrawing from the political process.

Surely, transparency remains an essential – although quite insufficient – ingredient of accountability. On the trust issue, one could easily think of a number of scenarios in which it is actually better that citizens do not trust their governments. In fact, systems of checks and balances and oversight institutions are not specifically conceived under the logic of trust. Quite on the contrary, such institutional designs assume some level of suspicion vis-à-vis governments: as put in the Federalist Paper No. 51, “If angels were to govern men, neither external nor internal controls on government would be necessary.”

Granted, in some cases a perfect world in which citizens trust their governments may well be desirable. It may even be that transparency leads – in the long run – to increased trust: a great way to sell transparency to governments. But if we want to walk the talk of evidence-based policymaking, we may consider dropping the trust rhetoric. At least for now.

How effective are whistleblower laws in combating corruption?

A new discussion paper by Rajeev Goel and Michael Nelson looks at the effectiveness of US whistleblower laws in combating corruption. The abstract is below:

Whistleblower laws are becoming important governance tools in both the public and private sectors. To examine the effectiveness of whistleblower laws and their awareness, this study creates a unique internet-based measure of awareness about whistleblower laws and provisions, focusing on the United States. Placing the analysis within the larger corruption literature, our results show that greater whistleblower awareness results in more observed corruption and this holds across specifications. Internet awareness of whistleblower laws appears to be more effective at exposing corruption than the quantity and quality of whistleblower laws themselves.

And a few excerpts from the conclusion, which highlights the role of the internet:

Couching the empirical analysis within the extant literature on the causes of corruption, our results show that greater internet awareness about whistleblower laws results in more corruption coming to light and being successfully prosecuted. In terms of magnitude, an increase in whistleblower hits by one sample standard deviation would increase average corruption convictions per million population by nearly thirty over a decade-long period. Interestingly, the internet awareness about corruption seems relatively more effective at exposing corruption than the quantity and quality of whistleblower laws themselves. Further, the direct government resources allocated to controlling crime and the indirect efforts via whistleblower awareness are found to be complementary. These findings are generally robust to alternate specifications, including an allowance for potential endogeneity of whistleblower awareness, and to broader measures of internet whistleblower awareness. (…)

The results for the United States in terms of the effectiveness of whistleblower laws in exposing corruption should be of interest to policy makers everywhere, especially in other nations that do not have adequate protections for whistleblowers. As internet diffusion grows and the digital divide narrows, it would be interesting to see a further impact of whistleblower awareness and, more generally, of the internet.

Read the full paper here [PDF].

Open Government and Democracy

The International Budget Partnership (IBP) has recently released the results of the Open Budget Survey 2012, which measures “the state of budget transparency, participation and oversight in 100 countries around the world”. In the survey report, the authors highlighted the positive relationship between budget transparency and democracy:

A democratic political system is a significant factor that supports budget transparency (…) In fact, a switch from autocracy to democracy is typically associated with an improvement in a country’s OBI score by almost 20 points, after controlling for other variables. In addition, transparency seems to depend much more on current levels of democracy than on how long a country has been a democracy: for countries in transition, this means that rapid improvements in transparency can be achieved without having to wait for slow processes of learning and adaptation.

This adds to a growing body of literature showing that democracies (and electoral competition) are indeed more transparent than other types of regime. If the relationship between democracy and openness comes across as obvious, it also opens space for some questions about the open government movement and its strategy to promote transparency.

Screen Shot 2013-03-07 at 00.42.12

Political Rights and Civil Liberties in 2013 – Freedom House

Transparency, it seems, is one of the vital signs of well-functioning democracies. Chronic lack of transparency, on the other hand, emerges as the symptom of flawed democracies or authoritarian regimes. If this logic is correct (and the evidence suggests it is) advocating for transparency would correspond to treating the symptoms of a disease, rather than preventing it in the first place.

This is not to say that promoting transparency reforms (e.g. open data, open budgets) is a useless act. Treating a symptom is not a problem in itself: it alleviates the pain and may even prevent further complications. But neglecting to treat the cause of the symptom is surely a bad practice.

This begs a fundamental question: are open government advocates efficiently channelling their energy and resources when asking for more transparency from governments that have little or no inclination to democracy? Or are they failing to strike a balance which combines a focus on transparency with more fundamental reforms that promote, for instance, free, fair and competitive elections?

Granted, transparency and democracy are mutually reinforcing: it is difficult to think of a democracy without informed consent. And even well-established democracies still have a long way to go towards more transparency. But, for instance, going as far as considering that open government may blossom in non-democracies seems questionable to me. All the technology and transparency in the world is unlikely to realize its full potential in the absence of fundamental political rights and civil liberties.

It might be time to start focusing on the role that political regimes play in promoting values that are dear to the open government movement, such as transparency, participation and collaboration. And democracy – or lack thereof – is the elephant in the room.


Further reading

Alt, J. E., Lassen, D. D., & Rose, S. (2005). “The causes of fiscal transparency: Evidence from the US States.” IMF Staff Papers, 53(Special Issue), 30–57.

Alt, J. E., & Lowry, R. C. (2010). “Transparency and accountability: Empirical results for US States.” Journal of Theoretical Politics, 22(4), 379–406.

Hollyer, J. R., Rosendorff, B. P., & Vreeland, J. R. (2011). “Democracy and transparency.” Journal of Politics, 73(4), 1191–1205.

Rosendorff, B. Peter, and James Raymond Vreeland. (2004). “Democracy and Data Dissemination: The Effect of Political Regime on Transparency.” Working Paper, Yale. 

Rosendorff, B. Peter and Doces, John A. (2006). “Democracy and Transparency”. Swiss Political Science Review, 12 (3), 99-112.

Wehner, J. and de Renzio, P. (2013) “Citizens, Legislators, and Executive Disclosure: The Political Determinants of Fiscal Transparency.” World Development, 41, 96-108.

Unequal Participation: Open Government’s Unresolved Dilemma

( cross-posted from techPresident)

In my frequent conversations about open government and citizen participation, the subject of elite capture (or “how representative it is”) is almost unavoidable. Some go as far as evaluating participatory initiatives on the grounds of an ideal notion of representativeness: participants should perfectly mirror the socio-demographic traits of the larger population from which they come.

But oddly enough, the same people who raise these concerns about participatory initiatives are much less inclined to apply the same reasoning and standards to traditional politics. In other words, few take the time to consider how representative and inclusive existing electoral democracy actually is. An article by Nicholas Carnes at the New York Times about political representation in the United States puts the issue into perspective:

If millionaires were a political party, that party would make up roughly 3 percent of American families, but it would have a super-majority in the Senate, a majority in the House, a majority on the Supreme Court and a man in the White House. If working-class Americans were a political party, that party would have made up more than half the country since the start of the 20th century. But legislators from that party (those who last worked in blue-collar jobs before entering politics) would never have held more than 2 percent of the seats in Congress.

I’ve yet to see a participatory process that produces similar results. But the limits of representation do not stop there. African Americans and Latinos are still greatly under-represented in US politics. The gender issue is no different: with the House of Representatives only 17 percent women, the Inter Parliamentary Union ranks the US 82nd in female representation in politics, behind countries such as the Arab Emirates, Sudan, Mauritania and Kazakhstan.

Obviously, the US is by no means exceptional in exclusion. Those working in the field of political participation have long been aware of the excluding effect of representative systems. As put by political scientist Arend Lijphart, unequal participation remains as representative democracy’s “unresolved dilemma.” Even more unfortunately, underlines Lijphart, inequalities in representation and influence “are not randomly distributed, but systematically biased in favor of more privileged citizens (…) and against less advantaged citizens”.

And it is from this unresolved dilemma that the raison d’être of participatory innovations stems. But rather than a replacement for representative systems (as misunderstood by some), participatory innovations are complementary mechanisms to enable the participation of individuals who are systematically excluded from traditional politics, ultimately increasing the overall diversity of voices that influence government.

This observation leads to a fundamental issue when assessing citizen participation initiatives: beyond questioning demographic representativeness, one must also consider the extent to which initiatives succeed (or not) in promoting the participation of previously marginalized sectors of society (i.e. inclusiveness).

So how inclusive are these mechanisms?

To continue with the US example, let’s consider one of the most exciting open government events taking place at the local level in the US: the recent adoption of participatory budgeting in NYC. Unlike most overhyped #opengov experiences, a team of researchers carried out an evaluation of the experience looking at, among other things, the extent to which it promoted inclusiveness. Below are some excerpts from the report [PDF]:

  • Twenty percent of PB voters identified themselves as African American; 14 percent as Hispanic or Latino/a; 2 percent as Asian and 2 percent as “Other.”
  • A higher percentage of African Americans participated in neighborhood assemblies (38 percent), compared to the full population in the four districts (31 percent).
  • Twenty-one percent of budget delegates and 19 percent of PB voters were born outside of the United States.
  • Participants that identified themselves as Black/African American were the most likely to volunteer to be budget delegates.
  • Women represented 64 percent of neighborhood assembly participants, 65 percent of budget delegates and 62 percent of voters in the PB process.

But how these numbers compare with participation in traditional politics is probably one of the highlights of the evaluation (emphasis is mine):

One of the most striking findings about who participated in PB [participatory budgeting] is how the data compares to other types of civic engagement, particularly voting patterns in NYC elections. Across the districts, PB engaged communities that have traditionally been uninspired by politics. People of color, low-income people and some immigrant groups turned out at higher rates than in previous elections.

A few numbers worth noting:

Latino/as represented 39 percent of voters in the 2009 City Council elections. However, 50 percent of PB voters identified themselves as Latino/a (District 8 NYC).

Black or African Americans represented 79 percent of voters in 2009 City Council elections. However, 87 percent of the district’s PB voters identified themselves as Black or African American (District 45 NYC).

Twenty-two percent of PB voters had a household income of less than $10,000 compared to 4 percent of the district’s voters in the 2009 City Council election (District 8 NYC).

I have very little doubt, if any, that the contrast would be even starker if we compared the income of those who sit on the City Council and those who participated in the NYC participatory budgeting. In City Councils across the US, less than 10 percent of members come from a blue-collar background. Conversely, the numbers on income of participatory budgeting participants speak for themselves.

As citizen engagement gains traction in the open government agenda, inclusiveness should be one of the top priorities: both from normative and empirical standpoints, more inclusive initiatives are likely to produce better outcomes. The NYC experience provides valuable lessons for donors, policymakers, advocates, and enthusiasts alike. They can find more about it here and here.