Techniques and Technologies for Mobilizing Citizens: Do They Work?

17146073016_5657607917_z1

Techniques for mobilizing citizens to vote in elections have become highly sophisticated in large part thanks to get-out-the-vote (GOTV) research, with fascinating experimental evidence from interventions to increase turnout. Until recently, the use of these techniques has been mostly limited to electoral processes, often resorting to resource intensive tactics such as door-to-door canvassing and telemarketing campaigns. But as digital technologies such as email and SMS lower the costs for targeting and contacting individuals, the adaptation of these practices to participatory processes is becoming increasingly common. This leads to the question: how effective are GOTV-type efforts when using technology outside of the electoral realm?

One of the first (if not the first) efforts to bring together technology and GOTV techniques to non-electoral processes took place in the participatory budgeting (PB) process of the municipality of Ipatinga in Brazil in 2005. An automated system of targeted phone calls to landlines was deployed, with a recorded voice message from the mayor informing residents of the time and the location of the next PB meeting closest to them. Fast forward over a decade later, and New Yorkers can receive personalized text messages on their phones indicating the nearest PB polling location. Rather than a mere coincidence, the New York case illustrates a growing trend in participatory initiatives that – consciously or not – combine technology with traditional GOTV techniques to mobilize participation.

However, and unlike GOTV in elections, little is known about the effects of these efforts in participatory processes, the reasons for which I briefly speculated about in a previous post. We have just published a study in the British Journal of Political Science that, we hope, starts to reduce this gap between practice and knowledge. Entitled “A Get-Out-the-Vote Experiment on the World’s Largest Participatory Budgeting Vote in Brazil”, the study is co-authored by Jonathan Mellon, Fredrik M. Sjoberg and myself. The experiment was conducted in close collaboration with Rio Grande do Sul’s State Government (Brazil), which holds the world’s largest participatory budgeting process.

In the experiment, over 43,000 citizens were randomly assigned to receive email and text messages encouraging them to take part in the PB voting process. We used voting records to assess the impact of these messages on turnout and support for public investments. The turnout effect we document in the study is substantially larger than what has been found in most previous GOTV studies, and particularly those focusing on the effect of technologies like email and SMS. The increase in participation, however, did not alter which projects were selected through the PB vote: voters in the control and treatment groups shared the same preferences. In the study, we also assessed whether different message framing (e.g. intrinsic versus extrinsic) mattered. Not that much, we found, and a lottery incentive treatment had the opposite effect to what many might expect. Overall, our experiment suggests that tech-enabled GOTV approaches in participatory processes are rather promising if increasing levels of participation is one of the goals. But the “more research is needed” disclaimer, as usual, applies.

You can find the final study (gated version) here, and the pre-published (open) version here.

 

Catching up (again!) on DemocracySpot

cover-bookIt’s been a while since the last post here. In compensation, it’s not been a bad year in terms of getting some research out there. First, we finally managed to publish “Civic Tech in the Global South: Assessing Technology for the Public Good.” With a foreword by Beth Noveck, the book is edited by Micah Sifry and myself, with contributions by Evangelia Berdou, Martin Belcher, Jonathan Fox, Matt Haikin, Claudia Lopes, Jonathan Mellon and Fredrik Sjoberg.

The book is comprised of one study and three field evaluations of civic tech initiatives in developing countries. The study reviews evidence on the use of twenty-three information and communication technology (ICT) platforms designed to amplify citizen voices to improve service delivery. Focusing on empirical studies of initiatives in the global south, the authors highlight both citizen uptake (yelp) and the degree to which public service providers respond to expressions of citizen voice (teeth). The first evaluation looks at U-Report in Uganda, a mobile platform that runs weekly large-scale polls with young Ugandans on a number of issues, ranging from access to education to early childhood development. The following evaluation takes a closer look at MajiVoice, an initiative that allows Kenyan citizens to report, through multiple channels, complaints with regard to water services. The third evaluation examines the case of Rio Grande do Sul’s participatory budgeting – the world’s largest participatory budgeting system – which allows citizens to participate either online or offline in defining the state’s yearly spending priorities. While the comparative study has a clear focus on the dimension of government responsiveness, the evaluations examine civic technology initiatives using five distinct dimensions, or lenses. The choice of these lenses is the result of an effort bringing together researchers and practitioners to develop an evaluation framework suitable to civic technology initiatives.

The book was a joint publication by The World Bank and Personal Democracy Press. You can download the book for free here.

Women create fewer online petitions than men — but they’re more successful

clinton

Another recent publication was a collaboration between Hollie R. Gilman, Jonathan Mellon, Fredrik Sjoberg and myself. By examining a dataset covering Change.org online petitions from 132 countries, we assess whether online petitions may help close the gap in participation and representation between women and men. Tony Saich, director of Harvard’s Ash Center for Democratic Innovation (publisher of the study), puts our research into context nicely:

The growing access to digital technologies has been considered by democratic scholars and practitioners as a unique opportunity to promote participatory governance. Yet, if the last two decades is the period in which connectivity has increased exponentially, it is also the moment in recent history that democratic growth has stalled and civic spaces have shrunk. While the full potential of “civic technologies” remains largely unfulfilled, understanding the extent to which they may further democratic goals is more pressing than ever. This is precisely the task undertaken in this original and methodologically innovative research. The authors examine online petitions which, albeit understudied, are one of the fastest growing types of political participation across the globe. Drawing from an impressive dataset of 3.9 million signers of online petitions from 132 countries, the authors assess the extent to which online participation replicates or changes the gaps commonly found in offline participation, not only with regards to who participates (and how), but also with regards to which petitions are more likely to be successful. The findings, at times counter-intuitive, provide several insights for democracy scholars and practitioners alike. The authors hope this research will contribute to the larger conversation on the need of citizen participation beyond electoral cycles, and the role that technology can play in addressing both new and persisting challenges to democratic inclusiveness.

But what do we find? Among other interesting things, we find that while women create fewer online petitions than men, they’re more successful at it! This article in the Washington Post summarizes some of our findings, and you can download the full study here.

Other studies that were recently published include:

The Effect of Bureaucratic Responsiveness on Citizen Participation (Public Administration Review)

Abstract:

What effect does bureaucratic responsiveness have on citizen participation? Since the 1940s, attitudinal measures of perceived efficacy have been used to explain participation. The authors develop a “calculus of participation” that incorporates objective efficacy—the extent to which an individual’s participation actually has an impact—and test the model against behavioral data from the online application Fix My Street (n = 399,364). A successful first experience using Fix My Street is associated with a 57 percent increase in the probability of an individual submitting a second report, and the experience of bureaucratic responsiveness to the first report submitted has predictive power over all future report submissions. The findings highlight the importance of responsiveness for fostering an active citizenry while demonstrating the value of incidentally collected data to examine participatory behavior at the individual level.

Does online voting change the outcome? Evidence from a multi-mode public policy referendum (Electoral Studies)

Abstract:

Do online and offline voters differ in terms of policy preferences? The growth of Internet voting in recent years has opened up new channels of participation. Whether or not political outcomes change as a consequence of new modes of voting is an open question. Here we analyze all the votes cast both offline (n = 5.7 million) and online (n = 1.3 million) and compare the actual vote choices in a public policy referendum, the world’s largest participatory budgeting process, in Rio Grande do Sul in June 2014. In addition to examining aggregate outcomes, we also conducted two surveys to better understand the demographic profiles of who chooses to vote online and offline. We find that policy preferences of online and offline voters are no different, even though our data suggest important demographic differences between offline and online voters.

We still plan to publish a few more studies this year, one looking at digitally-enabled get-out-the-vote (GOTV) efforts, and two others examining the effects of participatory governance on citizens’ willingness to pay taxes (including a fun experiment in 50 countries across all continents).

In the meantime, if you are interested in a quick summary of some of our recent research findings, this 30 minutes video of my keynote at the last TicTEC Conference in Florence should be helpful.

 

 

New Papers Published: FixMyStreet and the World’s Largest Participatory Budgeting

2016_7_5_anderson-lopes_consulta-popular_virtual

Voting in Rio Grande do Sul’s Participatory Budgeting  (picture by Anderson Lopes)

Here are two new published papers that my colleagues Jon Mellon, Fredrik Sjoberg and myself have been working on.

The first, The Effect of Bureaucratic Responsiveness on Citizen Participation, published in Public Administration Review, is – to our knowledge – the first study to quantitatively assess at the individual level the often-assumed effect of government responsiveness on citizen engagement. It also describes an example of how the data provided through digital platforms may be leveraged to better understand participatory behavior. This is the fruit of a research collaboration with MySociety, to whom we are extremely thankful.

Below is the abstract:

What effect does bureaucratic responsiveness have on citizen participation? Since the 1940s, attitudinal measures of perceived efficacy have been used to explain participation. The authors develop a “calculus of participation” that incorporates objective efficacy—the extent to which an individual’s participation actually has an impact—and test the model against behavioral data from the online application Fix My Street (n = 399,364). A successful first experience using Fix My Street is associated with a 57 percent increase in the probability of an individual submitting a second report, and the experience of bureaucratic responsiveness to the first report submitted has predictive power over all future report submissions. The findings highlight the importance of responsiveness for fostering an active citizenry while demonstrating the value of incidentally collected data to examine participatory behavior at the individual level.

An earlier, ungated version of the paper can be found here.

The second paper, Does Online Voting Change the Outcome? Evidence from a Multi-mode Public Policy Referendum, has just been published in Electoral Studies. In an earlier JITP paper (ungated here) looking at Rio Grande do Sul State’s Participatory Budgeting – the world’s largest – we show that, when compared to offline voting, online voting tends to attract participants who are younger, male, of higher income and educational attainment, and more frequent social media users. Yet, one question remained: does the inclusion of new participants in the process with a different profile change the outcomes of the process (i.e. which projects are selected)? Below is the abstract of the paper.

Do online and offline voters differ in terms of policy preferences? The growth of Internet voting in recent years has opened up new channels of participation. Whether or not political outcomes change as a consequence of new modes of voting is an open question. Here we analyze all the votes cast both offline (n = 5.7 million) and online (n = 1.3 million) and compare the actual vote choices in a public policy referendum, the world’s largest participatory budgeting process, in Rio Grande do Sul in June 2014. In addition to examining aggregate outcomes, we also conducted two surveys to better understand the demographic profiles of who chooses to vote online and offline. We find that policy preferences of online and offline voters are no different, even though our data suggest important demographic differences between offline and online voters.

The extent to which these findings are transferable to other PB processes that combine online and offline voting remains an empirical question. In the meantime, nonetheless, these findings suggest a more nuanced view of the potential effects of digital channels as a supplementary means of engagement in participatory processes. I hope to share an ungated version of the paper in the coming days.

Ask citizens where public money should go: the results might surprise you

picture by MyTudut on flickr

(NB: article originally published in Capital Finance International)

As citizen engagement gains traction in the development agenda, identifying the extent to which it produces tangible results is essential. Participatory budgeting, a process in which citizens decide upon and monitor budget allocation, offers promising results, including increased local government revenues and reduced infant mortality.

Promoting citizen engagement in the development community: a quest for evidence

In recent years there has been a growing interest in citizen engagement as a means to promote better development outcomes. The Open Government Partnership (OGP), for instance, is a multilateral platform where governments from 66 countries commit, among other things, to promote governments that are more open, participatory and accountable to their citizens. Similarly, Making all Voices Count is an international initiative supported by private donors and development agencies that provides funding to projects that promote “citizen engagement and open, responsive government.”

The rationale behind this renewed enthusiasm for civic engagement is seemingly simple: citizens know best what their needs are and how to address them. Or, as spelled out in the OGP declaration, public engagement “increases the effectiveness of governments, which benefit from people’s knowledge, ideas and ability to provide oversight.” Yet, the evidence on the benefits of citizen engagement often seems fuzzy, scattered and – sometimes – contradictory. However, a clearer picture emerges when we examine some particular practices that fall under the general “citizen engagement” umbrella, of which participatory budgeting is one. Originating from the Brazilian city of Porto Alegre in 1989, participatory budgeting (PB) can be broadly defined as the participation of citizens in the decision-making process of budget allocation and in the monitoring of public spending. Experts estimate that up to 2,500 local governments around the world have implemented PB, from major cities such as New York, Paris, Seville, and Lima, to small and medium cities in countries as diverse as Poland, South-Korea, India, Bangladesh, and the Democratic Republic of Congo.

Over the years, PB has attracted significant attention from scholars and development professionals. As it reaches over a quarter-century of existence, it is generating a substantial amount of evidence of the benefits of involving citizens in budgeting decisions. Here, we briefly examine some of this evidence.

Some argue – and there is growing evidence – that citizen participation increases government tax revenues

At the beginning of the 2000s, researchers studying participatory budgeting began to see an unexpected result, with some municipalities reporting substantive increases in their tax revenues. In 2004, for instance, a comparative study [PDF] of 25 municipalities in Latin America and Europe found a significant reduction in levels of tax delinquency after the adoption of participatory budgeting. But, in reality, how surprising were these findings?

Mostly unknown even among seasoned public engagement advocates, a growing body of evidence in the field of “tax morale” suggests a relationship between citizen participation and tax compliance. The argument, in an oversimplified manner, is as follows: citizens are more willing to pay taxes when they perceive that their preferences are properly taken into account by public institutions. This argument finds ever-growing empirical support. For instance, a number of studies in Switzerland – notably those by the economists Bruno Frey and Benno Torgler – show that Swiss cantons with higher levels of democratic participation present lower tax evasion rates, even when controlling for other factors. Suggesting that this is not simply a Swiss exception, a cross-national study by Friedrich Schneider and Désirée Teobaldelli found that “the effect of direct democratic institutions on the shadow economy is negative and quantitatively important.” These observational findings are increasingly supported by a growing number of controlled experiments across a variety of cultural settings. At odds with conventional economic reasoning, some evidence in the field of “tax morale” suggests that participation may be even more effective at curbing tax evasion than traditional and commonly adopted deterrence measures, such as fines and controls.

In the specific case of participatory budgeting, more robust data is also emerging. For example, a recent working paper by the Inter-American Development Bank presents similar effects of participatory budgeting on revenues in a randomized controlled trial in Russia. As noted by the authors, Diether Beuermann and Maria Amelina, these results are by no means negligible:

Implementing the planning cycle of participatory budgeting increased local revenues per capita by US$30.22 in regions without previous decentralized experience and by US$37.34 in regions with previous decentralized experience […] These are sizeable effects as they represent differences of around 70 percent with respect to the control group mean.

So participatory budgeting is good for tax revenues, but how good is it for citizens themselves?

Participatory budgeting promotes pro-poor spending, better access to services and may even reduce infant mortality

The available evidence suggests that participatory budgeting leads to significant shifts in priorities and policies, towards expenditures that directly benefit the poor. A 2008 World Bank report demonstrated that participatory budgeting has a statistically significant impact on a number of social indicators. Among others, the report highlights that PB is positively and strongly associated with improvements in poverty rates and access to water services.

Despite producing evidence of its effectiveness on a number of fronts over the years, only 25 years after its initial implementation in Brazil do we start to see systematic evidence of sound development outcomes. This is mainly due to two recently released, major studies of participatory budgeting in Brazil. The first, published by Sonia Gonçalves in World Development, finds that municipalities that adopted participatory budgeting in Brazil “favoured an allocation of public expenditures that closely matched the popular preferences and channeled a larger fraction of their total budget to key investments in sanitation and health services.” As a consequence, the author also finds that this change in the allocation of public expenditures “is associated with a pronounced reduction in the infant mortality rates for municipalities which adopted participatory budgeting.” Barely a year later, a study by Michael Touchton and Brian Wampler in Comparative Political Studies generated similar findings, demonstrating that the adoption of participatory budgeting in Brazil is strongly associated with increases in health care spending and decreases in infant mortality rates.

These studies also highlight another important takeaway for those working with development and public sector reform: the need to consider the fact that participatory institutions may take time to produce noticeable effects. As shown by Touchton and Wampler, for instance, the effects of PB adoption become significantly more visible after the fourth year of implementation.

As citizen engagement draws increasing interest in the development agenda, staying focused on which types of processes work and which do not will become particularly relevant. Participatory budgeting offers some promising evidence for policy reformers who want to see tangible impact on the ground, but it might take more than enthusiasm to get there. Determination, and a certain amount of patience, remain essential ingredients when it comes to delivering results.

Three New Papers (and a presentation) on Civic Tech

CaptureFMS

This blog has been slow lately, but as I mentioned before, it is for a good cause. With some great colleagues I’ve been working on a series of papers (and a book) on civic technology. The first three of these papers are out. There is much more to come, but in the meantime, you can find below the abstracts and link to each of the papers. I also add the link to a presentation which highlights some other issues that we are looking at.

  • Effects of the Internet on Participation: Study of a Public Policy Referendum in Brazil.

Does online voting mobilize citizens who otherwise would not participate? During the annual participatory budgeting vote in the southern state of Rio Grande do Sul in Brazil – the world’s largest – Internet voters were asked whether they would have participated had there not been an online voting option (i-voting). The study documents an 8.2 percent increase in total turnout with the introduction of i-voting. In support of the mobilization hypothesis, unique survey data show that i-voting is mainly used by new participants rather than just for convenience by those who were already mobilized. The study also finds that age, gender, income, education, and social media usage are significant predictors of being online-only voters. Technology appears more likely to engage people who are younger, male, of higher income and educational attainment, and more frequent social media users.

Read more here.

  • The Effect of Government Responsiveness on Future Political Participation.

What effect does government responsiveness have on political participation? Since the 1940s political scientists have used attitudinal measures of perceived efficacy to explain participation. More recent work has focused on underlying genetic factors that condition citizen engagement. We develop a ‘Calculus of Participation’ that incorporates objective efficacy – the extent to which an individual’s participation actually has an impact – and test the model against behavioral data from FixMyStreet.com (n=399,364). We find that a successful first experience using FixMyStreet.com (e.g. reporting a pothole and having it fixed) is associated with a 54 percent increase in the probability of an individual submitting a second report. We also show that the experience of government responsiveness to the first report submitted has predictive power over all future report submissions. The findings highlight the importance of government responsiveness for fostering an active citizenry, while demonstrating the value of incidentally collected data to examine participatory behavior at the individual level.

Read more here.

  • Do Mobile Phone Surveys Work in Poor Countries? 

In this project, we analyzed whether mobile phone-based surveys are a feasible and cost-effective approach for gathering statistically representative information in four low-income countries (Afghanistan, Ethiopia, Mozambique, and Zimbabwe). Specifically, we focused on three primary research questions. First, can the mobile phone survey platform reach a nationally representative sample? Second, to what extent does linguistic fractionalization affect the ability to produce a representative sample? Third, how effectively does monetary compensation impact survey completion patterns? We find that samples from countries with higher mobile penetration rates more closely resembled the actual population. After weighting on demographic variables, sample imprecision was a challenge in the two lower feasibility countries (Ethiopia and Mozambique) with a sampling error of /- 5 to 7 percent, while Zimbabwe’s estimates were more precise (sampling error of /- 2.8 percent). Surveys performed reasonably well in reaching poor demographics, especially in Afghanistan and Zimbabwe. Rural women were consistently under-represented in the country samples, especially in Afghanistan and Ethiopia. Countries’ linguistic fractionalization may influence the ability to obtain nationally representative samples, although a material effect was difficult to discern through penetration rates and market composition. Although the experimentation design of the incentive compensation plan was compromised in Ethiopia and Zimbabwe, it seems that offering compensation for survey completion mitigated attrition rates in several of the pilot countries while not reducing overall costs. These effects varied across countries and cultural settings.

Read more here.

  • The haves and the have nots: is civic tech impacting the people who need it most? (presentation) 

Read more here.

Unusual suspects? Effects of technology on citizen engagement

(Originally posted on the World Bank’s Let’s Talk Development blog)

What is the effect of technology on citizen engagement? On the one hand, enthusiasts praise the prospects offered by technology: from real-time beneficiary feedback to collaborative policymaking, the possibilities for listening at scale seem endless. Skeptics, on the other, fear that unequal access to technologies will do nothing but favor the “usual suspects”, empowering the already empowered and reinforcing existing inequalities. While the debate sometimes gets heated, a common feature unites both sides: there is limited evidence to support both views.

Providing evidence to better inform practice at the intersection of technology and citizen engagement is one of the core goals of the Bank’s Digital Engagement Evaluation Team (DEET). And, to contribute empirical data to the debate on the effects of technology on participatory processes, the team has been carrying out a number of studies, some of them covering as many as 132 countries.

The results of one of these studies have just been published, looking at the effects of Internet voting on the world’s largest participatory budgeting exercise, in the state of Rio Grande do Sul, Brazil. Every year, over one million people participate in the state-wide process, where citizens can vote either online or offline for projects that are to be included in the public budget. In this study we present the results of a unique survey of over 22,000 Internet voters, focusing on three key research questions:

1) Does an opportunity to vote online increase participation?
2) If so, what is the socioeconomic profile of new voters?
3) And finally, what is the level of pre-existing engagement of these online voters?

Anticipating some of our results here, nearly two-thirds of respondents answer the first question affirmatively, saying they would not have taken part in the vote if online voting (i-voting) was not available. This evidence supports the view that technology increases participation among individuals who would not have voted otherwise. In parallel to this, our study shows that introducing i-voting does not lead to a substitution effect, meaning that for the most part, those who vote offline will continue to do so, despite the introduction of i-voting.

On the second question, a picture of the “usual suspects” of online engagement emerges: all else equal, i-voting seems more likely to engage individuals who are younger, male, of higher income and educational attainment, and more frequent social media users. However, from a civic engagement perspective i-voting seems to engage rather unusual suspects, boosting inclusiveness and engaging individuals who were previously uninspired by traditional politics and community activities.

In short, i-voting increases participation among previously non-engaged strata of the population, promoting the inclusiveness of the process as a whole. However, these new participants – the online-only voters – are likely to be socio-economically more privileged: a compelling reason for combining multiple avenues (online and offline) for participation.

In the study we analyze these findings in light of the literature on convenience voting, participatory governance and collective intelligence. We conclude with the implications of the findings for future practice and research.

You can download the paper here

New Evidence that Citizen Engagement Increases Tax Revenues

pic by Tax Credits on flickr

Quite a while ago, drawing mainly from the literature on tax morale, I posted about the evidence on the relationship between citizen engagement and tax revenues, in which participatory processes lead to increased tax compliance (as a side note, I’m still surprised how those working with citizen engagement are unaware of this evidence).

Until very recently this evidence was based on observational studies, both qualitative and quantitative. Now we have – to my knowledge – the first experimental evidence that links citizen participation and tax compliance. A new working paper published by Diether Beuermann and Maria Amelina present the results of a randomized experiment in Russia, described in the abstract below:

This paper provides the first experimental evaluation of the participatory budgeting model showing that it increased public participation in the process of public decision making, increased local tax revenues collection, channeled larger fractions of public budgets to services stated as top priorities by citizens, and increased satisfaction levels with public services. These effects, however, were found only when the model was implemented in already-mature administratively and politically decentralized local governments. The findings highlight the importance of initial conditions with respect to the decentralization context for the success of participatory governance.

In my opinion, this paper is important for a number of reasons, some of which are worth highlighting here. First, it adds substantive support to the evidence on the positive relationship between citizen engagement and tax revenues. Second, in contrast to studies suggesting that participatory innovations are most likely to work when they are “organic”, or “bottom-up”, this paper shows how external actors can induce the implementation of successful participatory experiences. Third, I could not help but notice that two commonplace explanations for the success of citizen engagement initiatives, “strong civil society” and “political will”, do not feature in the study as prominent success factors.  Last, but not least, the paper draws attention to how institutional settings matter (i.e. decentralization). Here, the jack-of-all-trades (yet not very useful) “context matters”, could easily be replaced by “institutions matter”.

You can read the full paper here [PDF].