Bit by Bit: Social Research in the Digital Age


Pic by Jim Kaskade (flickr creative commons)

Matthew Salganik, Professor of Sociology at Princeton University, has recently put his forthcoming book on social research and big data online for an open review. Matthew is the author of many of my favorite academic works, including this experiment in which he and Duncan Watts test social influence by artificially inverting the popularity of songs in an online music market. He is also the brains behind All Our Ideas, an amazing tool that I have used in much of the work that I have been doing, including “The Governor Asks” in Brazil.

As in the words of Matthew, this is a book “for social scientists that want to do more data science, and it is for data scientists that want to do more social science.” Even though I have not read the entire book, one of the things that has already impressed me is the simplicity with which Matthew explains complex topics, such as human computation, distributed data collection and digital experiments. For each topic, he highlights opportunities and provides experienced advice for those working with big data and social sciences. His stance on social research in the digital age is brilliant and refreshing, and is a wake-up call for lots of people working in that domain. Below is an excerpt from his preface:

From data scientists, I’ve seen two common misunderstandings. The first is thinking that more data automatically solves problems. But, for social research that has not been my experience. In fact, for social research new types of data, as opposed to more of the same data, seems to be most helpful. The second misunderstanding that I’ve seen from data scientists is thinking that social science is just a bunch of fancy-talk wrapped around common sense. Of course, as a social scientist—more specifically as a sociologist—I don’t agree with that; I think that social science has a lot of to offer. Smart people have been working hard to understand human behavior for a long time, and it seems unwise to ignore the wisdom that has accumulated from this effort. My hope is that this book will offer you some of that wisdom in a way that is easy to understand.

From social scientists, I’ve also seen two common misunderstandings. First, I’ve seen some people write-off the entire idea of social research using the tools of the digital age based on a few bad papers. If you are reading this book, you have probably already read a bunch of papers that uses social media data in ways that are banal or wrong (or both). I have too. However, it would be a serious mistake to conclude from these examples that all digital age social research is bad. In fact, you’ve probably also read a bunch of papers that use survey data in ways that are banal or wrong, but you don’t write-off all research using surveys. That’s because you know that there is great research done with survey data, and in this book, I’m going to show you that there is also great research done with the tools of the digital age.

The second common misunderstanding that I’ve seen from social scientists is to confuse the present with the future. When assessing social research in the digital age—the research that I’m going to describe in this book—it is important to ask two distinction questions:

How well does this style of research work now?

How well will this style of research work in the future as the data landscape changes and as researchers devote more attention to these problems?

I have only gone through parts of the book (and yes, I did go beyond the preface). But from what I can see, it is a must read for those who are interested in digital technologies and the new frontiers of social research. And while reading it, why not respond to Matthew’s generous act by providing some comments? You can access the book here.


Could Corruption Be Good For Your Health? (or Side Effects of Anti-Corruption Efforts)

The literature on corruption is disputed territory, and one that is full of surprises. On one side, a number of scholars and development practitioners follow the traditional understanding, arguing that corruption is an evil to be eradicated at any cost. On the other, some scholars and practitioners see corruption as an “informal tax” that mediates access to goods and services in contexts of poor institutions and policies, commonly found in the early stages of development. In other words, corruption is a symptom, rather than a problem, with some even arguing that corruption may generate efficiencies in certain contexts: the so-called “greasing-the-wheel hypothesis.”

If these differences in perspective were not enough, a new paper adds more nuance to the debate and challenges conventional wisdom. Launched in 2003, the Brazilian anti-corruption program consists of a series of random audits by the federal government to assess whether municipalities effectively spend earmarked federal transfers according to pre-established guidelines. The results of the audits are then disseminated to the public, with auditors engaging with local councils and civil society to encourage them to monitor tax revenues. The program became famous in development and anti-corruption circles, in great part thanks to an earlier paper by Ferraz and Finan (2008) which found that “the release of the audit outcomes had a significant impact on incumbents’ electoral performance, and that these effects were more pronounced in municipalities where local radio was present to divulge the information.”

But if, when they know about it, citizens are more likely to vote corrupt politicians out of office, what is the effect of these audits on the quality of service delivery? This is the question that Guilherme Lichand, Marcos Lopes and Marcelo Medeiros (2016) try to answer in a new paper entitled “Is Corruption Good For Your Health?.”  Below is the abstract of the paper, (highlights are mine):

While corruption crackdowns have been shown to effectively reduce missing government expenditures, their effects on public service delivery have not been credibly documented. This matters because, if corruption generates incentives for bureaucrats to deliver those services, then deterring it might actually hurt downstream outcomes. This paper exploits variation from an anti-corruption program in Brazil, designed by the federal government to enforce guidelines on earmarked transfers to municipalities, to study this question. Combining random audits with a differences-in-differences strategy, we find that the anti-corruption program greatly reduced occurrences of over-invoicing and off-the-record payments, and of procurement manipulation within health transfers. However, health indicators, such as hospital beds and immunization coverage, became worse as a result. Evidence from audited amounts suggests that lower corruption came at a high cost: after the program, public spending fell by so much that corruption per dollar spent actually increased. These findings are consistent with those responsible for procurement dramatically reducing purchases after the program, either because they no longer can capture rents, or because they are afraid of being punished for procurement mistakes.

The paper’s final discussion is no less provocative. An excerpt below:

(…)  While the Brazilian anti-corruption program represents a major improvement in monitoring and transparency, the focus of administrative penalties and of public opinion on corruption, instead of on the quality of public services, all seem to have thrown the baby out with the bathwater. These findings suggest that policies that expand the scope of desirable outcomes beyond formal procedures, that differentiate between active and passive waste, and that support local procurement staff in complying with complex guidelines might be important steps towards balancing incentives between procuring and making proper use of public funds.

Given that many other governance/accountability interventions traditionally focus on corruption rather than on the performance of services delivered, practitioners should take note of these findings. In the meantime, the debate on corruption and development gets some good extra fuel.

You can download the paper here [PDF].

Additional resources:

Ferraz, C., & Finan, F. (2007). Exposing corrupt politicians: the effects of Brazil’s publicly released audits on electoral outcomes. Quarterly Journal of Economics. (ungated version) 

Avis, E., Ferraz, C., & Finan, F. (2016). Do Government Audits Reduce Corruption? Estimating the Impacts of Exposing Corrupt Politicians (No. w22443). National Bureau of Economic Research.

Dreher, A., & Gassebner, M. (2013). Greasing the wheels? The impact of regulations and corruption on firm entry. Public Choice, 155(3-4), 413-432.

Aidt, T. S. (2009). Corruption, institutions, and economic development.Oxford Review of Economic Policy, 25(2), 271-291.

Méon, P. G., & Weill, L. (2010). Is corruption an efficient grease?. World development, 38(3), 244-259.

Development Drums Podcast: Daniel Kaufmann and Mushtaq Khan debate the role and importance of tackling corruption as part of a development strategy.

Catching Up on DemocracySpot


It’s been a while, so here’s a miscellaneous post with things I would normally share on DemocracySpot.

Yesterday the beta version of the Open Government Research Exchange (OGRX) was launched. Intended as a hub for research on innovations in governance, the OGRX is a joint initiative by NYU’s GovLab, MySociety and the World Bank’s Digital Engagement Evaluation Team (DEET) (which, full disclosure, I lead). As the “beta” suggests, this is an evolving project, and we look forward to receiving feedback from those who either work with or benefit from research in open government and related fields. You can read more about it here.

Today we also launched the Open Government Research mapping. Same story, just “alpha” version. There is a report and a mapping tool that situates different types of research across the opengov landscape. Feedback on how we can improve the mapping tool – or tips on research that we should include – is extremely welcome. More background about this effort, which brings together Global Integrity, Results for Development, GovLAB, Results for Development and the World Bank, can be found here.


Also, for those who have not seen it yet, the DEET team also published the EvCaptureDEETguidealuation Guide for Digital Citizen Engagement a couple of months ago. Commissioned and overseen by DEET, the guide was developed and written by Matt Haikin (lead author), Savita Bailur, Evangelia Berdou, Jonathan Dudding, Cláudia Abreu Lopes, and Martin Belcher.

And here is a quick roundup of things I would have liked to have written about since my last post had I been a more disciplined blogger:

  • A field experiment in Rural Kenya finds that “elite control over planning institutions can adapt to increased mobilization and participation.” I tend to disagree a little with the author’s conclusion that emphasizes the role of “power dynamics that allow elites to capture such institutions” to explain his findings (some of the issues seem to be a matter of institutional design). In any case, it is a great study and I strongly recommend the reading.
  • A study examining a community-driven development program in Afghanistan finds a positive effect on access to drinking water and electricity, acceptance of democratic processes, perceptions of economic wellbeing, and attitudes toward women. However, effects on perceptions of government performance were limited or short-lived.
  • A great paper by Paolo de Renzio and Joachim Wehner reviews the literature on “The Impacts of Fiscal Openness”. It is a must-read for transparency researchers, practitioners and advocates. I just wish the authors had included some research on the effects of citizen participation on tax morale.
  • Also related to tax, “Consumers as Tax Auditors” is a fascinating paper on how citizens can take part in efforts to reduce tax evasion while participating in a lottery.
  • Here is a great book about e-Voting and other technology developments in Estonia. Everybody working in the field of technology and governance knows Estonia does an amazing job, but information about it is often scattered and, sometimes, of low quality. This book, co-authored by my former colleague Kristjan Vassil, addresses this gap and is a must-read for anybody working with technology in the public sector.
  • Finally, I got my hands on the pictures of the budget infograffitis (or data murals) in Cameroon, an idea that emerged a few years ago when I was involved in a project supporting participatory budgeting in Yaoundé (which also did the Open Spending Cameroon). I do hope that this idea of bringing data visualizations to the offline world catches up. After all, that is valuable data in a citizen-readable format.

picture by ASSOAL


picture by ASSOAL

I guess that’s it for now.

New IDS Journal – 9 Papers in Open Government

2016-01-14 16.51.09_resized

The new IDS Bulletin is out. Edited by Rosemary McGee and Duncan Edwards, this is the first open access version of the well-known journal by the Institute of Development Studies. It brings eight new studies looking at a variety of open government issues, ranging from uptake in digital platforms to government responsiveness in civic tech initiatives. Below is a brief presentation of this issue:

Open government and open data are new areas of research, advocacy and activism that have entered the governance field alongside the more established areas of transparency and accountability. In this IDS Bulletin, articles review recent scholarship to pinpoint contributions to more open, transparent, accountable and responsive governance via improved practice, projects and programmes in the context of the ideas, relationships, processes, behaviours, policy frameworks and aid funding practices of the last five years. They also discuss questions and weaknesses that limit the effectiveness and impact of this work, offer a series of definitions to help overcome conceptual ambiguities, and identify hype and euphemism. The contributions – by researchers and practitioners – approach contemporary challenges of achieving transparency, accountability and openness from a wide range of subject positions and professional and disciplinary angles. Together these articles give a sense of what has changed in this fast-moving field, and what has not – this IDS Bulletin is an invitation to all stakeholders to take stock and reflect.

The ambiguity around the ‘open’ in governance today might be helpful in that its very breadth brings in actors who would otherwise be unlikely adherents. But if the fuzzier idea of ‘open government’ or the allure of ‘open data’ displace the task of clear transparency, hard accountability and fairer distribution of power as what this is all about, then what started as an inspired movement of governance visionaries may end up merely putting a more open face on an unjust and unaccountable status quo.

Among others, the journal presents an abridged version of a paper by Jonathan Fox and myself on digital technologies and government responsiveness (for full version download here).

Below is a list of all the papers:

Rosie McGee, Duncan Edwards
Tiago Peixoto, Jonathan Fox
Katharina Welle, Jennifer Williams, Joseph Pearce
Miguel Loureiro, Aalia Cassim, Terence Darko, Lucas Katera, Nyambura Salome
Elizabeth Mills
Laura Neuman
David Calleb Otieno, Nathaniel Kabala, Patta Scott-Villiers, Gacheke Gachihi, Diana Muthoni Ndung’u
Christopher Wilson, Indra de Lanerolle
Emiliano Treré


World Development Report 2016: Digital Dividends

nationalgeographic_1746433-wblive (1)

The World Development Report 2016, the main annual publication of the World Bank, is out. This year’s theme is Digital Dividends, examining the role of digital technologies in the promotion of development outcomes. The findings of the WDR are simultaneously encouraging and sobering. Those skeptical of the role of digital technologies in development might be surprised by some of the results presented in the report. Technology advocates from across the spectrum (civic tech, open data, ICT4D) will inevitably come across some facts that should temper their enthusiasm.

While some may disagree with the findings, this Report is an impressive piece of work, spread across six chapters covering different aspects of digital technologies in development: 1) accelerating growth, 2) expanding opportunities, 3) delivering services, 4) sectoral policies, 5) national priorities, 6) global cooperation. My opinion may be biased, as somebody who made some modest contributions to the Report, but I believe that, to date, this is the most thorough effort to examine the effects of digital technologies on development outcomes. The full report can be downloaded here.

The report draws, among other things, from 14 background papers that were prepared by international experts and World Bank staff. These background papers serve as additional reading for those who would like to examine certain issues more closely, such as social media, net neutrality, and the cybersecurity agenda.

For those interested in citizen participation and civic tech, one of the papers written by Prof. Jonathan Fox and myself – When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness? – might be of particular interest. Below is the abstract:

This paper reviews evidence on the use of 23 information and communication technology (ICT) platforms to project citizen voice to improve public service delivery. This meta-analysis focuses on empirical studies of initiatives in the global South, highlighting both citizen uptake (‘yelp’) and the degree to which public service providers respond to expressions of citizen voice (‘teeth’). The conceptual framework further distinguishes between two trajectories for ICT-enabled citizen voice: Upwards accountability occurs when users provide feedback directly to decision-makers in real time, allowing policy-makers and program managers to identify and address service delivery problems – but at their discretion. Downwards accountability, in contrast, occurs either through real time user feedback or less immediate forms of collective civic action that publicly call on service providers to become more accountable and depends less exclusively on decision-makers’ discretion about whether or not to act on the information provided. This distinction between the ways in which ICT platforms mediate the relationship between citizens and service providers allows for a precise analytical focus on how different dimensions of such platforms contribute to public sector responsiveness. These cases suggest that while ICT platforms have been relevant in increasing policymakers’ and senior managers’ capacity to respond, most of them have yet to influence their willingness to do so.

You can download the paper here.

Any feedback on our paper or models proposed (see below, for instance) would be extremely welcome.


unpacking user feedback and civic action: difference and overlap

I also list below the links to all the background papers and their titles

Enjoy the reading.

Civic Tech and Government Responsiveness

For those interested in tech-based citizen reporting tools (such as FixMyStreet, SeeClickFix), here’s a recent interview of mine with Jeffrey Peel (Citizen 2015) in which I discuss some of our recent research in the area.


Tiny post on Big Data

Behind much of the excitement with the data revolution is an assumption that decision-making follows a rational actor model. And that’s a huge problem.