Over 40 Papers on Crowdsourcing for Politics & Policy

pic by James Cridland (flickr)

Today saw the beginning of the biennial conference on Internet, Politics and Policy, convened by the Oxford Internet Institute (University of Oxford) and OII-edited academic journal Policy and Internet. This year’s conference theme is Crowdsourcing for Politics and Policy. Skimming over some papers and abstracts,  here are some of my first (and rather superficial) impressions:

  • Despite the focus of the conference, there are few papers looking at an essential issue of crowdsourcing, namely its potential epistemic attributes. That is, when, why and how “the many are smarter than the few” and the role that technology plays in this.
  • In methodological terms, it seems that very little of the research presented takes advantage of the potential offered by ICT mediated processes when it comes to i) quantitative work with “administrative” data and ii) experimental research design.
  • On the issue of deliberation, it is good to see that more people are starting to look at design issues, slowly moving away from the traditional fixation on the Habermasian ideal (I’ve talked about this in a presentation here).
  • It seems that the majority of the papers focus on European experiences or those from other developed countries. At first, this is not surprising given the location of the conference and the resources that researchers from these countries have (e.g. travel budget). Yet, it may also suggest limited integration between North/South networks of researchers.

With regard to the last point above, it appears that there is a bridge yet to be built between the community of researchers represented by those attending this conference and the emerging community from the tech4accountability space. There’s lots of potential gain for both sides in engaging in a dialogue and, as importantly, a common language. The “Internet & Politics” community would benefit from the tech4accountability’s focus – although sometimes fuzzy – on development outcomes and experiences that emerge from the “South”. Conversely, the tech4accountability community would benefit a great deal by connecting with the existing (and clearly more mature) knowledge when it comes to the intersection of ICT, politics and citizen engagement.

Needless to say, all of the above are initial impressions and broad generalizations, and as such, may be unfair. The OII biennial conference remains, without a doubt, one of the major conferences in its field. You can view the full program of the conference here. I have also listed below in a simplified manner the links to the available papers of the conference according to their respective tracks.

Track A: Harnessing the Crowd

Experiments on Crowdsourcing Policy Assessmen

A Case Study in Modelling Government-Citizen Interaction in Facebook

The potential of Participedia as a crowdsourcing tool for comparative analysis of democratic innovations

Crowd Capital in Governance Contexts

Analyzing Crowd Discussion Towards a more complete model to measure and explain online deliberation

Predicting Events Using Learning Algorithms on Micro Blog Data

A Crowdsourcing Approach to Identify Common Method Bias and Self-Representation

Hate Speech, Machine Classification and Statistical Modelling of Information Flows on Twitter

Internet-mediated cooperative norm setting in the university

Monopsony and the Crowd: Labor for Lemons?

Online labour markets – leveling the playing field for international service markets?

TRACK B: Policy and Government

The Neo-Humanitarians: Assessing the Credibility of Organized Volunteer Crisis Mappers

Let The Users Be The Filter? Crowdsourced Filtering To Avoid Online Intermediary Liability

Regulating Distributed Peer-Production Infrastructures

Population as Auditor of an Election Process in Honduras: VotoSocial

Crowd-sourcing corruption: some challenges, some possible futures

Vertical crowdsourcing: The discourses of activity and the governance of crowds in emergency situations

TRACK C: Engaging the Crowd

What does crowdsourcing legislation entail for the participants? The Finnish case of Avoin Ministeriö

Let the crowd decide? Crowdsourcing ideas as an emerging form of multistakeholder participation

The question of technologically mediated civic political participation reformulated

Discussing Germany’s Future: The Evaluation of Federal Online Citizen Participation

Reprogramming power through crowdsourcing: time, space and citizenship in crowdsourcing for law in Finland

Crowdsourcing as Reflective Political Practice: Building a Location-based Tool for Civic Learning and Engagement

Civic crowdfunding as a marketplace for participation in urban development

Voices in the Noise: Crowdsourcing Public Opinion using Urban Pervasive Technologies

New Book on 25 Years of Participatory Budgeting

Screenshot 2014-06-09 17.17.40

A little while ago I mentioned the launch of the Portuguese version of the book organized by Nelson Dias, “Hope for Democracy: 25 Years of Participatory Budgeting Worldwide”.

The good news is that the English version is finally out. Here’s an excerpt from the introduction:

This book represents the effort  of more than forty authors and many other direct and indirect contributions that spread across different continents seek to provide an overview on the Participatory Budgeting (PB) in the World. They do so from different backgrounds. Some are researchers, others are consultants, and others are activists connected to several groups and social movements. The texts reflect this diversity of approaches and perspectives well, and we do not try to influence that.

(….)

The pages that follow are an invitation to a fascinating journey on the path of democratic innovation in very diverse cultural, political, social and administrative settings. From North America to Asia, Oceania to Europe, from Latin America to Africa, the reader will find many reasons to closely follow the proposals of the different authors.

The book  can be downloaded here [PDF]. I had the pleasure of being one of the book’s contributors, co-authoring an article with Rafael Sampaio on the use of ICT in PB processes: “Electronic Participatory Budgeting: False Dilemmas and True Complexities” [PDF].

While my perception may be biased, I believe this book will be a major contribution for researchers and practitioners in the field of participatory budgeting and citizen engagement in general. Congratulations to Nelson Dias and all the others who contributed their time and energy.

The Problem with Theory of Change

picture by skreened.com

If you are working in the fields of development or governance it’s highly likely that you’ve come across the term “theory of change” (ToC). At a conference a couple of weeks ago, while answering some questions, I mentioned that I preferred not to use the term. The comment didn’t go unnoticed by some witty observers on Twitter, and I was surprised by the number of people who came to me afterwards asking why I do not “like” theory of change.

I can see why some people are attracted to the term. First, “change” is a powerful word: it even helps win elections. And when it comes to governance issues, the need for change is almost a consensus. Second, the user of the word “theory” gives scientific verve to the conversation. However, the problem is precisely the appropriateness of its use if one thinks of the word in scientific terms. It seems that people are saying “theory” when they actually mean (at best) “hypothesis”.

We don’t have to go very far to find out what scientific theory actually is. Keeping to information that is just a click away, let’s take one of the definitions reproduced in Wikipedia’s entry for “theory”:

A scientific theory is a well-substantiated explanation of some aspect of the natural world, based on a body of facts that have been repeatedly confirmed through observation and experiment. Such fact-supported theories are not “guesses” but reliable accounts of the real world.

 Or “scientific theory”:

A theory is a good theory if it satisfies two requirements: It must accurately describe a large class of observations on the basis of a model that contains only a few arbitrary elements, and it must make definite predictions about the results of future observations.

 And here’s a rap video on the difference between theory and hypothesis:

Granted, the word “theory” is often used as a synonym of “hypothesis”, and even dictionaries do so. But the problem of this in the context of current usages of “theory of change” is that it masks the difference between what we know and do not know about something, often conveying a false sense of scientific rigor. And, particularly when it comes to issues such as development and governance, it is extremely important to have a clear distinction between well-substantiated explanations and every other color of hypotheses, assumptions, and guesses. In fact, in any field, it is a minimal requirement for the production of knowledge.

So here’s an interesting exercise. Search on the web for the use of “theory of change” combined with terms like “accountability” and “open government.” Find, for yourself, which ones are really “theories of change” or, rather, merely “hunches of change.”

Most likely, people will keep using theory of change indiscriminately until the next flavor of the moment comes up. In the meantime, beware.

***

Also read: Open Government, Feedback Loops and Semantic Extravaganza

10 Most Read Posts in 2013

Below is a selection of the 10 most read posts at DemocracySpot in 2013. Thanks to all of those who stopped by throughout the year, and happy 2014.

1. Does transparency lead to trust? Some evidence on the subject.

2. The Foundations of Motivation for Citizen Engagement

3. Open Government, Feedback Loops, and Semantic Extravaganza

4. Open Government and Democracy

5. What’s Wrong with e-Petitions and How to Fix them

6. Lawrence Lessig on Sortition and Citizen Participation

7. Unequal Participation: Open Government’s Unresolved Dilemma

8. The Effect of SMS on Participation: Evidence from Uganda

9. The Uncertain Relationship Between Open Data and Accountability

10. Lisbon Revisited: Notes on Participation

Rethinking Why People Participate

Screen Shot 2013-12-29 at 9.08.46 PM

Having a refined understanding of what leads people to participate is one of the main concerns of those working with citizen engagement. But particularly when it comes to participatory democracy, that understanding is only partial and, most often, the cliché “more research is needed” is definitely applicable. This is so for a number of reasons, four of which are worth noting here.

  1. The “participatory” label is applied to greatly varied initiatives, raising obvious methodological challenges for comparative research and cumulative learning. For instance, while both participatory budgeting and online petitions can be roughly categorized as “participatory” processes, they are entirely different in terms of fundamental aspects such as their goals, institutional design and expected impact on decision-making.
  2. The fact that many participatory initiatives are conceived as “pilots” or one-off events gives researchers little time to understand the phenomenon, come up with sound research questions, and test different hypotheses over time.  The “pilotitis” syndrome in the tech4accountability space is a good example of this.
  3. When designing and implementing participatory processes, in the face of budget constraints the first victims are documentation, evaluation and research. Apart from a few exceptions, this leads to a scarcity of data and basic information that undermines even the most heroic “archaeological” efforts of retrospective research and evaluation (a far from ideal approach).
  4. The semantic extravaganza that currently plagues the field of citizen engagement, technology and open government makes cumulative learning all the more difficult.

Precisely for the opposite reasons, our knowledge of electoral participation is in better shape. First, despite the differences between elections, comparative work is relatively easy, which is attested by the high number of cross-country studies in the field. Second, the fact that elections (for the most part) are repeated regularly and following a similar design enables the refinement of hypotheses and research questions over time, and specific time-related analysis (see an example here [PDF]). Third, when compared to the funds allocated to research in participatory initiatives, the relative amount of resources channeled into electoral studies and voting behavior is significantly higher. Here I am not referring to academic work only but also to the substantial resources invested by the private sector and parties towards a better understanding of elections and voting behavior. This includes a growing body of knowledge generated by get-out-the-vote (GOTV) research, with fascinating experimental evidence from interventions that seek to increase participation in elections (e.g. door-to-door campaigns, telemarketing, e-mail). Add to that the wealth of electoral data that is available worldwide (in machine-readable formats) and you have some pretty good knowledge to tap into. Finally, both conceptually and terminologically, the field of electoral studies is much more consistent than the field of citizen engagement which, in the long run, tends to drastically impact how knowledge of a subject evolves.

These reasons should be sufficient to capture the interest of those who work with citizen engagement. While the extent to which the knowledge from the field of electoral participation can be transferred to non-electoral participation remains an open question, it should at least provide citizen engagement researchers with cues and insights that are very much worth considering.

This is why I was particularly interested in an article from a recently published book, The Behavioral Foundations of Public Policy (Princeton). Entitled “Rethinking Why People Vote: Voting as Dynamic Social Expression”, the article is written by Todd Rogers, Craig Fox and Alan Berger. Taking a behavioralist stance, the authors start by questioning the usefulness of the rationalist models in explaining voting behavior:

“In these [rationalist] models citizens are seen as weighing the anticipated trouble they must go through in order to cast their votes, against the likelihood that their vote will improve the outcome of an election times the magnitude of that improvement. Of course, these models are problematic because the likelihood of casting in the deciding vote is often hopelessly small. In a typical state or national election, a person faces a higher probability of being struck by a car on the way to his or her polling location than of casting the deciding vote.”

(BTW, if you are a voter in certain US states, the odds of being hit by a meteorite are greater than those of casting the deciding vote).

Following on from the fact that traditional models cannot fully explain why and under which conditions citizens vote, the authors develop a framework that considers voting as a “self-expressive voting behavior that is influenced by events occurring before and after the actual moment of casting a vote.” To support their claims, throughout the article the authors build upon existing evidence from GOTV campaigns and other behavioral research. Besides providing a solid overview of the literature in the field, the authors express compelling arguments for mobilizing electoral participation. Below are a few excerpts from the article with some of the main takeaways:

  • Mode of contact: the more personal it is, the more effective it is

“Initial experimental research found that a nonpartisan face-to-face canvassing effort had a 5-8 percentage point mobilizing effect in an uncontested midterm elections in 1998 (Gerber and Green 2000) compared to less than a 1 percentage point mobilizing effect for live phone calls and mailings. More than three dozen subsequent experiments have overwhelmingly supported the original finding (…)”

“Dozens of experiments have examined the effectiveness of GOTV messages delivered by the telephone. Several general findings emerge, all of which are consistent with the broad conclusion that the more personal a GOTV strategy, the more effective. (…) the most effective calls are conducted in an unhurried, “chatty manner.”

“The least personal and the least effective GOTV communication channels entail one way communications. (…) written pieces encouraging people vote that are mailed directly to households have consistently been shown to produce a mall, but positive, increase in turnout.”

  • Voting is affected by events before and after the decision

“One means to facilitate the performance of a socially desirable behavior is to ask people to predict whether they will perform the behavior in the future. In order to present oneself in a favorable light or because of wishful thinking or both, people are generally biased to answer in the affirmative. Moreover, a number of studies have found that people are more likely to follow through on a behavior after they predicted that they will do so (….) Emerging social-networking technologies provide new opportunities for citizens to commit to each other that they will turnout in a given election. These tools facilitate making one’s commitments public, and they also allow for subsequently accountability following an election (…) Asking people to form a specific if-then plan of action, or implementation intention, reduces the cognitive costs of having to remember to pursue an action that one intends to perform. Research shows that when people articulate the how, when and where of their plan to implement an intended behavior, they are more likely to follow through.”

(Not coincidentally, as noted by Sasha Issenberg in his book The Victory Lab, during the 2010 US presidential election millions of democrats received an email reminding them that they had “made a commitment to vote in this election” and that “the time has come to make good on that commitment. Think about when you’ll cast your vote and how you’ll get there.”)

“ (…) holding a person publicly accountable for whether or not she voted may increase her tendency to do so. (…) Studies have found that when people are merely made aware that their behavior will be publicly known, they become more likely to behaving in ways that are consistent with how they believe others think they should behave. (…) At least, at one point Italy exposed those who failed to vote by posting the names of nonvoters outside of local town halls.”

(On the accountability issue, also read this fascinating study [PDF] by Gerber, Green & Larimer)

  • Following the herd: affinitive and belonging needs

“People are strongly motivated to maintain feelings of belonging with others and to affiliate with others. (…) Other GOTV strategies that can increase turnout by serving social needs could involve encouraging people to go to their polling place in groups (i.e., a buddy system), hosting after-voting parties on election day, or encouraging people to talk about voting with their friends, to name a few.”

“(…) studies showed that the motivation to vote significantly increased when participants heard a message that emphasized high expected turnout as opposed to low expected turnout. For example, in the New Jersey study, 77% of the participants who heard the high-turnout script reported being “absolutely certain” that they would vote, compared to 71% of those who heard the low-turnout script. This research also found that moderate and infrequent voters were strongly affected by the turnout information.”

  • Voting as an expression of identity

“(….) citizens can derive value from voting through what the act displays about their identities. People are willing to go to great lengths, and pay great costs, to express that they are a particular kind of person. (….) Experimenters asked participants to complete a fifteen-minute survey that related to an election that was to occur the following week. After completing the survey, the experimenter reviewed the results and reported to participants what their responses indicated. Participants were, in fact, randomly assigned to one of two conditions. Participants in the first condition were labeled as being “above-average citizens[s] … who [are] very likely to vote,” whereas participants in the second condition were labeled as being “average citizen[s] … with an average likelihood of voting. (….) These identity labels proved to have substantial impact on turnout, with 87% of “above average” participants voting versus 75% of “average” participants voting.”

For those working with participatory governance, the question that remains is the extent to which each of these lessons is applicable to non-electoral forms of participation. The differences between electoral and non-electoral forms of participation may cause these techniques to generate very different results. One difference relates to public awareness about participation opportunities. While it would be safe to say that during an important election the majority of citizens are aware of it, the opposite is true for most existing participatory events, where generally only a minority is aware of their existence. In this case, it is unclear whether the impact of mobilization campaigns would be more or less significant when awareness about an event is low. Furthermore, if the act of voting may be automatically linked to a sense of civic duty, would that still hold true for less typical forms of participation (e.g. signing an online petition, attending a community meeting)?

The answer to this “transferability” question is an empirical one, and one that is yet to be answered.  The good news is that while experiments that generate this kind of knowledge are normally resource intensive, the costs of experimentation are driven down when it comes to technology-mediated citizen participation. The use of A/B testing during the Obama campaign is a good example. Below is an excellent account by Dan Siroker on how they conducted online experiments during the presidential campaign.

Bringing similar experiments to other realms of digital participation is the next logical step for those working in the field. Some organizations have already started to take this seriously . The issue is whether others, including governments and donors, will do the same.

Why ‘I-Paid-A-Bribe’ Worked in India but Failed in China

source: China Daily

Interesting paper by Yuen Yuen Ang, Political Scientist at the University of Michigan:

Authoritarian states restrain online activism not only through repression and censorship, but also by indirectly weakening the ability of netizens to self-govern and constructively engage the state. I demonstrate this argument by comparing I-Paid-A-Bribe (IPAB) — a crowd-sourcing platform that collects anonymous reports of petty bribery — in India and China. Whereas IPAB originated and has thrived in India, a copycat effort in China fizzled out within months. Contrary to those who attribute China’s failed outcome to repression, I find that even before authorities shut down IPAB, the sites were already plagued by internal organizational problems that were comparatively absent in India. The study tempers expectations about the revolutionary effects of new media in mobilizing contention and checking corruption in the absence of a strong civil society.

And a brief video with Yuen Yuen

Also read

I Paid a Bribe. So What? 

Open Government and Democracy