Monthly Archives: August 2012

Competitions

Several crowdsourcing competitions were held in recent years.

Red Balloons challenged participants to find 10 red weather balloons moored at secret locations throughout the US.The winning team found all balloons within 9 hours using an incentive scheme that encouraged not only reporting the balloons, but also recruiting new participants.

The Tag Challenge was similar to the Red Balloons challenge, but the search was for 5 individuals moving around 5 cities based on their picture and t-shirt. A similar incentive scheme worked.

The MyHeartMap Challenge crowdsourced creating a map of AEDs throughout Philadelphia. Crowdsourcing did not produce the expected result here. It did work in the Netherlands, however. Cultural differences? Monetary incentives “crowding out” altruistic spirit? Not enough media support?

Incentives in Literature

Financial incentives and the “performance of crowds” by Winter Mason and Duncan J. Watts, KDD-HCOMP ’09

Designing Incentives for Inexpert Human Raters by Aaron Shaw, John Horton, and Daniel Chen (a blog post by Aaron Shaw)

Incentives and Truthful Reporting in Consensus-centric Crowdsourcing by Ece Kamar and Eric Horvitz (2-page poster)
(link contributed by Iyad Rahwan)

A Bayesian Truth Serum for Subjective Data by Dražen Prelec, Science ’04

Peer Prediction without a Common Prior by Jens Witkowski and David Parkes, ACM EC ’12

Crowd IQ: Measuring the Intelligence of Crowdsourcing Platforms by Michal Kosinski, Yoram Bachrach, Gjergji Kasneci, Jurgen Van-Gael, and Thore Graepel, ACM Web Sciences ’12

Incentives in Practice

Video by Jeff Howe, who coined the term “crowdsourcing”:
The Role of Non-Monetary Incentives in Crowdsourcing and Social
Production Projects

(link contributed by Iyad Rahwan)

Slides from a talk by Andrea Wiggins surveying incentives used in citizen science projects
Motivation by Design: Technologies, Experiences, and Incentives.

Stack Overflow has an elaborate point-based system to keep track of reputation and assign privileges. They also assign badges to acknowledge various types of contributions.

Open Street Map uses a seemingly much more limited points and badges system. Here is the info from https://help.openstreetmap.org/faq/ (retrieved 2012-09-24).

When a question or answer is upvoted, the user who posted it will gain some points, called “karma points”. These points serve as a rough measure of the community trust in the user. Various moderation tasks are gradually assigned to users based on those points.

For example, if you ask an interesting question or give a helpful answer, your input will be upvoted. On the other hand if the answer is misleading it will be downvoted. Each vote in favour will add 15 points, each vote against will subtract 1 points. There is a limit of 200 points that can be accumulated per question or answer. The table below explains karma requirements for each type of moderation task.

Extrinsic, Intrinsic, and Social Incentives for Crowdsourcing Development Information in Uganda: A Field Experiment by Michael G. Findley, Madeleine C. Gleave, Robert N. Morello, Daniel L. Nielson