Giannis Haralabopoulos
Paid Crowdsourcing, Low Income Contributors, and Subjectivity
Haralabopoulos, Giannis; Wagner, Christian; McAuley, Derek; Anagnostopoulos, Ioannis
Authors
CHRISTIAN WAGNER Christian.Wagner@nottingham.ac.uk
Professor of Computer Science
Derek McAuley
Ioannis Anagnostopoulos
Contributors
J. MacIntyre
Editor
I. Maglogiannis
Editor
L. Iliadis
Editor
E. Pimenidis
Editor
Abstract
Scientific projects that require human computation often resort to crowdsourcing. Interested individuals can contribute to a crowdsourcing task, essentially contributing towards the project's goals. To motivate participation and engagement, scientists use a variety of reward mechanisms. The most common motivation, and the one that yields the fastest results, is monetary rewards. By using monetary, scientists address a wider audience to participate in the task. As the payment is below minimum wage for developed economies, users from developing countries are more eager to participate. In subjective tasks, or tasks that cannot be validated through a right or wrong type of validation, monetary incentives could contrast with the much needed quality of submissions. We perform a subjective crowdsourcing task, emotion annotation, and compare the quality of the answers from contributors of varying income levels, based on the Gross Domestic Product. The results indicate a different contribution process between contributors from varying GDP regions. Low income contributors, possibly driven by the monetary incentive, submit low quality answers at a higher pace, while high income contributors provide diverse answers at a slower pace.
Online Publication Date | May 15, 2019 |
---|---|
Publication Date | May 15, 2019 |
Deposit Date | Mar 29, 2021 |
Publicly Available Date | Apr 23, 2021 |
Publisher | Springer Verlag |
Pages | 225-231 |
Series Title | IFIP Advances in Information and Communication Technology |
Series Number | 560 |
Book Title | Artificial Intelligence Applications and Innovations: AIAI 2019 IFIP WG 12.5 International Workshops: MHDW and 5G-PINE 2019, Hersonissos, Crete, Greece, May 24–26, 2019, Proceedings |
ISBN | 9783030199081 |
DOI | https://doi.org/10.1007/978-3-030-19909-8_20 |
Public URL | https://nottingham-repository.worktribe.com/output/4554105 |
Publisher URL | https://link.springer.com/chapter/10.1007%2F978-3-030-19909-8_20 |
Additional Information | Haralabopoulos G., Wagner C., McAuley D., Anagnostopoulos I. (2019) Paid Crowdsourcing, Low Income Contributors, and Subjectivity. In: MacIntyre J., Maglogiannis I., Iliadis L., Pimenidis E. (eds) Artificial Intelligence Applications and Innovations. AIAI 2019. IFIP Advances in Information and Communication Technology, vol 560. Springer, Cham. https://doi.org/10.1007/978-3-030-19909-8_20 |
Files
Paid Crowdsourcing Low Income Workers And Subjectivity Draft
(367 Kb)
PDF
You might also like
Alpha-cut based compositional representation of fuzzy sets and exploration of associated fuzzy set regression
(2022)
Conference Proceeding
Visualization of Interval Regression for Facilitating Data and Model Insight
(2022)
Conference Proceeding
Does Permitting Uncertain Estimates Help or Hinder the Wisdom of Crowds?
(2022)
Conference Proceeding
Downloadable Citations
About Repository@Nottingham
Administrator e-mail: discovery-access-systems@nottingham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search