Research Repository

See what's under the surface

Rating crowdsourced annotations: evaluating contributions of variable quality and completeness

Foody, Giles M.

Authors

Giles M. Foody giles.foody@nottingham.ac.uk

Abstract

Crowdsourcing has become a popular means to acquire data about the Earth and its environment inexpensively, but the data-sets obtained are typically imperfect and of unknown quality. Two common imperfections with crowdsourced data are the contributions from cheats or spammers and missing cases. The effect of the latter two imperfections on a method to evaluate the accuracy of crowdsourced data via a latent class model was explored. Using simulated and real data-sets, it was shown that the method is able to derive useful information on the accuracy of crowdsourced data even when the degree of imperfection was very high. The practical potential of this ability to obtain accuracy information within the geospatial sciences and the realm of Digital Earth applications was indicated with reference to an evaluation of building damage maps produced by multiple bodies after the 2010 earthquake in Haiti. Critically, the method allowed data-sets to be ranked in approximately the correct order of accuracy and this could help ensure that the most appropriate data-sets are used.

Journal Article Type Article
Publication Date Nov 15, 2013
Journal International Journal of Digital Earth
Print ISSN 1753-8947
Electronic ISSN 1753-8947
Publisher Taylor & Francis
Peer Reviewed Peer Reviewed
Institution Citation Foody, G. M. (2013). Rating crowdsourced annotations: evaluating contributions of variable quality and completeness. International Journal of Digital Earth, doi:10.1080/17538947.2013.839008
DOI https://doi.org/10.1080/17538947.2013.839008
Publisher URL http://www.tandfonline.com/action/showCopyRight?doi=10.1080%2F17538947.2013.839008#tabModule
Copyright Statement Copyright information regarding this work can be found at the following address: http://creativecommons.org/licenses/by/4.0
Additional Information This is an Open Access article. Non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly attributed, cited, and is not altered, transformed, or built upon in any way, is permitted. The moral rights of the named author(s) have been asserted.

Permission is granted subject to the terms of the License under which the work was published. Please check the License conditions for the work which you wish to reuse. Full and appropriate attribution must be given. This permission does not cover any third party copyrighted material which may appear in the work requested.

Files

Line_22_Rating_crowdsourced_annotations....pdf (368 Kb)
PDF

Copyright Statement
Copyright information regarding this work can be found at the following address: http://creativecommons.org/licenses/by/4.0




Downloadable Citations