Assessing the accuracy of land cover change with imperfect ground reference data

Foody, Giles M.

Abstract

The ground data used as a reference in the validation of land cover change products are often not an ideal gold standard but degraded by error. The effects of ground reference data error on the accuracy of land cover change detection and the accuracy of estimates of the extent of change was evaluated. Twelve data sets were simulated to allow the exploration of the impacts of a spectrum of ground data imperfections on the estimation of the producer’s and user’s accuracy of change as well as of change extent. Simulated data were used since this ensured that the actual properties of the data were known and to exclude effects due to other sources of ground reference data error; although the impacts of simulated reference data on two real confusion matrices is also illustrated. The imperfections evaluated ranged from the inclusion of small amounts of known error into the ground reference data through to the extreme situation in which ground data were absent. The results show that even small amounts of error in the ground reference data can introduce large error into studies of land cover change by remote sensing and reinforce the desire to avoid the expression ground truth as this might imply that the data are a gold standard reference. The effect of reference data imperfections was dependent on the degree of association between the errors in the cross tabulated data sets. For example, in the scenarios investigated, a 10% error in the reference data set introduced an under-estimation of the producer’s accuracy of 18.5% if the errors were independent but an over-estimation of the producer’s accuracy of 12.3% if the errors were correlated. The magnitude of the mis-estimation of the producer’s accuracy was also a function of the amount of change and greatest at low levels of change. The amount of land cover change estimated also varied greatly as a function of ground reference data error. Some possible methods to reduce or even remove the impacts of ground reference data error were illustrated. These ranged from simple algebraic means to estimate the actual values of accuracy and change extent if the imperfections were known through to a latent class analysis that allowed the assessment of classification accuracy and estimation of change extent without the use of ground reference data if the underlying model is defined appropriately.

Citation

Foody, G. M. (2010). Assessing the accuracy of land cover change with imperfect ground reference data. Remote Sensing of Environment, 114(10), https://doi.org/10.1016/j.rse.2010.05.003

• Journal Article Type: Article
• Publication Date: Jan 1, 2010
• Deposit Date: Jan 31, 2014
• Publicly Available Date: Jan 31, 2014
• Journal: Remote Sensing of Environment
• Print ISSN: 0034-4257
• Electronic ISSN: 1879-0704
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 114
• Issue: 10
• DOI: https://doi.org/10.1016/j.rse.2010.05.003
• Public URL: https://nottingham-repository.worktribe.com/output/1012581
• Publisher URL: http://www.sciencedirect.com/science/article/pii/S0034425710001434
• Additional Information: “NOTICE: this is the author’s version of a work that was accepted for publication in Remote Sensing of Environment. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Remote Sensing of Environment, 114(10), (2010), doi:10.1016/j.rse.2010.05.003