You are here

ACRyLIQ: Leveraging DBpedia for Adaptive Crowdsourcing in Linked Data Quality Assessment

Authors: 

Umair Ul Hassan, Amrapali Zaveri, Edgard Marx, Edward Curry, Jens Lehmann

Publication Type: 
Refereed Conference Meeting Proceeding
Abstract: 
Crowdsourcing has emerged as a powerful paradigm for quality assessment and improvement of Linked Data. A major challenge of employing crowdsourcing, for quality assessment in Linked Data, is the cold-start problem: how to estimate the reliability of crowd workers and assign the most reliable workers to tasks? We address this challenge by proposing a novel approach for generating test questions from DBpedia based on the topics associated with quality assessment tasks. These test questions are used to estimate the reliability of the new workers. Subsequently, the tasks are dynamically assigned to reliable workers to help improve the accuracy of collected responses. Our proposed approach, ACRyLIQ, is evaluated using workers hired from Amazon Mechanical Turk, on two real-world Linked Data datasets. We validate the proposed approach in terms of accuracy and compare it against the baseline approach of reliability estimate using gold-standard task. The results demonstrate that our proposed approach achieves high accuracy without using gold-standard task.
Conference Name: 
20th International Conference on Knowledge Engineering and Knowledge Management
Digital Object Identifer (DOI): 
10.1007/978-3-319-49004-5_44
Publication Date: 
04/11/2016
Conference Location: 
Italy
Research Group: 
Institution: 
National University of Ireland, Galway (NUIG)
Open access repository: 
No
Publication document: