Friday, February 8, 2013

Title: Scaling Crowdsourcing with Decision Theory
Speaker: Dan Weld (University of Washington)
Date: Wednesday, Feb 13th
Time: 12-1pm
Room: GHC 8102 

Abstract:
To ensure quality results from unreliable crowdsourced workers, task designers often construct complex workflows and aggregate worker responses from redundant runs. Unfortunately, this need to organize and manage crowd workers forms a bottleneck limiting the magnitude of tasks that can be handled.  We argue that machine learning and decision-theoretic control are key tools for scaling crowdsourced systems. As an illustration, we present Cascade, a novel workflow that creates a globally-coherent taxonomy over a large dataset of items (eg pictures or text passages) from the collective efforts of crowd workers, each of whom have only a very local glimpse into the data, perhaps performing only 20 seconds of work each. Our evaluation shows that the quality of Cascade’s taxonomy is 80-90% that of experts, can complete much more quickly, and by using decision-theoretic optimization is competitively priced.

Bio:
Daniel S. Weld is Thomas J. Cable / WRF Professor of Computer Science and Engineering at the University of Washington. Weld received a BS from yale in 1982, a PhD from MIT in 1988, a Presidential Young Investigator's award in 1989, an Office of Naval Research Young Investigator's award in 1990, was named AAAI Fellow in 1999, and ACM Fellow in 2006.


No comments:

Post a Comment