Improving Amazon Mechanical Turk User Interface for Turkers aka Crowd Workers

While creating solutions for crowdsourcing people seldom think about the way workers on Amazon Mechanical Turk will actually find your tasks. If you would try to use AMT as a worker you would find this surprisingly difficult.

The interface looks like it’s from last decade, search is cumbersome and limited. People used to well designed and snappy web applications like Google Plus, Facebook or Twitter will have a hard time there. So as a part of a research project for Chris Callison-Burch we’ve tried to improve that experience.

Early this year we’ve started working on a service called Crowd-Workers.com a simple site that uses a chrome extension to crawl Amazon Mechanical Turk for various kind of statistics that will help workers to choose the best possible tasks for them.

So how does it all work? First of all we present one key metric that Amazon Mechanical Turk is not showing, that is an average hourly rate for a single task type. We calculate it based on multiple entries from different workers. With that knowledge it’s much easier to choose a task that is worth working on.

In the future we are planning to show statistics like the average time after requester actually releases the funds for completed tasks and rejection rates together with rejection reasons. That way the transparency on requesters will improve a lot and will enable more effective choosing of work.

We are also planning to implement a recommendation tool that would notify workers of interesting or monitored hits and recommend hits to work on that have an expected set of features like average hourly rate, time to pay etc.

Currently we’ve got a couple hundred users and we’ve gathered data on around 154032 tasks submissions from 69478 hits and growing 🙂