MTurk Quality Experiment #1
As I’ve previously mentioned in my Twitter (nolar), I’ve been experimenting with Amazon MTurk, and here are the preliminary results of the quality of the results for one specific task of translation of the texts from English to Russian.
Initially I wanted to name this article as “Zoo of Mechanical Turkeys”, but changed my mind for better SEO of this article (I mean relevancy, not the traffic and popularity of the Dark SEO). I don’t know why MTurk is so associated with turkeys in my mind, when it has nothing to do with these birds or nationality.
I guess, the results will show why it happens so. But before I will give you the results, I want to describe what the task is, and why is it so unusual.