I am a worker on Crowdflower, and if you are a Task Author, then I have a bone to pick with you! Actually, several bones!
Read on...
The biggest bone: Flags. The bane of the worker's existence. Feared among the most diligent workers. Why? Because when a task author gives a major flag, not only is the worker prevented from doing level 3 tasks for THAT task author, but he is also prevented from doing level 3 tasks for EVERY OTHER task author!
Incredible, isn't it? I have no idea why some brainiac at Crowdflower ever thought that THAT was a great idea, but evidently SOMEONE did.
Were you aware of that? Did you know that you held such power? That if you had a problem with a worker, that it isn't just between you and that worker, but your actions also affects other task authors? And there is rarely any turning back. Once that major flag is given, it's lights out for the worker & level 3 tasks forever. Even a bank robber can get time off for good behavior, but get a major flag on Crowdflower and it's there for life with no parole. All because of a task that pays pennies. To quote a Crowdflower employee who thinks pretty colors on the blog is more important than actually improving their system "flags are a fact of life".
Is Crowdflower sounding a bit foolish right now? I hope so, because they are. And YOU'RE not sounding all that great either, what with some of the downright silly reasons for some of these flags. Giving you the benefit of the doubt, maybe you weren't fully aware of how these flags affect the workers, so when you hand out a major flag for trivial reasons such as submitting too fast, or taking too long to submit, or maybe you just woke up on the wrong side of the bed, you quite possibly had no idea that you were handing out a life sentence that would prevent that worker from doing level 3 tasks FOREVER for ALL task authors. If I was one of the other task authors, I would be really ticked off that you were taking it upon yourself to prevent workers from working on my tasks.
Another bone: Broken tasks and bad test questions. Seriously, put a bit more effort into creating and verifying your tasks before you submit them - some of the most frustrating things when trying to work on tasks are the tasks that either won't submit because they're broken, images that don't load, or being "corrected" on a test question when YOUR answer is the one that is wrong. And making it worse is when you don't correct your answer! Many times I have put too much faith in a task author, thinking surely he's corrected his error, only to have my answer marked wrong yet again. (Mr. Look at Advertisements, I'm pointing at you!)
This is enough to make a worker want to scream! AND it's enough to make a worker decide to give the answer he KNOWS is wrong, but it's the answer the task author THINKS is right, and as a result, the task author gets bad data. And at this point, he deserves it. Why would a worker want to continue to erode his accuracy level giving the actual right answer when the task author can't be bothered to correct it?
And submitting a ticket to Crowdflower is useless because they quote some ridiculous "40%" rule ("40% of the workers didn't submit a ticket so too bad so sad for you, there's nothing we will do"). That actually makes them partially responsible for the bad data you're getting.
Crowdflower is equally useless regarding the unscrupulous task authors that seemingly create tasks just so they can flag the workers knowing that those flags are usually permanent. Unfortunately Crowdflower buries their heads in the sand and/or passes the buck when this gets reported, likely because they consider the workers to be expendable. They just deny deny deny.
Hence the main reason for this rant. The workers have noone on their side. Oh sure, there's a Crowdflower employee that visits the message forums claiming to want to help, asking for ideas to improve the task list. This seems to be a yearly request that serves only to waste time and make the workers even more frustrated. On any given day in a particular forum there is a long list of broken tasks being reported, which one would think would raise red flags that something is amiss when there are THAT many broken tasks, but sadly that isn't the case. Sure, a few of the tasks get fixed, but this employee seems to need a narrated powerpoint presentation highlighting why the task is broken before she'll even consider taking action. Describing the problem in plain English doesn't seem to sink in.
And there's more - but to me, besides the flag fiasco, one of the most important issues is the fact that the workers have no way to identify the task authors so that the bad ones can be avoided. How utterly stupid. They might as well say "here, do some tasks from this list. Some of the task authors will give you a major flag that will never be removed, just because they can. We won't tell you which ones, or how to avoid them. Think of it as a game, a gambling game. Happy tasking!"
My point? Since Crowdflower is all talk and no action when it comes to protecting the workers, I want you, the task authors, to step up to the plate for us. Surely you can't think it's fair for you to give flags that effectively ends any possibility of the worker ever doing level 3 tasks for ANYONE. As responsible task authors, that HAS to make you uncomfortable! Let Crowdflower know that that is not acceptable. Tell them that while you have the right to prohibit a worker from working on YOUR tasks, that you shouldn't have the right to prohibit workers from working on the tasks of other task authors.
As for Crowdflower - where to begin...for starters, how about a little perspective - these are tasks for pennies! PENNIES! This flag system you have going is like trying to kill a fly with a sledge hammer - it's overkill!
The flag system has to go - it SHOULDN'T be a fact of life! Look, I know you have a problem with cheaters - I've seen first hand some of the things they try to pull - but you're catching too many innocent workers in your nets, and that is not acceptable!
And the passing the buck has to stop - there are good honest task workers who have received undeserved flags, and are pleading for help from your reps, only to be told "I have no control, I can't help". Do you have any idea how discouraging and frustrating that is? Here's a thought - if YOU can't help, then find someone who CAN! You need to come up with a better system to punish the proven cheaters that DOESN'T leave honest workers stranded, badgeless, and without hope.
I remember the days BEFORE badges - when tasks had a training mode that you could keep repeating until you finally passed and were able to move on to work mode. We didn't have to worry about The Hammer coming down from abusive task authors (and YES, Ms. CF Rep, there ARE abusive TAs - you can deny it all you want but experienced workers know better).
I don't know what happened - maybe CF got a new CEO fresh out of military school who was determined to WHIP THOSE WORKERS INTO SHAPE!! No excuses! No mercy!! No humanity for those peons! We will show them WHO IS BOSS!!
Sounds ridiculous, doesn't it. Welcome to OUR world.
