Hey, CrowdFlower!  Wake up and smell the coffee cheaters!


Last 24 hours

[Venezuela] *****    1,912
[Venezuela] *****    1,420
[Venezuela] *****    1,119
[Venezuela] *****    1,107
[Venezuela] *****    1,091
[Venezuela] *****    944
[Venezuela] *****    929
[Venezuela] *****    926
[Venezuela] *****    911
[Venezuela] *****    883

the names have been changed to protect the cheaters

It's obvious by now that CF doesn't care about closing any holes - or maybe they are that incompetent that they don't know how

Rainforest:  they make changes to benefit and help the workers

CrowdFlower:  they make changes to the detriment of the workers while speaking with a forked tongue

I've said it before and I'll say it again: For the love of all that is holy, will SOMEONE please tell Mr. Look At Advertisements to fix his danged test questions?!  And make sure the right answer is actually a freakin' option!!

Am I the ONLY one that actually cares about good data??

Now I have to get a new desk - this one broke when I was banging my head against it

A tearful farewell today from a CF peep

Wonder if she got flagged...if she did, then she wouldn't be able to work for any other employer.  ever.  Or until the flag got removed.  In six years.

Can you say "karma"

To the clown that got the very helpful Rainforest thread closed on ClixSense - you are a piece of work.  And then you have the gall to start your own Rainforest thread, threatening to report anyone that is rude towards anybody??  After you posted that the Rainforest engineers could go to hell?  And you posted a whole lot more rude remarks regarding the Rainforest team. I'm not going to quote them here, but I'm sure there are many screenshots.

I hope each and every reader of my blog reports this chucklehead and his remarks to Rainforest as they do NOT tolerate that kind of rudeness towards their staff.  You all have screenshots.  I know you do.

****

Yeah, I edited my above post a bit.  I shouldn't have lowered myself to that chucklehead's level.  In my anger I called him something that I shouldn't have, and I apologize.  In this blog and all of my prior blogs I have always made an effort to avoid using inappropriate language.

Now, some of you peeps may be wondering why I am so defensive of Rainforest, but not of Crowdflower.

It's simple - Rainforest asks for our opinion and seems to genuinely make an effort to improve, though they may not always succeed.  But they do try.

Crowdflower asks for our opinion with zero intention of making things better for the workers.  They.Just.Don't.Care. Period.

Lots of sour grapes coming from the cheaters who were banned

Really, dudes, you cheated, you got caught, you got banned.  Quit trying to deflect the blame.  The blame - 100% of it - is on YOU, and noone else.

Frankly, with the magnitude of this cheating that happened, you should all be banned from every GPT site you completed tasks on.

If you were cheating on CF to THAT degree, what else are you cheating on?

I'm just gobsmacked.

Another round of bans is happening today.  I think I know why, and I say good stuff!

I quite often have a bone to pick with TAs and some of them need to be smacked upside the head (not really.  or maybe yes really) but that one I feel bad for.  All those tasks and all that data is gone out the window.

I just hope that the good peeps like myself don't end up paying the price again a la QuizModeGate

If CF is supposedly so serious about cheaters, why don't they take a look at those that do 11,000+ tasks in a month - that's one task every 3.638 minutes 24 hours a day for 30 days straight.  STRAIGHT.  NON STOP.

Those that share the test answers should be the LEAST of CF's worries.

I put forth that CF doesn't care one bit about cheaters.  What they care about is creating the illusion that they DO care.



And they create this illusion on the backs of the honest workers.

Trust and safety my arse.

Oh, did I already say that?

Well this just proves one of the points I made in my previous post.  A little looky loo at the forums and there is ONE lonely "broken task" thread.  Not too long ago there were times when the whole front page of the forum was nothing but "broken task" threads.

And the response to this one?  "file a ticket"

Which probably goes straight to the circular file.

CF's workload just decreased immensely by preventing the workers from reporting broken tasks.

CF's new motto:  Hide Hide Hide

What CF's motto SHOULD be:  We Provide Quality Jobs for Quality Workers

but providing quality jobs involves actual work on the part of CF AND the TAs

yeah, I know. what am I thinking!

another vote against QuizCrap:

how would contributor know now about fair and unfair correction

they won't know.  that's the whole reason behind this.  CF is getting tired of all the "broken task" threads in the forums, and all of the "broken task" tickets, and rather than get firm with the TAs, they decided just to hide everything from the workers.

cheating can still happen - if  a cheater gets 100% in quiz mode then they'll know they had all the right answers, and those questions and correct answers can be passed along to their co-cheaters

next on CF's agenda - removing the score from the prying eyes of the workers

they don't need to know how well they're doing - all the workers need to know is if they're still in the game.

mark my words, this will happen.

trust and safety my arse

The negative reaction to CF's latest jab at the contributors continues.  I'm not sure where CF gets this "mixed feelings" bit.  Unless they mean that feelings are mixed in that some are disgusted and some are pissed.

And CF thinks it's great to see a dialogue between everyone to understand our thoughts?

Translation:  "We know you're mad as hell and we don't give a damn cuz there's plenty more where YOU came from so we'll continue to treat you like scum and there's not a darn thing you can do about it"

Now there's going to be a Webinar Q&A so the contributors can voice their concerns.

Why?

We know it's going to be a sham.

Fool us once, shame on you.

Keep fooling us and your precious TAs are going to look elsewhere for quality work because continuing to destroy the morale of the workers will ensure they don't get that quality work here.

Turns out this new quiz mode crapola was suggested over a year ago and earned itself 7 votes.  Kudos to the person that suggested it if his evil plan was to make CF look even worse than it already does when it comes to trust.  Or the lack thereof.

Another suggestion that had 8 votes was to have an optional practice mode before going into quiz mode, where the worker wouldn't be penalized for incorrect answers, and would be shown their mistakes along with an explanation.

That suggestion was fairly well received by other TAs, and Lukas himself liked it, adding that asking contributors how confident they are in their answers might be interesting as well.

So why is it that CF chose the suggestion that alienates the workers and creates an atmosphere of distrust, rather than the one that would be most beneficial to all involved?

Because CF assumes that ALL workers are cheaters, and would (lazily) rather punish the majority due to the actions of the minority.

Sometimes I like to google things just to see what I can find

I decided to google CF as I was curious if others are starting to question their competence, as I am

Lo and behold what do I find but something said - by a TA, no less:

CrowdFlower have since changed their pricing structure, without warning and without any announcements on their part

That was from a year ago

Since their Trust and Safety Initiative has been a Big Fat Fail, not to mention a sham, maybe they should switch gears and work on communication

Oh, who am I kidding.  I'm sure they'd find a way to screw that up too

Continuing with my googling I found this gem, from a CF employee:

we know there are issues with CrowdFlower tasks, but we really are making strides to improve the Contributor experience. And we certainly do listen to our Contributors and value all the feedback we receive. 

the kicker?  this is from over THREE YEARS AGO!  And they've been doing the EXACT OPPOSITE!

I have no words...none that I can say in a G-rated blog

Soooo...did I really think that CF would do the honorable thing?

Yes, I did.  I'm THAT naive.

I had no idea that they would take this destructive path

I wonder what else they have in their (unethical dishonest) bag of tricks

This sums up the general consensus of CF's latest CrapFest:

I think that blog post is insulting

Trust is a two way street and crowdflower is gradually eroding that with it's taskers.

Yep

The crooked TAs and CF are like 2 peas in a pod

"I want my workers to be browbeaten, mistreated, discouraged and belittled"

said a TA never

Finally They Speak!

So CF finally posted an explanation and an apology for QuizModeGate

Just when I think they can't top themselves in the B.S. Department, they go and prove me wrong.

It is extremely important to us that customers trust CrowdFlower to keep their data safe and contributors feel safe tasking on our platform.

Every freaking thing CF has done totally proves that the last part of that sentence is a bold faced lie!

The contributors feel LESS SAFE now than they ever have been.

Does CF not read and COMPREHEND the complaints in the forums??

Someone at CF needs to give their head a shake

***

For those that don't have hip waders and are therefore unable to get through that CF post, I've condensed it for you:

We at Crowdflower will continue to protect the TAs and screw the contributors.

Still no update on QuizModeGate

Is doing the right thing really such a difficult thing to do?

With CF is sure seems that way.  TAs might want to consider if they are giving their business to the right company.  After all, if the workers can't trust CF to do the right thing for them, how can CF be trusted to do the right thing for the TAs?

"Sure, I lied to HER, but I promise, I won't lie to YOU"

Uh huh.

Holy crap on a cracker, how long does it take CF to address the "No Explanations for Corrections cuz It's a Secret" fiasco?  C'mon, peeps! It's been over a week!  How long does it take to admit an error in judgement?

Or maybe it takes this long to fire the bozo that dreamed up that change?

I'm going to put on my tinfoil hat and suggest that maybe a competing company has a mole inside CF whose job it is to come up with various harebrained schemes to show the extreme ineptness going on and then make a hostile takeover.

If that's what's going on, then it's a job well done.

Ok, this one made me laugh, in regards to the new change of the worker not being told where they went wrong if they're expelled in quiz mode:

I suspect the error is in the formatting ...., the TA wants them in a different format to what the instructions say, but he could have been expecting me to type the names of cheeses for all I know!

Methinks CF needs to go back to the drawing board on this one - clearly it was a knee jerk reaction that wasn't thought through, and now we have to hope they don't let pride stand in the way of doing the right thing.

I love reading little gems that are said much better than the way I have said things:

Well, I'd have to say that this new "plan" certainly DOES go well with the recently adopted system that allows the OPINION of ONE TA to determine that a contributor account should be TOTALLY SUSPENDED.....

Respect for contributors? Apparently THAT is not something CF has heard of

Then there's this from the CF rep in regards to one of the gazillion broken tasks:

 Most contributors didn't pass quiz. Won't affect your stats.

OMG, CF peeps - does that NOT PROVE HOW INSANE YOUR LATEST "IMPROVEMENT" IS??

Sorry for the caps.  They implied that I was yelling.

Actually, I was.

I really find it hard to believe that Crowdflower truly cares about the "integrity of the data" when they continue to make it difficult for the workers to do the tasks well.

In fact, I'll go so far as to say they DON'T CARE about the data.  Not. One. Bit.

Ya hear that, Oh Mighty TAs?  THEY DON'T CARE ABOUT YOUR PRECIOUS DATA!

All they care about is attracting more TAs, and if they have to crush the souls of the workers to do that, then so be it.

But remember this - if Crowdflower continues to lie and hide things from the workers, what are they hiding from YOU, the TAs?  And what lies are they telling YOU?  Can you really trust the data you are getting?  I'm thinking the answer to that is a big fat NEGATORY.

Think about it.

Seriously, Crowdflower?  SERIOUSLY?

Does someone at Crowdflower actually get paid to come up with schemes to make the workers helpless and protect the almighty TAs?

Hey, *I* can come up with stupidly insane ideas!  Can you put me on the payroll?

The latest asinine (oopsie, is that not nice?  tough) idea to come from The Crowdflower Powers That Be is that the workers can no longer see the corrections if they don't pass into work mode.

Yeah.

But wait.  Isn't quiz mode supposed to be for training?  So if we're not told what we did wrong, then where, exactly, is the training?

AND

It's been proven time and time again that the intelligence level and integrity of some of these TAs is, shall we say, lower than average.

AND

We know how often some of these TAs have the wrong answers in their test questions.

So maybe the TAs were getting tired of the workers contesting their bad answers?  TOO FREAKING BAD.  Then they can get their act together, shape up, or ship out.

We need a system of checks and balances here.  And this ain't it.

The Crowdflower Integrity Train just left the station to points unknown.

Hopefully common sense will prevail, and whoever thought this crap would fly will soon be on his way to the unemployment office.

Signed

Utterly Disgusted

***

Hey!  I TOLD you I could come up with stupid ideas, and I won't let you down!  How about this:  remove the ability to contest bad test questions!  Heck yeah!  I mean, you obviously worship at The Altar of the TAs and they can do no wrong, so why not just save everyone some time? 

That one is a freebie. I want to be paid for the next one.

***

 As a forum member so aptly puts it:  It seems it's just a change to hide the problems

Ask and ye shall receive

An announcement was made.  Humanity is not lost.

Thank you.

Peeps that were suspended recently are saying that this was their only source of income.  Why would they risk that by cheating?  If they DID cheat then they deserve to be suspended no matter what their financial situation is.

But on the other side of the coin, since this WAS their only source of income, then CF better be darned sure there really was cheating going on.  And even if there was, these are still human beings.  Make an announcement stating the reason and let these people have some closure.

So according to The Power That is She, the Ban Hammer really did hit yesterday, and it hit hard apparently!

Really glad I'm an honest and hard working little Worker Bee.  I don't always have to be watching over my shoulder for Mr. Hammer.  Well, I do worry a bit because I don't have 100% confidence that The Hammer doesn't miss its target from time to time, and I don't want to be collateral damage.

Shame on those that got caught.  There's no excuse.  Why ruin a (usually) good thing?

Back to work I go.

Hi ho Hi ho.

Cleanup in Aisle 7!

Is that what's going on?  Lots of reports of peeps being suspended today

If it's not a glitch, then I applaud CF's efforts on Cheat Patrol

Now if only they were as diligent in cleaning up Aisle 1

That would be the TA aisle

Did everybody from Crowdflower go on vacation?  Why aren't the ClixSense users being paid for the tasks they complete?  Noone's been paid in 2 days.  The lack of info is unacceptable!  Is it Crowdflower's fault?  Is it the fault of ClixSense?

WHAT IS GOING ON??

I'm gonna start handing out flags if this doesn't get resolved PRONTO!

ETA:  this situation seems to have been resolved. ThankYouVeryMuch

A horse is a horse of course of course

unless you're a TA, then it's a lion

Glad I have a screencap of that. 

I think the reason CF frowns on screencaps is because they show the whole world the sheer stupidity that the workers see on a daily basis

of course of course

from the forums today:

I've said it before I'm beginning to wonder where those Authors come from ,this is getting more ridiculous every day

You and I both, bud, you and I both

It's like someone's playing a cruel joke on the workers

Besides doing work for CrowdFlower, I also work for MicroWorkers.  While reading the MW facebook page it's like deja vu when it comes to complaints about employers:

bloody idiot employer, some of them have below average IQ

and I think to myself "I hear ya"

then I read of yet another broken task on CF where the correction goes something like this:

For the question titled "blah blah blah?" You have answered "Yes" and the correct answer is "Yes"

Proof positive that employers/TAs all come from the same special gene pool

With MicroWorkers the rejection/dissatisfied rating isn't permanent.  It'll disappear after a short while and a worker's rating will be restored.

CF has a heckuva lot of nerve giving individual TAs the power to speak for ALL TAs, and making a flag stick for life no matter how intelligence-challenged a TA may be.

And one more thing - MW didn't have a hissy fit over that remark the way CF would.  It seems that MW are well aware of what the workers have to deal with, whereas CF prefers to deny deny deny.

But hey - we did get pretty new colours on the blog.

Update:

Hold the phone!

So  "For the question titled "blah blah blah?" You have answered "Yes" and the correct answer is "Yes""  was from a CF Training Job?

Why am I not surprised.

Lemme guess

It's a Training Job to become a TA

That gene pool is bigger than I thought

It's almost scary

If a worker can be marred for life for messing up on one single task for one single TA, then the whole CF employee base should be sent packing and replaced with someone a wee bit more competent

Like a worker

You know, when you ask for worker's thoughts on specific tasks so they can maybe be passed along to the TAs, and you feel you have to tell the workers to be "kind and polite", that tells me that you KNOW that the workers have every reason to NOT be "kind and polite"

So why doesn't CF do something about it?  Like punish the TAs for a change.  The TAs ARE the cause of everyone's misery.  It doesn't take a rocket scientist to see that.

Interesting quotes from workers who are getting fed up with CF and their ridiculous flag system:

Something has to be done for contributors to regain their status if Task Authors can just ZAP us without any review or feedback or even consequences for Task Authors.

It seems that CF will penalize Contributors quicker than Task Authors.

If a TA doesn't want you to work on their tasks, there should be some way to exclude you - other than a ban! If there is absolute solid proof that one has cheated, then that's a different story. But this is way too harsh - how do you prove that you didn't do anything wrong? Very hard to prove a negative!

I don't think that bans should last forever! Just plain wrong.

it is not fair to work and maintain a high accuracy and some idiot flags you and you lose all your progress. 

Makes you wonder what CF's angle is.  It certainly is not one of integrity and fairness.


And you know what?  I think CF is AFRAID to allow workers to give ratings to individual TAs like on mTurk because they know there would be very few with a good rating.

Well this is interesting.  CF peeps have always feigned innocence when accusations were made of quiz mode being a deliberate attempt by the TAs to get free work done since quiz mode doesn't pay.

Then I read this:

CrowdFlower was (and probably still is, just not on the MTurk platform) a bunch of thieves. Not only did they pay garbage, but they bundled their HITs so they always got one free answer before workers got the "Sorry, but your answers weren't what we expected" message. A worker (much smarter than me) figured it out and started raising hell, right about the same time this lawsuit started moving forward.

 How come *I* didn't get an email about the lawsuit?  I did CF HITS on mTurk.  I feel left out.

Maybe I should start raising heck too.  If quiz  mode is going to take my time AND affect my stats, it should also pay me.

No free answers for YOU

Now I don't necessarily believe every word of that passage, but it DOES make me skeptical of CF's motives behind the non paying quiz mode

Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task Broken task

as another worker says:


Enough is enough!

Yeah, I know.  I complain too much about TAs and I haven't given much thought to the other side of the coin.

After reading this post on reddit, even though it's referring to mTurk, I can appreciate that it isn't all roses and sunshine for the TAs either.

That doesn't negate my frustration with the flag system and lazy TAs, but it might make me a little less ornery

maybe

Would someone please tell Mr. Look At Advertisements to check his test questions?  Seriously, he has to be one of the laziest TAs I've come across.

I really take issue with this.  It's inexcusable.

And don't bother with the "everyone else did fine with this task", cuz if they did, then it's because they remembered the bad test questions

I remembered them too, but I was optimistic that they would have been fixed by now.  Look where THAT got me.  Expelled City.

I would post a screencap but then I'm sure I would hear black helicopters flying over my house with someone blaring with a megaphone "delete that post, and STEP AWAY FROM THE KEYBOARD"

@CFComm you changed the colours on the blog!  Wow!

is the sentiment of that tweet positive or negative?

you can't answer that without knowing my frame of mind

you have no idea if I said:

"you changed the colours on the blog!  Wow!", while muttering to myself "you've gotta be freakin' kidding me!  the million and one things that need to be fixed and they waste time with pretty colours??"

or if I said:

"you changed the colours on the blog!  Wow!", while thinking to myself "good job!  they have such talented graphic artists!"

and THAT, boys and girls, is why sentiment tasks should NEVER be counted in GA

Otherwise, the TAs should take their work down the street to Psychics-R-Us where their motto is "We KNOW what you're thinking!"

Is the 6th time the charm?

Today there has been FIVE attempts by a TA to get a working task running.  The task is broken - the dropdown boxes don't work.  So it gets pulled, then reposted.  Still broken.  So it gets pulled, then reposted.  Still broken. So it gets pulled...well, you get the idea.

So again I ask.  Why in the name of all that is holy, would ANYONE in their right minds think that these intelligence challenged boneheads should have the power to flag the workers?  They don't even have the smarts to post a gosh danged working task!!

It's like putting Donald Trump in charge of a country

Oh, wait...

Typical response from CF to a plea for help regarding an unfair flag:

"The task author flagged your work after thoroughly reviewing your submissions, support is unable override this decision and am unable to lift the flag on your account as a result."

and this response from another incident:

"It is not appropriate to contact a TA directly"





MY response:

No, what's NOT appropriate is to put the TAs in a virtual witness protection program to shield them from the workers they screw over time and time again.  They need to take responsibility and own their actions.  If they can't take the heat over their carelessness and lack of integrity then they shouldn't be posting tasks in the first place.

To add to my previous post about bad test questions

for example:  What is 2 + 2

obviously the answer is 4

 but wait

"for the question "what is 2 + 2" you answered 4, but the correct answer is 9"

so this gets reported to the CF rep in the forums and quite often the response is "reported, will not affect your accuracy"

but every so often the response is "Most contributors did fine in this job"

Now put on your thinking cap.  Think real hard.  I know you can do it.


Maybe reports of bad test questions should be taken a bit more seriously even though it appears that there are contributors doing fine in a specific job.  That is, if CF is TRULY concerned about bad data

And they are, right?  RIGHT?

Another day, another batch of broken/unfair tasks that are 'cleaned up" by CF.

Yeah, these TAs are who *I* want in charge of handing out flags.  Yeppers.  They've proven themselves deserving of such a high level of responsibility.  They're totally right in not allowing mistakes - they've set a fine example of not making any themselves.

/sarcasm off

awww...really bummed that I didn't win the poetry contest - I thought my poem was great!

I'm reminded of a quote from an old movie:

You can't handle the truth!

Ok, seriously now  - I didn't actually enter my poetic creation.  Why?  Read the movie quote above ;-)

Does CrowdFlower ever get tired of cleaning up the TA's messes?  Today I counted not less than SEVEN broken/unfair tasks (confirmed by CF peep).

Yet these TAs are given the power to ruin a person's tasking "career".

They shouldn't be given the power to scratch their nose, let alone affect someone's income.

Clearly there's a problem here, and that problem ISN'T with the workers.  So WHY are the workers suffering?

With an overwhelming majority, the new task list design is a big fail.  At this moment the votes are 81% saying it sucks (paraphrasing).  Seriously, what was CrowdFlower thinking?  Were they trying to distract us from the REAL issues that need fixing?

Newsflash:  We're not that dumb

Whoever designed it should be flagged for poor quality work.

oh cool...my badges now have 6 sides - yeah, that was worth putting any kind of USEFUL changes on hold...yes indeedy

is CF freaking serious? 

this has to be absolutely THE WORST insult to the workers yet.  Don't fix the flag problem. Don't do anything about the constant broken tasks.  Don't do anything about the bad task authors.

No sirree bub.

Let's work on the important stuff like making the task listing page as ugly as sin.

Ooooohhh...I saw a tweet to CF from a TA today!

I feel my devil horns twitching...must resist...must. resist.

I need a distraction.  I think I'll go check out the new look of the task listings that was promised for today.

Oh wait...it appears they're still working on it

Finally got time to work on that post I mentioned last week.  It took me so long to get back to it because I was busy trying to figure out who Negan killed.  It better not be Glenn or Daryl or I will be:

a) extremely highly annoyed

Humanity

It's something that needs to be stressed.  The human factor.  I'm not talking about the scammers and cheaters.  They can take a long walk off a short pier.

I'm talking about the honest workers. Like me.  And from what has become apparent, Crowdflower has let the scammers and cheaters cloud their vision, and they don't see the rest of us. The "everyday" people.  The human beings who have lives, families, hopes, dreams.  The ones who would rather take a wrong correction for a RIGHT answer than give a WRONG answer that would be accepted.

Some time ago there was a letter writing campaign as part of the turker's We Are Dynamo project.  Turkers wrote to Amazon CEO Jeff Bezos to tell him a bit about themselves and why they were turking.  They did this to show their humanity, and that they are all different and they turk for different reasons.  They wanted to show that they aren't just some "algorithm" getting the work done.  They are human beings.  Real people that deserve to be treated with respect and be treated fairly.

As an honest Crowdflower worker I feel the same way. We are real people too.  We also deserve to be treated fairly and with respect.  And if only ONE thing gets accomplished with this blog, I hope it's this.

So I will continue with the good fight.  Some may not like what I have to say, but that only means that what I am saying is hitting close to home.

Busy completing some tasks with some, yeah, you guessed it - wrong corrections.

One of the upvotes said that other workers would screen cap the wrong corrections so they can give the wrong, but accepted, answer next time - meaning the TA is getting bad data.

Since CF doesn't seem to encourage the TAs to fix their bad test questions, does that not imply that CF shares in the blame for the bad data the TA gets?

I'd say it is about 80% the (lazy) TA's fault, and 20% the (indifferent) CF's fault.

As for me - I gave the wrong (but accepted) answer ONE time weeks ago, and have felt guilty ever since, so I won't do it again.

My reward as I continue to give the correct (but NOT accepted) answers will probably be expulsion, or worse, A Flag.

Doing the right thing sometimes doesn't pay.

You know what would be really awesome?  If TAs followed their own instructions.  That would be super duper.

And actually fixing their bad test questions would be a bonus.

You wonder why workers get testy sometimes.

This is why.

I just had to share this comment that I read this morning - the poster says that the quality of the tasks on Crowdflower has dropped considerably lately, and is wondering what has happened to Crowdflower.

I've been thinking that too.  It seems the "good guys" have left, perhaps to move on to a company that treats the workers like human beings?  heh

And now we're stuck with yahoos who can't/won't properly create a task; can't/won't verify their test questions; can't/won't correct their BAD test questions (just because an ad is vertical it doesn't make it different from the exact same ad that is horizontal) and who seem to be under the impression that a flag is a way to limit how many tasks a worker can do.

And chuckleheads who think the flag system is fair and are adamant that it is here to stay.

Have you ever heard the expression "same crap, different day"?

I've been working on another post, but it's taking me some time to put my thoughts together.  In the meantime I was reading a few message forums and saw this:

the flag system gives the Task Authors the power to be the Judge, Jury and Executioner

I've spent weeks writing on this blog, and this person sums up what I've been trying to get across in one sentence.

_____

And the wisdom continues:

 This isn't giving me a lot of motivation for working minijobs, it's more of creating atmosphere of fear of being flagged again, even though I am doing my best...

I'm beginning to think I was the wrong person to create this blog.  Others are coming up with much better ways to express the frustration

_____

You know what?  As someone posted on a message forum today,"It’s clear by now that the Task Authors don’t really understand the impact of a flag on contributor profile"

And I agree.

I'm reminded of some situations on mTurk where a requester would "block" a worker to prevent them from completing a survey more than once.  They didn't realize the impact of a block.  Fortunately, on mTurk a worker can contact a requester.  And it took the workers to let the requesters know the effect of blocks, and that the correct way to prevent a worker from doing a survey more than once was to give a qualification, and then only allow workers who did NOT have the qualification to do the survey.

Unfortunately, Crowdflower does not give the workers the right to contact the TAs.  And I have zero confidence that Crowdflower has made clear the impact of a flag, just as mTurk does not make it clear to the requesters the impact of a block.

One additional note - Microworkers just added a function that allows the workers to add an employer to an "ignore" list so we can eliminate his jobs from the list.  Also, a "dissatisfied" rating from an employer will lower a worker's percentage rate, but the bad rating will disappear in around 30 days, allowing the percentage rating to increase. It's not a life sentence as it is on Crowdflower.

Why is it that in some ways, Microworkers and mTurk are light years ahead of CF?  And why is CF so danged stubborn?

oooohhhh!  a poetry contest!

the things I could say, the rhymes I could make
but I think I'll decline for my account safety's sake

while I do get upset and I do want to scream
when I see the unfairness that is awfully extreme

sometimes I snarl, and at times want to swear
when CF is unhelpful and I wonder if they care

I try to be calm, I try to keep hope
and if CF remains stubborn, I'll just have to cope

so I'll keep doing tasks and will keep up the good work
and my fingers are crossed that I don't get a TA that's a jerk

the end

copyrighted by me Mar 31/2016

Read the new CF blog post today.  A little overdone on the being nicey nicey stuff, but what the hey.

After reading that, I decided to peruse the Terms of Service.

OMG peeps - the space bar is your friend!

Starting at #7 under the "You WILL NOT"

Things like that is like nails on a chalkboard!

a.please fix it

b. thank you

It left me wondering what the heck is fengage and ginsert

Am I:

a) a wee bit annoyed
b) slightly annoyed
c) a bit more annoyed
d) annoyed
e) more annoyed
f) even more annoyed
g) extremely annoyed



Just trying to illustrate the insanity of splitting hairs when it comes to those types of tasks.

If a TA really needs that kind of data, fine.  But we all come from different walks of life, have different frames of reference, different cultures.  These types of tasks should not count for Global Accuracy.

the correct answer was g) 

if you got it wrong, I'll flag you  

Me, after attempting a subjective task where I was expected to be a mind reader:

The flogging flagging will continue until morale improves

And morale is pretty low right now

If I didn't know better (but do I?) I'd be thinking that TAs get a discount directly related to the number of flags they hand out.

To quote another worker:  "is like crushing us contributors"

2016 - The Year of the Flag

No, I didn't come up with that - other workers did.  It really does give an indication of how the workers are feeling these days.

As for me, I'm leaning more towards

2016 - The Year of the Broken Promises

Prove me wrong

Just saw a story about Uber in my facebook feed today.

Now I'm sad all over again.

#IWantToAuditTickets

#You'reJustCruel

Can you give me one good reason why I should try a brand new task which could result in putting my levels at risk?  Sometimes I see a new task and think to myself "hmmm...that looks interesting".  But I've read too many horror stories lately, so I check the forums and sure enough I see posts like "xyz new task, major flag!!"  with many "me too" responses.  So then I think "whew, dodged a bullet there!"

I've had level 3 pretty much from the getgo when levels were first introduced.  I've never received a flag nor have I ever lost a level.  My accuracy is in the high 90s.

You'd think by now I would have established myself as a good and honest worker.  So why should I live in fear of one of Those Types of Task Authors?  If I give them the benefit of the doubt and complete their task and do end up with a flag, will I have to jump through hoops in getting it removed?  Why should I have to?  Why doesn't my history speak for itself?  And WHY would you allow a new and unproven TA to trump someone's excellent history?

Enquiring minds want to know

It's been a few weeks since my first post regarding unfair Crowdflower policies and unfair and downright crooked task authors.

Are we any closer to a resolution?  Sure, CF and their reps are making noises and asking opinions, and even posting a few tasks that ask for feedback from the workers, but are they just placating us by throwing water on the flames in the hopes that this will blow over?  Will things remain unchanged after giving empty promises and false hope?

I know CF has it tough when dealing with the scam workers.  I remember reading in the old CF forum of a worker complaining of being banned.  Turns out he had copied and pasted gibberish into text boxes in a task. His excuse?  He was in a rush and didn't have time to type in the correct answers.  After my eyes finished rolling right out of my head, I thought to myself that this moron should not only be banned from Crowdflower, but from the internet as well.  Seriously.  In what world would that excuse be ok?  Was he really stupid enough to think that CF would give him a second chance?

I'm sure this wasn't an isolated incident, and that CF has to deal with that kind of crap on a daily basis.  What concerns me is that good workers are being lumped in with the bad.  Flags are handed out like cookies, CF denies responsibility and the workers have no recourse and no way to contact the TA.  mTurk may have its faults (not paying Canadians in real money is one of them, but I digress, and that's a topic for another blog) but at least they offer a way to contact a requester.

While I don't have the answers, I do think that things need to change.  There needs to be better training for the TAs, tasks need to be double and triple checked before going live, there needs to be a mechanism in place to dispute flags, there needs to be stricter guidelines for handing out flags.  Really, is submitting a task 5 seconds before some Top Secret Time Limit actually worthy of a flag?

If a worker is deliberately trying to scam like the example given above, that worker should be banned permanently.  But any other type of flag should have an expiry date.  As I mentioned a few weeks ago, even a bank robber can get parole - he isn't in prison for life.  Why should CF workers get life without parole? 

They shouldn't.

I have an idea for a new task

Title:  Please Evaluate this Task Before It Goes Live

1)  Are the instructions thorough and easy to understand?

2)  Are the answers to the test questions correct?  If not, tell me which ones are incorrect.  What do you think are the right answers?

3)  Does the task submit properly?

Optional:  Would you recommend this task to other workers or would you tell them to avoid at all costs?

Would you jump into a snake pit if someone told you that the snakes MIGHT bite you, or they might not.?  And they MIGHT be poisonous and they might not.  And if a poisonous snake bites you, we MIGHT give you the antidote right away, or we might make you wait and wait and wait...or we just might not give it to you at all.

Sounds like fun, huh?

Sure would be helpful to have the ability to identify the biting snakes from the non biting ones, and the poisonous snakes from the non poisonous ones.

Is Crowdflower trying to pull a fast one with this recent development of double quiz mode or is it a glitch?  One can make a mistake on the first page but won't know WHAT got corrected and WHY, which is imperative if one is to learn how to do the task properly, especially when it isn't a true learning mode like yesteryear.  Then there's the possibility it was an unfair correction that needs to be disputed. Though that almost never happens.  /snark

I want to give them the benefit of the doubt but I'm not sure they've earned that yet.

-----

Got a real bad Uber-craving goin' on - throw in some wine and I'll be one happy camper

Evidently even CF created tasks aren't immune to bad test questions

good. grief.

If you CF Peeps were as strict with the TAs and their task creation abilities, or lack thereof/bad test questions as you are with the workers the Tasking World would be a much better place.

Wishing we could give flags to TAs, or at least give them a time out for bad behaviour.


P.S. I'm talking about the fraudulent "click on my banner" one, and the knucklehead whose corrections say "you answered xyz and the correct answer was xyz"

**all references to answers are fictitious and the names have been changed to protect the innocent.  (me)**

If Crowdflower is thinking of adding a new level in addition to Levels 1, 2 and 3, I have a suggestion - a level for the Certified Mind Reader.  So many tasks seem to be aimed at those that have that ability.

Really appreciative of the tasks you've been posting asking what features we would like to see, and what our favourite task is and what we want to tell the TA.  It's giving me a little bit of encouragement.

And now a request - on the job history page we can search by job id.  We'd have to keep our own record of IDs in order to use that.  How about adding a keyword search?   That would be more useful. ThankYouVeryMuch.

Per the CF rep in the forums, the higher level workers should be held to a higher standard and judged more strictly in their work.  Is this called CrowdLogic? 

We can only work with what we are given - broken tasks, bad TAs and bad test questions will not result in high quality work.

And even IF the tasks weren't broken, the TAs were fair and the test questions were correct, (I can dream) if a "lower level" worker and a "higher level" worker are each going to get 2 cents for doing the same task, why should the "higher level" worker knock themselves out for the same pay as the "lower level" worker?  Higher quality comes at a premium.

Nice double speak, btw.  Did you have a big pow-wow to try and come up with counter-arguments?  If you can't dazzle them with brilliance...

The flag system still needs to be fixed.  The TAs have too much power and some of them abuse it.

It should be mandatory for TAs to fix bad test questions if the task affects global accuracy.  If they don't want to be bothered, then the task should simply not count against the GA.  Though even if it doesn't, it is very disrespectful to not fix the bad questions because it would still affect our earnings should we be expelled unjustly.  However, that is more preferable than a hit to our GA.

Word on the street is that the workers are getting fed up.  There's talk of petitions, protests and boycotts.  Crowdflower's response will be a pretty good indicator of their true feelings about the workers.  I suspect I know what their (lack of) response will be, but it will be interesting to see what happens, if anything.

I predict some promises, like the one made almost one year ago about at least 1 guaranteed work mode after quiz mode.  We know how well THAT promise was kept.  I think some Crowdflower worker's nose grew 2 inches that day.

I want to clarify something that may have been misunderstood in my posts.  So far in all of the years I have been doing CF tasks, I have only had a few problems that were cleared up quickly.  I have never received a flag, but I live in fear of waking up one morning to see that my turn has come.

When I read in the forums that someone has had a flag since 2013 - that's THREE YEARS - I can't help but ask myself if I am next.  Even if this person's flag was warranted, this kind of punishment it not.  Some tasks are just difficult, some TAs are just plain jerks, and some TAs couldn't find their way out of a paper bag with 3 sides missing.

The flag system needs to go.  Admit that it was a failure and move on.  A life sentence for such trivial things should never happen.

There's ongoing problems with broken tasks and incorrect test questions that NEVER GET CORRECTED. That is where your focus should be right now.

--------

Now let's talk about "quiz mode".  It has been described as a "training tool" by a CF rep in the forums.  I disagree.  It's more of a "get these questions correct NOW or you'll be kicked to the curb" tool. No second chance to actually learn what the TA wants.  Learn it NOW or see ya later.  And the kicker - if work mode counts against your global accuracy, then so does quiz mode.  And it doesn't even pay! The things that make you say "huh??"  Why would I want to jeopardize my accuracy for something that doesn't even pay?!

I remember years ago when there was a true training tool.  You couldn't move on to work mode until you successfully completed training.  You could keep at it until you finally passed.  There were times when I didn't think I would EVER get out of training, and I stuck with it mostly out of stubbornness.  But I learned how to do the task!  Sometimes it takes awhile to figure out where the TA's head is at and what they are looking for.  Sometimes the instructions are so ambiguous that you don't know WHAT the TA wants.  But now you get one shot and one shot only.

WHY would you remove something that actually did train the workers and replace it with a one shot deal??

It's both heartbreaking and infuriating to wake up in the morning to read new stories of people who received an unfair flag and lost their badges.  And it's especially infuriating to read this response from a CF rep:

if a high enough percentage of people file a support ticket about a job, then you're automatically forgiven.

forgiven??  FORGIVEN??  Some SOB TA hands out an unfair flag because he doesn't know how to create a task properly, or the task is broken, or the worker's opinion isn't the same as the TA's, or because the TA has his own agenda going on, and it's the WORKER than needs to be forgiven??  REALLY??

and let's not overlook this gem:

if a high enough percentage of people file a support ticket about a job

incredible.

my response to that:  even ONE unfair flag is one too many

The flag system needs to be pulled immediately and either put out to pasture (my preference) or revamped.  It's just not working as it is.

Question:  Is it in the Task Author Handbook that "easy task, great pay!" is code for "scam task bwahahaha"?

----------

It appears that my blog is getting a little popular.  Not to worry, CF Peeps, I don't plan on being vindictive or rude or overly insulting.  Just telling it like it is from a worker's perspective.  Maybe you expected the workers to stay silent and accept the crumbs thrown their way, but enough is enough.  We're tired of the unfair flags. Heck, we're tired of the flags, period.  We're tired of task authors who can't be bothered to verify their test questions before posting the task.  We're tired of task authors who don't correct their bad questions after the task is live.  We're tired of task authors decreasing the amount of pay AS WE ARE WORKING ON THE TASK! If they want to reduce the pay, pull the task, change the pay, then re-post it with a new ID!!  Common courtesy!

We want to be respected.

So I'm doing some tasks, get a "correction" when the TA's answer is obviously wrong.  He must have been distracted when he set up that particular question.  So I challenged it and continued on.  The same question comes up again, I answer, and bam!  Another "correction".  The TA apparently can't be bothered to fix HIS mistake.

Sailing along quite nicely, and the SAME question comes up again.  This time a little devil appeared on my right shoulder whispering in my ear to give the TA the answer he wants even though it's the wrong answer.  I gave in to the temptation and....no "correction".   Me - 1;  TA - 0

And now I feel guilty.  Thanks alot ya chump!

Crowdflower, Task Authors, Oh My

I am a worker on Crowdflower, and if you are a Task Author, then I have a bone to pick with you!  Actually, several bones! 

Read on...

The biggest bone:  Flags.  The bane of the worker's existence.  Feared among the most diligent workers.  Why? Because when a task author gives a major flag, not only is the worker prevented from doing level 3 tasks for THAT task author, but he is also prevented from doing level 3 tasks for EVERY OTHER task author!

Incredible, isn't it?  I have no idea why some brainiac at Crowdflower ever thought that THAT was a great idea, but evidently SOMEONE did.

Were you aware of that?  Did you know that you held such power?  That if you had a problem with a worker, that it isn't just between you and that worker, but your actions also affects other task authors?  And there is rarely any turning back.  Once that major flag is given, it's lights out for the worker & level 3 tasks forever.  Even a bank robber can get time off for good behavior, but get a major flag on Crowdflower and it's there for life with no parole.  All because of a task that pays pennies.  To quote a Crowdflower employee who thinks pretty colors on the blog is more important than actually improving their system "flags are a fact of life".

Is Crowdflower sounding a bit foolish right now?  I hope so, because they are.  And YOU'RE not sounding all that great either, what with some of the downright silly reasons for some of these flags.  Giving you the benefit of the doubt, maybe you weren't fully aware of how these flags affect the workers, so when you hand out a major flag for trivial reasons such as submitting too fast, or taking too long to submit, or maybe you just woke up on the wrong side of the bed, you quite possibly had no idea that you were handing out a life sentence that would prevent that worker from doing level 3 tasks FOREVER for ALL task authors.  If I was one of the other task authors, I would be really ticked off that you were taking it upon yourself to prevent workers from working on my tasks.

Another bone:  Broken tasks and bad test questions.  Seriously, put a bit more effort into creating and verifying your tasks before you submit them - some of the most frustrating things when trying to work on tasks are the tasks that either won't submit because they're broken, images that don't load, or being "corrected" on a test question when YOUR answer is the one that is wrong. And making it worse is when you don't correct your answer!  Many times I have put too much faith in a task author, thinking surely he's corrected his error, only to have my answer marked wrong yet again.  (Mr. Look at Advertisements, I'm pointing at you!)

This is enough to make a worker want to scream!  AND it's enough to make a worker decide to give the answer he KNOWS is wrong, but it's the answer the task author THINKS is right, and as a result, the task author gets bad data.  And at this point, he deserves it.  Why would a worker want to continue to erode his accuracy level giving the actual right answer when the task author can't be bothered to correct it?

And submitting a ticket to Crowdflower is useless because they quote some ridiculous "40%" rule ("40% of the workers didn't submit a ticket so too bad so sad for you, there's nothing we will do").  That actually makes them partially responsible for the bad data you're getting.

Crowdflower is equally useless regarding the unscrupulous task authors that seemingly create tasks just so they can flag the workers knowing that those flags are usually permanent.  Unfortunately Crowdflower buries their heads in the sand and/or passes the buck when this gets reported, likely because they consider the workers to be expendable.  They just deny deny deny.

Hence the main reason for this rant.  The workers have noone on their side. Oh sure, there's a Crowdflower employee that visits the message forums claiming to want to help, asking for ideas to improve the task list.  This seems to be a yearly request that serves only to waste time and make the workers even more frustrated.  On any given day in a particular forum there is a long list of broken tasks being reported, which one would think would raise red flags that something is amiss when there are THAT many broken tasks, but sadly that isn't the case.  Sure, a few of the tasks get fixed, but this employee seems to need a narrated powerpoint presentation highlighting why the task is broken before she'll even consider taking action.  Describing the problem in plain English doesn't seem to sink in.

And there's more - but to me, besides the flag fiasco, one of the most important issues is the fact that the workers have no way to identify the task authors so that the bad ones can be avoided.  How utterly stupid.  They might as well say "here, do some tasks from this list. Some of the task authors will give you a major flag that will never be removed, just because they can. We won't tell you which ones, or how to avoid them.  Think of it as a game, a gambling game. Happy tasking!"

My point?  Since Crowdflower is all talk and no action when it comes to protecting the workers, I want you, the task authors, to step up to the plate for us. Surely you can't think it's fair for you to give flags that effectively ends any possibility of the worker ever doing level 3 tasks for ANYONE.  As responsible task authors, that HAS to make you uncomfortable!  Let Crowdflower know that that is not acceptable.  Tell them that while you have the right to prohibit a worker from working on YOUR tasks, that you shouldn't have the right to prohibit workers from working on the tasks of other task authors.

As for Crowdflower - where to begin...for starters, how about a little perspective - these are tasks for pennies!  PENNIES! This flag system you have going is like trying to kill a fly with a sledge hammer - it's overkill!

The flag system has to go - it SHOULDN'T be a fact of life!  Look, I know you have a problem with cheaters - I've seen first hand some of the things they try to pull - but you're catching too many innocent workers in your nets, and that is not acceptable!

And the passing the buck has to stop - there are good honest task workers who have received undeserved flags, and are pleading for help from your reps, only to be told "I have no control, I can't help".  Do you have any idea how discouraging and frustrating that is?  Here's a thought - if YOU can't help, then find someone who CAN! You need to come up with a better system to punish the proven cheaters that DOESN'T leave honest workers stranded, badgeless, and without hope.

I remember the days BEFORE badges - when tasks had a training mode that you could keep repeating until you finally passed and were able to move on to work mode.  We didn't have to worry about The Hammer coming down from abusive task authors (and YES, Ms. CF Rep, there ARE abusive TAs - you can deny it all you want but experienced workers know better).

I don't know what happened - maybe CF got a new CEO fresh out of military school who was determined to WHIP THOSE WORKERS INTO SHAPE!!  No excuses!  No mercy!!  No humanity for those peons!  We will show them WHO IS BOSS!!

Sounds ridiculous, doesn't it.  Welcome to OUR world.