Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

Amazon Updated Amazon Mechanical Turk Terms — What’s Changed

Discussion in 'AMT News' started by electrolyte, Oct 18, 2017.

  1. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    16,135
    Likes:
    38,238
    It's interesting how MTurk reached out to script creators and workers for input on the new worker site but as far as anyone has said, MTurk didn't reach out to anyone for input on these changes, which sound like they're potentially very major.
     
    Collapse Signature Expand Signature
    <Gucci> and Drwho10 like this.
  2. Drwho10

    Drwho10
    Expand Collapse
    Mister Gank Yo Hits

    • Contributor
    Joined:
    Jan 11, 2016
    Messages:
    12,346
    Likes:
    20,546
    Thats what makes me think these changes weren't written by the mturk admins or with any feedback. Just put in place by the legal team who never bothered to check. All speculation but highly likely in my opinion
     
    Collapse Signature Expand Signature
    It's Gamera! and <Gucci> like this.
  3. stuffalleysays

    stuffalleysays
    Expand Collapse
    Think of all the days I could've done summin more

    Joined:
    Sep 9, 2017
    Messages:
    1,222
    Likes:
    2,278
    Well, I guess until it is sorted I'll just have to look people up manually on turkopticon. Scripts have thus far been our only protections, due to the fact Amazon offers us none and can terminate without cause at any moment. Would be nice if they had a presence on the forum and encouraged some open dialogue.
     
    Collapse Signature Expand Signature
    scout likes this.
  4. stevenmturks

    stevenmturks
    Expand Collapse
    SAJ O.o

    • Contributor
    Joined:
    Aug 6, 2017
    Messages:
    3,832
    Likes:
    9,136
    Maybe one day Mturk will actually understand how Mturk works
     
    Collapse Signature Expand Signature
    scout, turker and <Gucci> like this.
  5. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    16,135
    Likes:
    38,238
    Hi. I come with information.

    I just talked to a director at MTurk on the phone who wanted to provide some reassurance and answers for us. Here are a few points that should help:
    • The new guidelines are not a new direction Amazon's taking. They're meant to be a clarification of the original TOS that was written almost 13 years ago, now with updated language acknowledging how workers work these days.
    • There will not be any mass suspension over people using scripts like they do currently. (I asked this specifically.)
    • People who will be suspended for script use are the same people who would have been suspended before. He told me directly: "they'll know what they're doing is wrong." It'll be people who are intentionally scamming, cheating, or doing things we all know are wrong.
    • I pointed out that some of the language is confusing, such as the parts that say using scripts to sort/search are okay but you can't be hitting the site too frequently. He acknowledged this and will get clarification on what "too frequently" means, but again repeated the above point about people who will be suspended will be those who who would have been before, who know they're doing the wrong thing.
    • I'm meeting him in person on Friday afternoon to get more information and ask more questions. After that meeting, I should have more detailed details to share.
     
    Collapse Signature Expand Signature
  6. catnapped

    catnapped
    Expand Collapse
    Well-Known Member

    • Contributor
    • Crowd Pleaser
    • HIT Poster
    Joined:
    Jan 13, 2016
    Messages:
    20,864
    Likes:
    42,888
    Right before you posted I was going to reply with what was said on FB (from TN--Kristy M?) which effectively said the same points--only the bad people really need to look out.
     
    scout, themildone, <Gucci> and 5 others like this.
  7. ecko

    ecko
    Expand Collapse
    Well-Known Member

    • Contributor
    • Mentor
    Joined:
    Feb 2, 2016
    Messages:
    2,475
    Likes:
    3,920
    Thank you so much for this.
     
    scout, electrolyte, <Gucci> and 2 others like this.
  8. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    16,135
    Likes:
    38,238
    Yup, he said he was calling her as well. :)
     
    Collapse Signature Expand Signature
    scout, themildone, <Gucci> and 2 others like this.
  9. Yatagarasu

    Yatagarasu
    Expand Collapse
    all up in the videos, all on the records, dancin'

    • Contributor
    • Crowd Pleaser
    • HIT Poster
    Joined:
    Jan 13, 2016
    Messages:
    9,406
    Likes:
    11,588
    Thank you for this update!
     
    Collapse Signature Expand Signature
    scout, electrolyte, <Gucci> and 4 others like this.
  10. YAHU

    YAHU
    Expand Collapse
    YAHUUUUUUU

    • Contributor
    Joined:
    Jan 11, 2016
    Messages:
    8,015
    Likes:
    26,402
    tell him I said hi
     
    Collapse Signature Expand Signature
    scrmcasey, kryss, electrolyte and 5 others like this.
  11. Sciency McScienceface

    Sciency McScienceface
    Expand Collapse
    Knowledge is both lock & key

    Joined:
    May 1, 2016
    Messages:
    838
    Likes:
    2,246
    Here's a scenario that illustrates just the type of worker that these new policy changes apply to:

    1. Malicious worker decides they want to easily make $300-$500/day the dishonest way.
    2. Worker makes a script to scrape batches of HITs (e.g. Barcodes) shortly after they drop. This is done while logged-out. The script opens each HIT in the batch, extracts the HIT ID and the UPC and saves them, and then skips to the next HIT. This process generates a ton of page calls to Mturk.com (10's-100's per minute) The whole batch can be scraped within minutes of it dropping.
    3. All UPCs and corresponding MTurk HIT IDs are stored offline in a custom offline database, and the UPCs are automatically searched via another script against existing online UPC databases. UPCs that result in no products being found are flagged. The UPC search results from the online databases are automatically saved to the workers offline database (this takes <10 minutes)
    4. Dishonest worker now logs into MTurk, and runs another script that previews each Barcode HIT in the batch and checks each against their custom database. Once a HITs ID containing a UPC that corresponded to no product is encountered, the script automatically accepts the HIT, checks the box for "No Product Found", and then auto-submits the HIT. Script moves on to find the next HIT, and repeats the process until the batch is exhausted.

    Such a malicious worker would violate all three of these 3 key policy points:
    1. High frequency page calls
    2. Extracting & storing data
    3. Substituting their human judgement for letting a script automatically search for and input the information into the HIT.

    To reiterate what was recently discussed above, this is VERY different from how the average Turker uses scripts and does not apply to something like HitDB
     
    Collapse Signature Expand Signature
    #31 Sciency McScienceface, Oct 18, 2017
    Last edited: Oct 18, 2017
    Pacis Amans, Steez, scout and 7 others like this.
  12. catnapped

    catnapped
    Expand Collapse
    Well-Known Member

    • Contributor
    • Crowd Pleaser
    • HIT Poster
    Joined:
    Jan 13, 2016
    Messages:
    20,864
    Likes:
    42,888
    And tell him a certain somebody really needs a rejection. :whistle2:
     
    jklmnop, Jaded, <Gucci> and 1 other person like this.
  13. YAHU

    YAHU
    Expand Collapse
    YAHUUUUUUU

    • Contributor
    Joined:
    Jan 11, 2016
    Messages:
    8,015
    Likes:
    26,402
    :ave:
     
    Collapse Signature Expand Signature
    DCI, Jaded and <Gucci> like this.
  14. catnapped

    catnapped
    Expand Collapse
    Well-Known Member

    • Contributor
    • Crowd Pleaser
    • HIT Poster
    Joined:
    Jan 13, 2016
    Messages:
    20,864
    Likes:
    42,888
    The one problem with your theory is those batches only last 20-30 minute nowadays. How's he going to make any worthwhile money?

    Hard enough doing it the legitimate way nowadays with it being nothing more than a typical overhyped batch
     
    Jaded and <Gucci> like this.
  15. Sciency McScienceface

    Sciency McScienceface
    Expand Collapse
    Knowledge is both lock & key

    Joined:
    May 1, 2016
    Messages:
    838
    Likes:
    2,246
    My coworker demonstrated a similar script using one of the HITs which require supplying the direct phone, email, and website of academic researchers listed as authors of research papers. His script scraped the whole batch, queried SciFinder and some other sciencey databases, and connected all of the requested info with the info supplied in the HIT.

    It took him a whopping 20 minutes to write the script and validate it. He then scraped 100% of the information from the 2500 or so HITs (0.25 or 0.30 each) in the batch in less than 1 hour. He didn't actually submit any of the HITs because he knew it would have been a violation so this was just him proving to me how awesome he is...

    Anyway, had he actually pulled the trigger and ran the script he would have made about $600 in about 2 hours. Countless batches that he could have done the same to... Of course he'd eventually get his account suspended.

    However, he has a PhD in computer science, so his typical "side jobs" net him 10x-100x what any of us make Turking.
     
    Collapse Signature Expand Signature
    #35 Sciency McScienceface, Oct 18, 2017
    Last edited: Oct 18, 2017
    Pacis Amans, Steez, scout and 5 others like this.
  16. stuffalleysays

    stuffalleysays
    Expand Collapse
    Think of all the days I could've done summin more

    Joined:
    Sep 9, 2017
    Messages:
    1,222
    Likes:
    2,278
    Thank you to all who sought clarification and provided that information for us.
     
    Collapse Signature Expand Signature
    scout, booboo33, TaiGir and 3 others like this.
  17. catnapped

    catnapped
    Expand Collapse
    Well-Known Member

    • Contributor
    • Crowd Pleaser
    • HIT Poster
    Joined:
    Jan 13, 2016
    Messages:
    20,864
    Likes:
    42,888
    I totally agree what you said is possible (likely even being done)...more the matter of being able to grab enough hits through the piranhas/sharks and the throttling to make it worth it.
     
  18. Jharkan

    Jharkan
    Expand Collapse
    Internets EXPERT

    • Contributor
    Joined:
    Jul 9, 2016
    Messages:
    8,675
    Likes:
    14,174
    are you going to take any selfies with him/her?
     
    Collapse Signature Expand Signature
    Pacis Amans, THFYM and squee like this.
  19. Achilles2357

    Achilles2357
    Expand Collapse
    Member

    Joined:
    May 24, 2017
    Messages:
    486
    Likes:
    727
    For various reasons I haven't done much turking in the past week, and now I check in here and panic!
    But... after a few minutes of thought, I'm thinking this might be much ado about nothing.
    I think it is now fairly obvious to me that there is no way in Hell that Amazon would actually systematically punish things like pounding the mturk server or storing data, since I am certain that at least some people who work for mturk know that the very viability of mturk hinges on these things. But for a range of legal and commercial reasons Amazon cannot ever acknowledge what it actually knows about the "mturk worker" world. I suspect this was written by lawyers who don't know much, and may be meant as a way to justify punishment of obvious wrongdoers. I suspect that people who are completing work to the satisfaction of requesters, and thus ensuring that Amazon keeps getting paid, have little to fear. But who knows? Mturk is very, very, very strange....
     
    Collapse Signature Expand Signature
    scout likes this.
  20. Drwho10

    Drwho10
    Expand Collapse
    Mister Gank Yo Hits

    • Contributor
    Joined:
    Jan 11, 2016
    Messages:
    12,346
    Likes:
    20,546
    Thanks for putting in the leg work as usual :thumbup:
     
    Collapse Signature Expand Signature
    scout and electrolyte like this.