Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

Rep Present New Requester Veridium ID - HIT Posted

Discussion in 'Requesters' started by Veridium ID, Apr 3, 2018.

  1. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0
    Veridium ID Data Collection - mturk project

    Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
    Keywords: data collection, mobile phone
    Number of Assignments per HIT: 5
    Reward per Assignment: $0.10
    Number of HITS: 200

    Average Time per Assignment: 35 seconds
     
  2. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Contributor
    Joined:
    Jan 10, 2016
    Messages:
    19,184
    Likes:
    45,927
    Title: Data Collection using mobile phone and a custom application | AcceptA
    Requester: Veridium ID [A22LGXWTD0S4ZK] Contact
    TV: No Reviews
    TO: No Reviews
    TO2: No Reviews
    Reward: $0.10
    Duration: 5 minutes
    Available: 200
    Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
    Requirements: None
     
    Collapse Signature Expand Signature
    Roscoe E Goldchain likes this.
  3. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Contributor
    Joined:
    Jan 10, 2016
    Messages:
    19,184
    Likes:
    45,927
    Reading through your instructions, it says: "If you decide to quit before completing all 200 HITs, you will not receive any partial payments. In this case, the collected data will be discarded."

    Does this mean if someone isn't able to get 200 HITs, then you will reject the HITs that they have already completed?
     
    Collapse Signature Expand Signature
  4. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0
    If workers do more than 180 HITs, then they will get payed. The experiment needs a minimum number of HITs from a worker to be able to do the research project thus we put as mandatory request so that workers try to get to the end of the HITs.
     
  5. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Contributor
    Joined:
    Jan 10, 2016
    Messages:
    19,184
    Likes:
    45,927
    Hmm, that could be a problem. Workers can't "reserve" 180 HITs for themselves, so it's quite likely that by the time one worker gets around to doing a large number, there could just not be that many HITs left to do because they've already been taken by other workers.
     
    Collapse Signature Expand Signature
  6. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0

    We have developped a mechanism which counts the number of workers and also we track how many sessions are done for each worker. So, by adding their worker id, we reserve the 200 hits for that worker.
     
  7. largeeyes

    largeeyes
    Expand Collapse
    Active Member

    Joined:
    Jan 12, 2016
    Messages:
    899
    Likes:
    1,429
    So, someone does 179 and then you give them 179 rejections? That can destroy a worker's account. I'm no engineer(actually, I am), but how can 180-200 be such a specifically significant sample size?
     
  8. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0
    If there are 179 HITs we will pay the worker, it is an extreme example. We want to emphasize the fact that a worker should do all 200 to be relevant for us. From what we see as comments, I think the bonus approach is better.

    The data sample acquired will be used for a machine learning flow and through the MTurk platform we gather more than the required samples, just to be sure that we reach the required sample size. The extra sample size is used for testing and also to simulate some real life scenarios.
     
  9. largeeyes

    largeeyes
    Expand Collapse
    Active Member

    Joined:
    Jan 12, 2016
    Messages:
    899
    Likes:
    1,429
    The bonus system is absolutely better. This way you can still pay if they don't complete them all, but also provide great pay if they do.
     
  10. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0
    Title: Veridium ID Data Collection
    Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
    Keywords: data collection, mobile phone
    Qualification Requirement:

    Number of Assignments per HIT: 1
    Reward per Assignment: $15.00
    Assignment duration: 2 hours
    Approximate duration: 1 and 1/4 hours
    Auto Approval Delay: 7 days

    OBSERVATIONS:
    1. Interface modified for a better user experience.
    2. Hit mechanism modified - Submit button is available only after 200 Authentications, thus we reserve the worker the HIT he is doing.
    3. Worker duplicate - by design, a worker can do only 200 Authentications. If he accepts another HIT, he will not be able to do another 200 Authentications. This way, we assure the uniqueness of the workers.
    4. HIT Time - from our experiments, the HIT which consists in 200 Authentications, lasts approximately 1 hour and 15 minutes. We have set it 2 hours for convenience.
     
  11. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0
    Title: Veridium ID Data Collection
    Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
    Keywords: data collection, mobile phone
    Qualification Requirement:

    Number of Assignments per HIT: 1
    Reward per Assignment: $18.00
    Assignment duration: 3 hours
    Approximate duration: 2
    Auto Approval Delay: 7 days
     
  12. A6_Foul_Out

    A6_Foul_Out
    Expand Collapse
    TV Reviewing Meme Panda

    • HIT Poster
    • Contributor
    Joined:
    Aug 17, 2016
    Messages:
    11,254
    Likes:
    20,991
     
    Collapse Signature Expand Signature
  13. DareAngel3

    DareAngel3
    Expand Collapse
    Well-Known Member

    • Contributor
    Joined:
    Jan 12, 2016
    Messages:
    1,695
    Likes:
    4,261
    @Veridium ID@Veridium ID Hello! I just completed this HIT and had some polite constructive criticism. I tried contacting you via mturk, a response to the registration email, and via the email you provided in the HIT for support, but I never got a response.
    My HIT started glitching about halfway through and would not progress. I finally figured out that I could restart the iframe/Mturk window to make it continue, but it continued to glitch throughout the experiment. I struggled to finish within the 3 hours. I would have been really frustrated if I could not submit after putting 2+ hours in, due to a glitch. The way the HIT is designed, I couldn't submit even if my timer ran out at 249/250 completed.
    Also, the camera seemed to be completely random as to when it would fail or complete. Multiple times I took seemingly identical images, and some were accepted and others were not. I'm assuming this is what you're testing. However, you may consider increasing the timer so that it allots for both technical errors and "failed" captures, and gives enough time for workers to contact you and for you to respond.
    Thanks, and I hope you get some really useful data!
     
    electrolyte likes this.
  14. Veridium ID

    Veridium ID
    Expand Collapse
    New Member

    • Requester
    Joined:
    Apr 3, 2018
    Messages:
    7
    Likes:
    0

    Hi DareAngel3,

    Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

    As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

    As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

    The last request that we have, can you give us a private e-mail with the model of the phone?
     
  15. DareAngel3

    DareAngel3
    Expand Collapse
    Well-Known Member

    • Contributor
    Joined:
    Jan 12, 2016
    Messages:
    1,695
    Likes:
    4,261
    I sent a message to your account here on the forum with my model number. Thanks for your response!
     
  16. Drex

    Drex
    Expand Collapse
    New Member

    Joined:
    Jan 12, 2016
    Messages:
    4
    Likes:
    0
    EDIT: for some reason it did not quote him
     
    #16 Drex, Apr 18, 2018
    Last edited: Apr 18, 2018
  17. Drex

    Drex
    Expand Collapse
    New Member

    Joined:
    Jan 12, 2016
    Messages:
    4
    Likes:
    0
    Where are we supposed to send the email to, because I sent a few yesterday and have yet to receive a response. I used the email that was listed in the hit. My problem was that it got to 202 sessions and would not advance past 202 no matter how many additional sessions that I completed. I sent an email as soon as I encountered the problem which occurred at about 2 hours and 10 minutes into the hit. I sent another a few minutes later with a screenshot of the problem and then sent one about 40 minutes later when the hit was about to expire. I had enough time to finish the hit, but it would not let me advance. I have no problem completing the additional sessions, but you have to understand how frustrating working on a hit for 2+ hours and it all of the sudden just stops working is for a worker and then to not get any responses to your emails and having no idea if you are going to be paid for your work is.
     
    #17 Drex, Apr 18, 2018
    Last edited: Apr 18, 2018
  18. InfiniteChanges

    InfiniteChanges
    Expand Collapse
    Hiding In Your Ceiling

    • Contributor
    Joined:
    May 1, 2017
    Messages:
    3,426
    Likes:
    8,150
    Just wondering, will you be opening up HITs to workers who have already completed on of these tasks for you?
     
    Collapse Signature Expand Signature
  19. Drex

    Drex
    Expand Collapse
    New Member

    Joined:
    Jan 12, 2016
    Messages:
    4
    Likes:
    0
    I have currently sent 7 emails and have yet to receive a response. I sent three the day it happened including a screen shot, I sent one last Friday and another this past Monday all to the email listed in the hit. I have sent one today to a company email that another Turker who did get paid said they replied from and I sent another to the Turk address associated with the account. I did not expect replies to the ones that I sent today yet, but in case the requester posts tomorrow I want them to see this. I know that people got emails and hits setup yesterday for compensation, I was not one of them, I cant even get them to respond to an email.
     
  20. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Contributor
    Joined:
    Jan 10, 2016
    Messages:
    19,184
    Likes:
    45,927
    Tagging @Veridium ID@Veridium ID to come hopefully respond to the questions in this thread. (Perhaps they're not receiving notifications about thread replies.)
     
    Collapse Signature Expand Signature
    Drex likes this.