MTurkification Web and API Update

Project Overview

Our client want to use Amazon mturk to invite game players to play set of games. Each Human Intelligence Task (HIT) will be for completing a "task" (analogous to a subset of levels in the game versions) in exchange for payment. Ideally, we would keep this as short and simple as possible - i.e. one "task" gets the payment (instead of multiple progressive levels).

In this challenge we are making several changes and update to the mturk web dashboard, and mturk api to implement the FULL flow from creating a hit until the assignment review is submitted.

Challenge Requirements

You will address the following in this challenge :

• Creating a hit - We need to change the creation to support creation of hit using ExternalQuestion
  • We need to store the form as template to be used by all hits created that use External Questions.
    • This is a sample of javascript-only of External Qustion html.
    • The template should be modified to have following in the form :
      • Please open this link in a new page/tab to play the game (add a link with target="_blank"), let's call the link : playLink
      • Read and follow instructions there
      • After completing the task, you will receive a confirmation code. Pasteit here [______________] (note to developer : this to be submitted to mturk result)
      • hidden fields contains : assignmentId, taskId (note to developer : to be submitted to mturk result)
    • Update the sample to address the following :
      • Write a javascript that read assignmentID, workerID, and taskID from the ExternalURL query string. Call generateToken endpoint to generate a unique Token and append it to the playLink as a query string parameter with parameter name "token"
      • If assignemntID is ASSIGNMENT_ID_NOT_AVAILABLE as outlined in the ExternalQuestion documentation then don't generate/call the generate unique token endpoint. Don't display link, display ""<p><b>You are previewing this HIT.</b> To view play link of this HIT, please accept it.</p>";
  • In mturk admin web dashboard add radio button groups so user select between using External Question vs. Regular Question. The question area should be disabled if user select External Question option.
    • The external quesiton html form will be stored in Amazon S3. But for flexibilty read the external question form link from configuration and send it to mturk when creating the hit.
  • Update preview in mturk admin web dashboard to render the external question form.
• Create public endpoint /generateToken that accepts assignmentID, workerID and taskID to generate unique token.
  • Endpoint should validate assigmentID exists, and workerID passed is associated with assignment.
  • use crypto.js generate a unique random token code with length of 10 characters.
  • Store the generated unique token in database in new table called worker_tokens with assignmentID, workerID, and taskID.
  • Return token to caller, If error occurred, return error without exposing any stacktrace or critical info.
• When worker view the hit, accept it, and click the link, It will navigate user to the link to play the page.
  • Simulate this process by creating a page that parse the token, and validate it exists.
  • If exists give user random confirmation code, and store the confirmation code in the worker_tokens table. Add new field for that.
  • If not exists, give user friendly message "Not assignment found for your request"
  • worker will fill confirmation code in the hit and submit it.
• Assignments mturk admin web dashboard :
  • Add new action button in hits table in dashboard page to 'view assignments'
  • 'view assignments' page will include a table of assignments with brief info about it.
  • the 'view assignments' table should link to 'assignment details' page where user can view the assignment details.
    • The page should render the response from mturk properly.
  • 'assignment details'  page should have a form to submit (accept/reject) assignment and provide feedback (optional).
    • The page should also retrieve the corresponding field from worker_tokens and display if the assignment is valid or not depends on whether the worker_tokens's record values match the one returned from the assignment answer.
  • You will need to add required endpoints to mturk api that are missing and needed to address this flow.

Testing

• You will use mturk sandbox for testing.
• You can view created hit by navigation to worker sandbox search for the title of the hit.
• You will need to accept it and manually navigate to the website to simulate the player flow.
• Make sure you can complete the outlined flow above as described. Reviewers are expected to do the same.

General Notes

• Add audit fields to new table.

Github Repository

API exists in private repository https://github.com/topcoderinc/csfv_mturk_api request access in challenge forums if you don't have it already.

The mturk admin web dashboard is attached in challenge forums.

References

• Mturk Amazon API Documentation
• Sample External Question API

Final Submission Guidelines