Examples

Note

The first line of your main file should be:

>> import crowdlib as cl, crowdlib_settings

Minimal complete example

import crowdlib as cl, crowdlib_settings, time
hit_type = cl.create_hit_type("Who invented this?", "Answer a simple question.")
hit = hit_type.create_hit([cl.text_field("Who invented the umbrella?", "q1")])
while hit.is_available:
   time.sleep(10) # poll every 10 seconds
print( tuple(hit.assignments)[0]["q1"] )

Change to production mode

Put this at the top of your main file.

cl.settings.service_type = "production"

Post a HIT using question fields

hit_type = cl.create_hit_type(
  title       = "Find the inventor of each product.",
  description = "Search the web for an answer to a simple question.",
  reward      = 0.10
)

hit_type.create_hit([cl.text_field("Who invented the umbrella?", "q1")])

Post a HIT using an external site

This reuses the declaration of hit_type from above.

hit = hit_type.create_hit(
  url = "http://…/ui.cgi", 
  height = 800
)

This URL should be pointed at your own web page, which will be displayed in an IFRAME within Mechanical Turk. AMT will add a few parameters to that URL: assignmentId, hitId, turkSubmitTo and workerId. Thus, your web page will be called with something like this:

http://…/ui.cgi?assignmentId=AAA123EXAMPLE
               &hitId=HHH123EXAMPLE
               &turkSubmitTo=http%3A%2F%2Fwww.mturk.com
               &workerId=WWW123EXAMPLE

Your page should contain a form that submits the worker’s answers back to either http://www.mturk.com/mturk/externalSubmit or http://workersandbox.mturk.com/mturk/externalSubmit, depending on the value of the turkSubmitTo URL parameter it receives. The assignmentId URL parameter should be returned with the answers, most likely in an <input type="hidden" name="assignmentId" value="…"> element. The hitID and workerId parameters do not need to be passed back to the server, and will be ignored if you do. The rest of the form fields will become the question ID when you fetch the results.

The web page returned by your server might look like this:

<html>
<body>
<form method="get" action="https://www.mturk.com/mturk/externalSubmit">
  <p>
    Who invented the umbrella?<br>
    <input type="text" name="q1">
  </p>
  <div>
    <input type="hidden" name="assignmentId" value="AAA123EXAMPLE"/>
    <input type="submit" value="Submit"/>
  </div>
</form>
</body>
</html>

Post a HIT using XML

This reuses the declaration of hit_type from above.

hit = hit_type.create_hit(xml="""
<QuestionForm xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">
  <Question>
    <QuestionIdentifier>q1</QuestionIdentifier>
    <QuestionContent>
      <Text>
        Who invented the umbrella?
      </Text>
    </QuestionContent>
    <AnswerSpecification>
      <FreeTextAnswer/>
    </AnswerSpecification>
  </Question>
</QuestionForm>
""")

For more details on the QuestionForm XML, including a richer example, see QuestionForm in the AMT API documentation.

Post a HIT using unrestricted HTML (hosted by AMT)

This reuses the declaration of hit_type from above.

hit = hit_type.create_hit(
  html = '…', 
  height = 800
)

The contents of the html parameter will be an HTML document. It is still displayed inside an IFRAME, but Amazon hosts it for you. This works just like using an external URL, except that Amazon hosts the content, and you’re only allowed a single, static HTML page. You can of course include any JavaScript or CSS that you like in that HTML page. In this case, JavaScript is required in order to parse out the assignmentId parameter.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">

  <title>Who invented this?</title>

  <script type="text/javascript">
  window.onload = function() {
    // Get the Assignment ID, which will be added to your URL by Mechanical Turk.
    var assignment_id = location.search.match(/assignmentId=(\w+)/)[1];
    document.getElementById("assignment_id").value = assignment_id;

    // Assignment ID "ASSIGNMENT_ID_NOT_AVAILABLE" of indicates preview mode.  Warn worker.
    if( assignment_id == "ASSIGNMENT_ID_NOT_AVAILABLE" ) {
      document.getElementById("submit_button").disabled = true;
      document.getElementById("click_accept_warning").style.display = "block";
    }

    // Get the Submit URL, which will be added to your URL by Mechanical Turk. */
    var submit_to_url_base = (location.search.match(/turkSubmitTo=([^=&]+)/)||[])[1];
    if(submit_to_url_base) {
      document.getElementById("navform").action = submit_to_url_base + "/mturk/externalSubmit";
    }
  }
  </script>

</head>
<body onload="return onBodyLoad();">
  <form method="get" id="form" action="javascript:void(0)">

    <!-- Banner at the top to warn workers if this is preview mode;  hidden otherwise -->
    <h2 style="color:red; display:none" id="click_accept_warning">PREVIEW MODE</h2>

    Who invented the umbrella?
    <input type="text" name="q1">

    <!-- Pass back the assignment ID, as required by AMT -->
    <input type="hidden" id="assignment_id" name="assignmentId" value=""/>
    <input type="submit" id="submit_button" value="Submit" />

  </form>
</body>
</html>

For more details on the QuestionForm XML, including a richer example, see QuestionForm in the AMT API documentation.

Watch status of ongoing work

hit_type.poll()

Fetch all assignments for a HITType and print to console

for asg in hit_type.assignments:
  fields = (asg.worker.id, asg.submit_time, asg["q1"], asg["q2"], asg.id, asg.hit.id)
  line = "\t".join(str(f) for f in fields)
  print(line)

Note

Assignments may be treated as dicts keyed by question ID. For example, asg["q1"] returns the crowdlib.Answer (as an object) for the crowdlib.Question with id "q1".

Fetch all assignments for a HITType and save as an Excel-friendly report

report = cl.assignment_report(hit_type.assignments)
cl.write_file("results.txt", report)

Review (approve / reject) assignments

for asg in hit_type.assignments:
   if asg["q1"].text != "":
      asg.reject("Answer was blank")
   else:
      asg.approve("Good!")

Use an agreement as a qualification requirement

This may be especially useful for satisfying university IRB requirements.

irb_qual_req = cl.create_agreement_requirement(
   name         = "Agree to Informed Consent (IRB#12345)",
   description  = "Agree to a consent form for a university research project.",
   xhtml        = "This HIT is for <b>research</b>.",
   agree_text   = "I agree.",
   keywords     = ("UMD","informed consent", "12345")
)

hit_type = cl.create_hit_html(
   name = "…",
   description = "…",
   qualification_requirements = [irb_qual_req]
)

Send a worker a bonus

asg.grant_bonus(0.25, "Good job!")

Send a worker an email

asg.worker.send_message(
   subject="Thanks...",
   message="Dear worker,\nThanks for the great work!\nSincerely,\n…"
)

Reject all work from a particular worker

for asg in worker.assignments:
    asg.reject("You didn’t follow directions.")

Cancel all HITs

Use this if you make a mistake and need to stop everything as quickly as possible.

cl.set_all_hits_unavailable()

Cancel a single HIT

hit.is_available = False