Notifications

Notifications

Notifications are an underused aspect of the MTurk application. Notifications are an automated method to receive updates about a HITType while it is available to workers. This is useful for monitoring any of the following events:

• A Worker accepts a HIT
• A Worker abandons an assignment
• A Worker returns an assignment
• A Worker submits an assignment
• A HIT becomes "reviewable"
• A HIT expires

Note: Notifications are configured at the HITType level, so all HITs of a given HITType will trigger notifications once configured.

Setup Email Notifications

To setup a notification for a given HITType, simply create a HITType and then use GenerateNotification and SetHITTypeNotification to configure a notification for that HITType. The easiest way to receive notifications is via email. The following example shows how to receive an email any time a HIT of the HITType expires:

hittype <- 
RegisterHITType(title="Example 1",
                description="Example",
                reward=".20", 
                duration=seconds(hours=1), 
                keywords="example")

a <- GenerateNotification("[email protected]", event.type = "HITExpired")
SetHITTypeNotification(hit.type = hittype$HITTypeId, 
                       notification = a,
                       active = TRUE)

If you want to test this notification (i.e., see whether it is working as expected), it is possible to send a test notification by passing the notification object (returned by GenerateNotification) to SendTestEventNotification.

SendTestEventNotification(a, test.event.type="HITExpired")

This should trigger an email to the specified account, which looks something like:

Greetings from Amazon Mechanical Turk,

You are receiving this email because you subscribed to be notified when
certain events related to your HITs or Qualifications occurred.

Specific event information is shown below:

    Event Type: HITExpired
    Event Time: 2014-11-18T20:01:06Z
    HIT Type ID: 09876543210987654321
    HIT ID: 12345678901234567890
    Assignment ID: 1234567890123456789012345678901234567890

    Event Type: HITExpired
    Event Time: 2014-11-18T20:01:06Z
    HIT Type ID: 09876543210987654321
    HIT ID: 12345678901234567890
    Assignment ID: 1234567890123456789012345678900987654321


Sincerely,
Amazon Mechanical Turk
https://requestersandbox.mturk.com
410 Terry Avenue North
SEATTLE, WA 98109-5210 USA

Setup SQS Notifications

An alternative to email notifications uses the Amazon Simple Queue Service (SQS) to track notifications and make them retrievable later. While email notifications are convenient for small projects with a small number of notifications, larger projects might trigger dozens if not hundreds or thousands of notifications. At this scale, email becomes an inconvenient method of tracking notifications. SQS offers an alternative way of receiving notifications and, possibly, automatically processing those notifications.

Setting up SQS notifications is similar to email notifications. Indeed, the same functions are used, but there is a bit of additional work required to setup an SQS queue to receive those notifications. This tutorial from AWS provides some details.

Begin by visiting the SQS console. If it's your first time logging in, you'll see a screen like the following asking you to "Create New Queue":

A window will popup asking you to enter a name for the queue. Some additional options can be specified, but are safe to ignore. This will create a new queue that you can further configure. At the bottom of the new screen, select the "Permissions" tab and click the "Add a Permission" button:

A window will popup asking you to configure this permission. You need to grant MTurk permission to send notifications to this queue on your behalf. Specifically, you want to setup the following options:

• Set Effect to Allow
• Set Principal to arn:aws:iam::755651556756:user/MTurk-SQS
• Set Actions to SendMessage

Then click Add Conditions (optional) and setup the following details:

• Set Qualifier to None (the default)
• Set Condition to StringEquals (the default)
• Set Key to aws:SecureTransport
• Set Value to true

The result should be something like the following:

Click Add Permission and confirm on the popup window by clicking Yes, Add Condition.

Your SQS queue is now created. Note that you can send all notifications for all HITTypes to the same queue, or you can setup different queues for different HITTypes. This is up to you.

It is now possible to send notifications to this queue using MTurkR. The process works the same as sending email notifications, except you need to find the URL for your queue which you will specify instead of an email address in GenerateNotification. You can find this at the bottom of your main queue page:

hittype <- 
RegisterHITType(title="Example 1",
                description="Example",
                reward=".20", 
                duration=seconds(hours=1), 
                keywords="example")

b <- GenerateNotification("https://sqs.us-east-1.amazonaws.com/920667304251/MTurkR", "SQS", event.type = "HITExpired")
SetHITTypeNotification(hit.type = hittype$HITTypeId, 
                       notification = b,
                       active = TRUE)

You can then test this notification using SendTestEventNotification:

SendTestEventNotification(b, test.event.type="HITExpired")

You will then be able to see the resulting notification in your SQS queue: