Guide to: The Instruction Editor

In this article, you will learn how to create accurate instructions for the best contributor experience and quality data collection.

In some cases embedding different media can allow you to save space as well as show processes more effectively than words can.. If your instructions are very long and need to contain a lot of detail, it can be a good idea to upload or provide a link to a hosted pdf. See this article to learn about embedding various media into your instructions.

Instruction Editor Sections

Our instructions template (automatically included in your job) includes the following sections:

• Overview
• Steps
• Rules & Tips
• Examples

In general, instructions should comprehensively outline the steps necessary to complete the job, and provide useful tips and examples.

We will use an actual example of instructions from a sentiment analysis job to demonstrate how your instructions should be structured.

Fig. 1: Instructions Template

Fig. 2: Preview of job examples

Overview

The overview should give a clear high-level picture of what a contributor will be doing. This can be brief with just a few sentences.

The example above explains the overall goal this job and its final purpose. The final purpose should describe what product or work this task will contribute to. 

• Ex: "Your contributions will help an airline company improve customer experience and build a comprehensive chatbot for customer service requests" 

Be sure to also include any risks with performing the task in this section as well. The crowd should be aware if there is a potential to come into contact with harmful and/or offensive content.

• Ex: "Some of the Tweets that you are rating may be explicit in nature"

Steps

Describe the step by step process contributors will take to complete the task. Try not to jump around when explaining the steps as it may cause confusion.

In the example above, each of the steps reflects an action the contributor will need to take. In this use case, the steps show contributors how to determine if the sentiment of the tweet toward American Airlines is positive, neutral or negative.

Rules and Tips

Provide rules to contributors on what they should and should not do. Explaining the way an answer was determined can help teach them how to provide the best response.

In the example above, this section provides additional definitions and text examples of how to determine if a tweet is positive, neutral, or negative. This section helps contributors understand the differences between each type of sentiment asked for.

Examples

Providing real examples from the source data, can help contributors gain a better understanding of what they will be doing. This is especially important as contributors will get a glimpse of the type of data they will be working with and how to provide quality answers.

As seen above, there are examples of a positive, neutral, or negative tweet and a definition of why the tweet is placed in that category.

Thank you!

Our contributors really appreciate a customized thank you note! It only needs to be one or two sentences.

Best practices for writing instructions:

• Simplify your wording as much as possible, this will help contributors to understand and complete your task.
• Create an easy, repeatable step by step process as a guide for contributors to understand how to provide answers.
• Provide at least three examples -- it can be good to include counterexamples so contributors also understand what not to do, as well as edge cases to show reasoning around how ambiguities can be resolved.
• Remember to provide all relevant definitions in your instructions.
• Preview your instructions before launch to ensure they appear as expected.

The commented HTML code for the instructions template can be found as an attachment to this article. We highly recommend using them to get the best results from our on-demand workforce.

Also attached is a commented Markdown version of the instructions template. This should be used  for those that do not want to work in HTML.