In this article, you will learn how to create accurate instructions for the best contributor experience and quality data collection. We will use an actual example of instructions from a sentiment analysis job to demonstrate how your instructions should be structured.
Also ensure that the instructions and task aligns with the job title. This will help reinforce expectations for the contributors and entice them to work on jobs that are interesting/appealing to them.
- ex. "Sentiment Analysis Evaluator: Project Airline"
Fig. 1: Instructions Template
Fig. 2: Preview of job examples
The overview should give a clear high-level picture of what a contributor will be doing. This can be brief with just a few sentences.
The example above explains the overall goal this job and its final purpose. The final purpose should describe what product or work this task will contribute to.
- Ex: "Your contributions will help an airline company improve customer experience and build a comprehensive chatbot for customer service requests"
Be sure to also include any risks with performing the task in this section as well. The crowd should be aware if there is a potential to come into contact with harmful and/or offensive content.
- Ex: "Some of the Tweets that you are rating may be explicit in nature"
Describe the step by step process contributors will take to complete the task. Try not to jump around when explaining the steps as it may cause confusion.
In the example above, each of the steps reflects an action the contributor will need to take. In this use case, the steps show contributors how to determine if the sentiment of the tweet toward American Airlines is positive, neutral or negative.
Rules and Tips
Provide rules to contributors on what they should and should not do. Explaining the way an answer was determined can help teach them how to provide the best response.
In the example above, this section provides additional definitions and text examples of how to determine if a tweet is positive, neutral, or negative. This section helps contributors understand the differences between each type of sentiment asked for.
By providing real examples from the source data, it can help contributors gain a better understanding of what they will be doing. This is especially important as contributors will get a glimpse of the type of data they will be working with and how to provide quality answers.
As seen above, there are examples of a positive, neutral, or negative tweet and a definition of why the tweet is placed in that category.
Our contributors really appreciate a customized thank you note! It only needs to be one or two sentences.
Best practices for writing instructions:
- Simplify your wording as much as possible, this will help contributors to understand and complete your task.
- Create an easy, repeatable step by step process as a guide for contributors to understand how to provide answers.
- Provide at least three examples, such as images or edge cases from the source data.
- Remember to provide all relevant definitions in your instructions.
- Preview your instructions before launch to ensure they appear as expected.
The commented HTML code for the instructions template can be found as an attachment to this article. We highly recommend using them to get the best results from our on-demand workforce.
Also attached is a commented Markdown version of the instructions template. This should be used for those that do not want to work in HTML.