When designing a job, keep in mind that careful planning pays off. Creating wireframes, writing focused and clear questions, and thinking about the job from the point of view of contributors will go a long way toward launching a successful job.
Write Clear and Concise Questions
- Keep the questions concise: Lengthy questions can slow down and confuse contributors.
- Use clear language: Contributors come from many linguistic and cultural backgrounds. By keeping the design simple, it allows contributors to do their job efficiently and effectively.
- Be mindful of the language: Review the language in the instructions and questions to ensure there is no bias toward a particular answer.
Consider User Experience
- Minimize scrolling & clicking: CML questions with lots of moving parts like scrollbars and buttons can become confusing. Carefully consider the layout to reduce contributor fatigue and improve efficiency.
- Provide shortcuts and hyperlinks: If the job requires navigating to other web pages, make sure to include hyperlinks such as a predefined google search.
- View the job through the contributor’s eyes: Use the Preview tab to see how the job is currently designed. This is also a great way to test the logic and validators in the job.
- Provide any additional information: Including examples and definition of option answers can help contributors understand the job.
Consider Data Collection
- Use validation: When using text fields it is recommended to use a validator to normalize text entries. Appen offers validators for phone numbers, addresses, integers, proper case, and many other types of responses. For more information, see check out this article on Validations.
- Use Regular Expressions with Test Questions: Regular Expression can be used to create flexible criteria when trying to match phrases or collect URLS. For more information, please check out this article on Regular Expression.
Graphical Editor vs. Code Editor
- The Graphical Editor is the ideal tool for creating a simple UI.
Note: Some users have found success setting up basic UIs in the Graphical Editor before switching to the Code Editor to add more complex features. However, once the job is saved in the Code Editor it will not be possible to return to using the Graphical Editor.
Once the job has been designed
- Test out the job: Keep in mind that there's always room for improvement as things rarely go precisely as planned at the beginning, which is a great reason to test out the job prior to launching the full data set. Do this by either launching the job internally or launching a small set of the data
- Monitor the job's test questions: By tracking which test questions are highly missed, this can highlight problem areas and provide insight into improvements for future iterations.