Job Launch Checklist

This is a checklist that can be used to help you review the settings of your job. Completing this checklist will ensure that you’ve taken some of the more important features into consideration before you launch a job.

This checklist was built for all of the features of the Appen Enterprise Plan. To learn more about upgrading to Enterprise, please contact our Sales Team on the Appen website.



"Settings" can be reviewed via the gear icon on the right-side navigation. Choose this to review your settings prior to Launch.


Fig. 1: Contributor settings (Settings)


Quality Control Settings

Under the Quality Control tab there are three sections: Test questions, Quality Control Settings and Dynamic Judgments.

Fig 2: Quality Control Settings

Test Questions

Here you will find your settings for Test Questions (More Info), and you can check that you have set the following (if required):

 minimum accuracy for contributors to keep working on your job
 number of test questions for quiz & work modes
frequency of test questions

Fig 3: Test Questions Settings


Quality Control Settings

Here you will find other quality control settings (More Info); and you can check that you have set the following (if required):

minimum time per page
maximum judgments per contributor
answer distribution monitoring

Fig. 4: Quality Control Settings

Answer Distribution Rules enabled:


Fig. 5: Answer Distribution Selection (Settings > Quality Control > Quality Control Settings > Check "Contributor Answer Distribution Rules")

Dynamic Judgments

would you like to turn on dynamic judgments in order to collect more judgments in case of contributor disagreement? (More info)

Dynamic Judgments enabled:


Fig. 6: Dynamic Judgments Settings



Check settings related to receiving platform notifications, adding tags to a job, sharing a job, and adding a job to an existing project. (More Info)

people will be notified with updates about this job
job is shared to team members for editing and review
tags that will help you find your job easily

job is added to an existing project


Fig. 7: Sharing & Visibility Settings 


Check your API (More info) settings (if required)

do you need to use the API? 


Fig. 8: API Settings


Pay Page

how many rows should be shown on each page?

how much should contributors be paid for each page of work?



Fig. 9: Pay Page Settings

Note: if Fair Pay is enabled, this will be moved to the launch page.

Fair Pay UI Price & Rows Settings

If Fair Pay is enabled for your team, the Price & Rows Settings will be on the Launch page of your job


Fig. 10: Fair Pay Price and Rows Settings

Launch page settings (More info)

How many rows do you want to launch in this run?
How much do you want to pay for each data row?
How many contributors do you want to answer each question?


Fig. 11: Launch page settings (Launch - top navigation)


Contributor settings are found in Settings > Contributors if Fair Pay is not enabled for your Team

 Geography: Which countries should contributors come from? (More info)

Language: Does your job require a certain language capability other than English?

 External or Internal Contributors: Do you want the on-demand workforce or an internal team? Both? (More info)

 Do you have any explicit content in your data? (More info)




Fig. 12: Contributor Settings (Launch > Crowd Settings) as shown with Fair Pay enabled



Was this article helpful?
13 out of 14 found this helpful

Have more questions? Submit a request
Powered by Zendesk