This is a checklist that can be used to help you review the settings of your job. Completing this checklist will ensure that you’ve taken some of the more important features into consideration before you launch a job.

This checklist was built for all of the features of the Appen Enterprise Plan. To learn more about upgrading to Enterprise, please contact our Sales Team on the Appen website.

 

---

Settings

"Settings" can be reviewed via the gear icon on the right-side navigation. Choose this to review your settings prior to Launch.

Fig. 1: Contributor settings (Settings)

 

Quality Control Settings

Under the Quality Control tab there are three sections: Test questions, Quality Control Settings and Dynamic Judgments.

Fig 2: Quality Control Settings

Test Questions

Here you will find your settings for Test Questions (More Info), and you can check that you have set the following (if required):

 minimum accuracy for contributors to keep working on your job

 number of test questions for quiz & work modes

frequency of test questions

Fig 3: Test Questions Settings

 

Quality Control Settings

Here you will find other quality control settings (More Info); and you can check that you have set the following (if required):

minimum time per page

maximum judgments per contributor

answer distribution monitoring

Fig. 4: Quality Control Settings

Answer Distribution Rules enabled:

Fig. 5: Answer Distribution Selection (Settings > Quality Control > Quality Control Settings > Check "Contributor Answer Distribution Rules")

Dynamic Judgments

would you like to turn on dynamic judgments in order to collect more judgments in case of contributor disagreement? (More info)

Dynamic Judgments enabled:

Fig. 6: Dynamic Judgments Settings

 

Sharing/Visibility

Check settings related to receiving platform notifications, adding tags to a job, sharing a job, and adding a job to an existing project. (More Info)

people will be notified with updates about this job

job is shared to team members for editing and review

tags that will help you find your job easily

job is added to an existing project

 

Fig. 7: Sharing & Visibility Settings 

API

Check your API (More info) settings (if required)

do you need to use the API? 

 

Fig. 8: API Settings

 

Pay Page

how many rows should be shown on each page?

how much should contributors be paid for each page of work?

 

Fig. 9: Pay Page Settings

Note: if Fair Pay is enabled, this will be moved to the launch page.

Fair Pay UI Price & Rows Settings

If Fair Pay is enabled for your team, the Price & Rows Settings will be on the Launch page of your job

Fig. 10: Fair Pay Price and Rows Settings

Launch page settings (More info)

How many rows do you want to launch in this run?

How much do you want to pay for each data row?

How many contributors do you want to answer each question?

Fig. 11: Launch page settings (Launch - top navigation)

Contributors

Contributor settings are found in Settings > Contributors if Fair Pay is not enabled for your Team

 Geography: Which countries should contributors come from? (More info)

Language: Does your job require a certain language capability other than English?

 External or Internal Contributors: Do you want the on-demand workforce or an internal team? Both? (More info)

 Do you have any explicit content in your data? (More info)

 

 

Fig. 12: Contributor Settings (Launch > Crowd Settings) as shown with Fair Pay enabled