Content
- Overview
- How it works
- Crowd Settings - Crowd Channel and GEO Settings
- Prices & Rows Launch Settings
- Review & Launch
Overview
Fair Pay aims to ensure that each contributor working on the Appen platform is paid fairly for work based on their geographical location. Users select their Price Per Judgment (PPJ), utilizing Appen's machine learning Fair Pay Price Per Judgment as a base price. This Fair Pay Price Per Judgment equates to an hourly rate that matches a little over the minimum wage of contributors in their respective countries.
Note: Fair pay is encouraged for all customers who use the ADAP platform. If your Team does not have Fair Pay enabled, contact help@appen.com or your Customer Success Manager to enable it.
How it Works
Before a job is launched, ADAP uses a machine learning model to predict how long contributors will need to submit a judgment. This is known as Estimated Time Per Judgment (TPJ). Then, based on geolocation preferences, the Fair Pay logic will recommend a Fair Pay Price Per Judgment (PPJ) to ensure contributors are paid above the minimum wage in their countries.
The Launch Page contains settings that will impact the cost of your Job. The Fair Pay flow is centered around Launch Page settings.
Crowd Settings - Crowd Channel and GEO Settings
Crowd Channels
Figure 1: Crowd Channel selection via Launch Page
Here you can select which crowd(s) you want to target:
-
External
- General
- Custom Channel
- Appen Cavite
-
Internal
- Fair Pay does not impact this channel selection as payment is not used through the Internal Channel.
- NDA (if enabled for your team)
Please see the Contributor Channels article for more information on channel settings.
External
General
-
Geography Based Filters
-
Geo-targeting is used to determine the target rate for the countries you select. Once geo-locations are determined, the Fair Pay logic will recommend the appropriate PPJ to achieve the best rate that includes all the regions that have been selected. Example: If your job has both US and Brazil selected as targeted regions, then the Fair Pay price will be set such that the most expensive country is paid fairly.
-
Figure 2: Geo-targeting
-
Language-Based Filters
- Please see this article for information on geography and language targeting. Note that this has no impact on Fair Pay
Prices & Rows Launch Settings
Figure 3: Price & Rows Settings
Estimated Time per Judgment (TPJ)
Fair Pay utilizes a machine learning model that predicts the time a contributor will take to submit a judgment in your Job. This model is trained on thousands of jobs run on the platform and considers the job design's complexity.
Fair Pay Price per Judgment
Fair Pay logic calculates the appropriate Price Per Judgment (PPJ) utilizing the Estimated Time Per Judgement and Country rate requirements. Country Rate Requirement in Seconds x Estimated Time Per Judgement = Fair Pay PPJ. For example, the chosen county rate is $5/hr, and the Estimated TPJ is 50 seconds. Fair Pay recommended PPJ will be $0.35.
Price Per Judgment (PPJ)
You can adjust the PPJ by inputting directly or utilizing a slider. Payments are recommended to be equal to or greater than recommended Fair Pay PPJ.
Judgments per Row
The number entered by the user of unique, trusted judgments that the system will collect for each source data row. The number of judgments per row determines the number of individual contributors you would like to view and judge each source data row.
Note: Test Question rows are not affected by the number of judgments per row.
Rows per Page
Contributors see one to several rows on a single page. A page is a collection of rows a contributor must complete before submitting their answers. This setting determines how many rows are on each page of contributor work.
Estimated Job Cost
The amount users will pay for a Job considering PPJ and the number of rows available.
Review & Launch
Figure 4: Review & Launch Settings
Rows to Order
You can select how many rows you would like to order for each run of the Job. The default is 100 rows for any new job created. It is recommended that 100 rows be run first to determine if test questions, job design, and instructions are straightforward.
Cost Buffer
If the Job is estimated to cost less than $10.00 USD, then a cost buffer is added to reach this minimum. Any funds not used in the Job run will be allocated back to your account when the Job has finished.
Estimated Job Cost
The amount users will pay for a Job considering PPJ and the number of rows available.
Out of Allocated Funds
Jobs may run out of funds if too many contributors are failing out. This could be due to difficult test questions, unclear instructions, or overall a challenging job design. In the case that your Job runs out of funds, you will receive an email alerting you, along with a notification on the Launch Page of the Job. We provide the estimated cost range for the ordered rows that have not yet finalized. You will need to allocate more funds and resume the Job.