Fair Pay Job Cost FAQ

Estimated Time per Judgment (TPJ)

Fair Pay utilizes a pre-launch machine learning model that predicts the time a contributor will take to submit a judgment in your Job. This model is trained on thousands of jobs run on the platform and considers the job design's complexity. 

• “Estimated Time per Judgment“ can be changed to “Seconds“, “Minutes“, and “Hours“.
• The minimum value for the “Estimated Time per Judgment“ can be 10 seconds, the maximum 3 hours.
• If you input time per judgment manually, then the TPJ input by the customer overrides the model-predicted TPJ.

Fair Pay Price per Judgment 

• Fair Pay logic calculates the appropriate Price Per Judgment (PPJ) utilizing the Estimated Time Per Judgment and Country rate requirements. Country Rate Requirement in Seconds x Estimated Time Per Judgment = Fair Pay PPJ. For example, the chosen country rate is $5/hr, and the Estimated TPJ is 50 seconds. Fair Pay recommended PPJ will be $0.35.
• The minimum for the “Price per Judgment“ is $0.01.

Judgments per Row

The number entered by the user of unique, trusted judgments that the system will collect for each source data row. The number of judgments per row determines the number of individual contributors you would like to view and judge each source data row.

• Note - Test Question rows are not affected by the number of judgments per row.

Rows per Page

Contributors see one to several rows on a single page. A page is a collection of rows a contributor must complete before submitting their answers. This setting determines how many rows are on each page of contributor work.

Estimated Job Cost

• The formula is the base_cost * cost_ratio * team_markup
  • The base_cost is the ideal cost assuming that no contributors are removed from the job after submitting units and there are enough rows to always show a complete page. 
    • The base_cost is num_units * judgments_needed_per_unit * payment_cents
  • The cost_ratio is a small additional buffer to account for untrusted judgments, test question judgments, and partial pages.
    • The cost_ratio is 1.02 if the job has no test questions
    • The cost_ratio is 1.05 + (gold_per_assignment / units_per_assignment) if the job has test questions
  • The team_markup can vary but using 20% as an example, the team_markup would be 1.2

Note

• • • If Dynamic Judgements is enabled, the system will always take the max judgments for calculating the Estimated Job Cost.