Note: This video tutorial displays a previous version of the Appen job user interface and may look slightly different than your current job page. Regardless, the functionality of buttons and links on the interface should remain the same.
Lesson Overview:
- This lesson reviews job monitoring, which allows you to check on the health of your job and identify areas for improvement.
- The first step in monitoring your job is reviewing your dashboard. This provides a high-level overview of the job.
- By far the most important aspect of monitoring the job is reviewing your test questions.
- Reviewing test questions:
- There is the option to sort by missed test questions. This brings the most-missed test questions to the top.
- Also helpful is to sort by contested, which brings the most-contested test questions to the top. This list will typically include the same questions as the most-missed, but not always: sometimes the most-missed test question is perfectly reasonable to the contributors so they do not contest it.
- When a test question is highly missed and highly contested, a ‘review’ tag appears next to it. Once you’ve reviewed the question and either made changes or deemed it correct as-is, you can mark it as “passed review.” This lets you know you don’t have to come back to it later.
- When you change the answer to a test question, this will update the accuracy of all contributors who answered. Anyone who was originally marked incorrect will be “forgiven”, and their accuracy will increase accordingly.
Steps To Complete:
- Navigate to ‘Monitor’ page
- Review job overview metrics
- Navigate to ‘Quality’
- Review all Test Questions that are highly missed/contested
- Update erroneous test questions
- Navigate back to ‘Monitor’ page
- Observe how updating inaccurate test questions changed the job overview metrics
