The tables below contain descriptions of each job and unit state found on the platform and in reports.
Job States
Each job, at any given time, will be found in one of these states.
State | Description |
Unordered |
Job has not been launched. It may or may not contain units. But never contains judgments. |
Running |
Job has been launched and is currently collecting judgements. |
Paused |
Job has been paused. A paused job will always contain units. But may or may not contain judgments. Job can be resumed at any time. |
Canceled |
Job was cancelled. May contain units and judgements but is not currently running. |
Finished |
All ordered units have completed collecting judgements. |
Disabled |
Unit data and judgments are present. Job can be copied with all rows. |
Archived |
Job that has been in the finished state for 90+ days. Can be restored as needed. |
Launching |
Job is in the process of being launched. Typically shows in between ‘Not Launched’ and ‘Running’ states. Appears only for a few seconds. |
Unit/Row States
Each data row has a state that describes its status.
State |
Description |
new |
Initial state upon data load. These rows have not collected any judgements yet. |
judgeable |
The unit has been ordered and is awaiting judgment collection. |
judging |
The unit is in the process of collecting judgements. |
judged |
Enough judgements have been collected. But may still revert to ‘judgeable’ if the confidence score is not high enough at any point. |
golden |
Indicates a test question. |
hidden_gold |
Indicates a test question that has been disabled. |
canceled |
The unit has been canceled and is only resumable by an admin user manually changing the state. |
finalized |
The unit has collected enough trusted judgments to satisfy its requirements. |
preprocessing |
Applies only to Video Annotation jobs. When the job is launched, the unit goes into the preprocessing state. At this point the video is broken into a series of frames that will be shown to contributors. Once the process is complete, the unit transitions to 'judgeable'. |