Follow

Job and Unit States

The tables below contain descriptions of each job and unit state found on the platform and in reports.


Job States 

Each job, at any given time, will be found in one of these states. 

State Description

Unordered

Job has not been launched. It may or may not contain units. But never contains judgments.

Running

Job has been launched and is currently collecting judgements.

Paused

Job has been paused. A paused job will always contain units. But may or may not contain judgments. Job can be resumed at any time.

Canceled

Job was cancelled. May contain units and judgements but is not currently running.

Finished

All ordered units have completed collecting judgements.

Disabled

Unit data and judgments are present. Job can be copied with all rows.

Archived

Job that has been in the finished state for 90+ days. Can be restored as needed.

Launching

Job is in the process of being launched. Typically shows in between ‘Not Launched’ and ‘Running’ states. Appears only for a few seconds.

 


Unit/Row States

Each data row has a state that describes its status. 

 

State

Description

new

Initial state upon data load. These rows have not collected any judgements yet.

judgeable

The unit has been ordered and is awaiting judgment collection.

judging

The unit is in the process of collecting judgements.

judged 

Enough judgements have been collected. But may still revert to ‘judgeable’ if the confidence score is not high enough at any point.

golden

Indicates a test question. 

hidden_gold 

Indicates a test question that has been disabled.

canceled

The unit has been canceled and is only resumable by an admin user manually changing the state.

finalized

The unit has collected enough trusted judgments to satisfy its requirements.

preprocessing

Applies only to Video Annotation jobs. When the job is launched, the unit goes into the preprocessing state. At this point the video is broken into a series of frames that will be shown to contributors. Once the process is complete, the unit transitions to 'judgeable'.


Was this article helpful?
9 out of 9 found this helpful


Have more questions? Submit a request
Powered by Zendesk