Guide to: Job Rotator

Overview

Appen’s API lets developers build applications that use and interact with Appen in an automated fashion. The Job Rotator allows users to integrate workflows running large volumes of data with our API to create a more efficient job creation and launching experience. 

Note: For Dedicated customers, this feature is currently not available On-Premises and only available via the cloud multi-tenant Appen Data Annotation Platform.

The Benefit

To avoid running into the platform’s 250,000-row maximum limit in a job and disrupting data processes, the job rotator references a job’s "alias" and creates a new copy of a job once a configurable row threshold is met. This eliminates the need to update API processes every 250,00 rows (approximately) with a new job id when you are using the same job design continually. If you are looking for a way to further automate and optimize launching large volumes of data to a single job design, this feature is highly useful.

How it Works

1. An Appen admin will create a rotation schedule by assigning an alias to a specific job ID via the job’s API Settings page. 
   • When the schedule is created, a maximum unit threshold is set. 
     • As best practice, we recommend setting the threshold around 220,000 or less to ensure the job does not hit the maximum row limit prior to rotating successfully. 
   • Additionally, the job rotation schedule will define if test questions should be included in new copies of the alias. 
2. Appen’s job rotation scheduler scrapes the platform every 30 minutes to check if any aliases have reached the maximum unit threshold set in their rotation schedule. 
3. If it has, the job rotator will copy the job attached to the alias into a new job. 
4. The alias will then be transferred to the new job copy. 
   • When this happens, no changes to API scripts or calls are needed as units are being sent to the alias rather than a specific job ID. 

Important Notes

1. This feature is configured by platform admins; if you are interested in using the job rotator, please reach out to your Customer Success Manager or Platform Support team (via chat or creating a support ticket) for more information. 
2. The job rotator is dependent on the user that owns the job with the alias assigned as it uses the job owner’s API key to rotate the jobs. Due to this, the job owner must be an active user on the Appen platform for the rotator to work successfully.