In addition to our Internal or External (General) channel selection, there is another option known as Custom Channels. This selection allows the launching of jobs to groups of preferred contributors created by the requestor's team. This feature is currently available for customers with a paid subscription. Contact your Customer Success Manager for more details.

Custom Channels improves judgment quality by only launching a job to contributors who have demonstrated specific skills when working on similar jobs or data in the past. This results in:

• Lower crowd cost by reducing the number of untrusted judgments from unskilled contributors.
• Ability to run more complicated/advanced Appen jobs that require contributors with a particular skillset. 

Custom Channels can be created on the Contributor page (located in Settings) on any job.

Fig. 1: Settings Page

Channels created this way are available to all team members to be used on any team job.

Accuracy Based Custom Channels

Accuracy based custom channels can be used to create a group of preferred contributors from a previously run job. To do so, select one or more finished jobs and an accuracy threshold for each job. In the absence of qualifying jobs, you can also run a specialized job, similar to the target job, to create a Custom Channel. 

Appen automatically evaluates contributors that meet or exceed the accuracy threshold you set to identify qualifying contributors and associate them with the named Custom Channel. When multiple jobs are chosen to source a Custom Channel, any contributor who meets accuracy threshold in any of the jobs qualifies to be in the channel.

Note: When Creating Custom Channels throughput can be impacted (especially for jobs that were previously targeted to the leveled crowd) but as more contributors are chosen, throughput should increase. 

Fig. 2: Create Custom Channel Via Previously Ran Jobs

 

Manual Custom Channels

Manual Custom Channels can be created using a list of preferred contributors who have been determined qualified to work in a specific job. This channel can be created by uploading a CSV file with contributor IDs.

Fig. 3: Create Custom Channel Via CSV

Note: The CSV must contain exactly one column header 'worker_id' with the contributor IDs stored under that column.

Fig. 4: Example CSV file

All contributors, their ID, overall accuracy, number of judgments and additional information can be found in the job Monitor > Contributors page. Performant contributors can also be found from reviewing the Contributor or Full report in a job.

Fig. 5: Contributors Page

How to Update a Custom Channel

Accuracy based Custom Channels can be updated by adding or deleting specified jobs and by modifying the accuracy threshold associated with each job. When a Custom Channel is updated, it is regenerated by clearing the previous list of contributors and adding contributors who qualify according to the new criteria. 

Fig. 6: Accuracy Setting

An existing manual Custom Channel can be updated by uploading a new CSV file. The new set of contributors replaces the existing set. If you want to add contributors to a manual Custom Channel, upload a file with both the existing and new contributor IDs. This can take a few minutes to update.

Fig. 7: Update Existing Custom Channel Manually 

How to Download a Custom Channel

To see which contributors are in a custom channel, you can download a CSV containing the contributor IDs (worker_ids). Next to each custom channel, you'll see a download icon (arrow pointing downwards):

Fig. 8: Download a Custom Channel

Once the download icon is clicked, a CSV list of contributor IDs for that channel is generated and downloaded to your browser.

Please note: The number of contributor IDs may be larger than the number of contributors in the channel. This is because contributors may sign up to work in Appen through different labor channels, which each assign a contributor ID. When a contributor is assigned to a custom channel, that assignment applies to each of the contributor IDs they're associated with.

Launching jobs to Custom Channels

Jobs can be launched to one or more custom channels of any type using the Contributor settings page. These jobs are exclusive to custom channels and cannot be launched to other contributors in the General External Crowd. The job may still be worked on via the Internal Channel. If multiple custom channels are selected, a contributor who is part of at least one of them can work on the job. 

Fig. 9: Selecting a Custom Channel 

To get the best results, always make sure to generate custom channels from similar jobs and use cases as Custom Channels override other skills applied to a job. For example, if a job was originally launched to level 3 contributors with knowledge of French, once a custom channel is applied to the job, the level and language setting are ignored. But if a Custom Channel is generated from a job with similar settings, the contributors will inherently have these skills.

Also, once a custom channel is applied to a job, level, language and geography preferences are removed from the job. If you wish to target the crowd again, you will need to re-apply these settings. Note that, to change channel settings, a job must be paused. On resuming the job, new judgments are collected only from qualifying contributors. 

Custom Channels and Test Questions

Custom Channels result is cost savings by reducing the number of untrusted judgments. However, similar to jobs launched to leveled contributors, it is important for jobs launched to Custom Channels to have test questions. While custom channels enable targeting contributors who have demonstrated quality in previous jobs, test questions evaluate judgments from these contributors continuously to ensure that the contributors maintain quality over time. 

Contributors who fail to maintain quality are removed from the job similar to jobs targeted to the General External Crowd. Once flagged or banned, contributors in the custom channel are not eligible to work on the job.