Overview
"Annotate With AI" enables you to pre-annotate tasks using generative AI, thereby expediting the process for your contributors. This initial version of the feature is currently supported by the cml:radios element.
Contributors will see pre-annotated radio buttons, which they can choose to accept or edit. Leverage this functionality to enhance efficiency and accuracy in the annotation process.
Building a Job
Once you have LLM enabled and at least one model configured, you can set up your job using the graphical editor or the code editor.
The Graphical Editor
Step 1: Click on the "Manage Language Models" link, located in the top right corner of the design page.
Step 2: Select the model you will use in this job and then click Save.
Step 3: Create a multiple choice question
Step 4: Click on the "AI" icon in the multiple choice question preview, located in the "Content" section of the design page. (If you do not see this icon, you may not have LLM enabled, please contact help@appen.com or reach out to your CSM for assistance.)
Step 5: Choose the model from the drop down. (You can also add a new model here by selecting the "Add a model" option, see this article for details on adding models. Once configured, return here to select your newly added model, and remember to also enable it for the job, see Step 1).
Step 6: Enable/disable "Indicate to contributors that AI was used in pre-annotation". If this checkbox is ticked, contributors will see the message "Pre-annotated with AI" along with the pre-annotation. Click Save.
Step 7: Click "Preview" to see your generative AI pre-annotation in action!
The Code Editor
After ensuring that you have a model configured and enabled for your job (see Step 1, of graphical editor instructions above, taking note of the model ID located next to the model name), you can add the cml attribute ai-annotation="[MODEL_ID]" within the cml:radios tag. This attribute takes an array where the first field is obligatory and must contain the model ID, as shown in the following example:
By default the entire job design is sent to the model, however you can choose to send specific information to the model by utilizing a second optional field in the ai-annotation attribute, that will reference html elements (and their children) in the cml by a class name or ID, as shown here:
In the above image the second field of the attribute array specifies "examples" that will be sent to the model. Note, there is an additional div, that does not have any id attribute. The contents of this div will not be sent to the model.
The syntax is as follows:
ai-annotation="[MODEL_ID, ['div1', 'div2', ...]]
To enable or disable the "Pre-annotated with AI" label, include the following attribute:
-
show-ai-annotation(defaults to "true")- set
show-ai-annotation="false"if you do not want the label to show to contributors
- set
Output
In quality flow, the output will be displayed in the judgmentRawJson column, as well as a separate field for the original model configuration.
Example of a cell in the judgmentRawJson column:
{"what_is_the_current_capital_of_russia:"st_petersburg","what_is_the_current_capital_of_russia_model_annotation":"st_petersburg","_meta":{"what_is_the_current_capital_of_russia":{"validators":["required"],"fieldType":"Radios"}}}
In the above example, the contributor agreed with the model pre-annotation and submitted the judgment without making any changes, hence the results in "what_is_the_current_capital_of_russia" and "what_is_the_current_capital_of_russia_model_annotation" are the same.