Guide to: Model Mate - Flexible Model Integration

Model Mate enables you to flexibly and easily integrate models in your job designs. Usecases for this feature are many and varied. In this article we provide some example usecases and step by step instructions for enabling the feature. Finally, we include some special considerations for image generation and processing.

Usecase Examples

You can incorporate models into your workflows in many ways. Here are just a few examples, however you can utilize this tool however you like, based on your specific needs.

1. A/B testing / Response Quality Assessment
   • Evaluating multiple live model responses and gathering feedback
   • Addressing specific questions about each response, such as accuracy and safety
   • Ranking responses
   • Categorizing images or provide feedback on visual characteristics
2. AI Assistant
   • Contributors can use AI for tasks such as content summarization, content generation and language translation
3. Content Screening
   • Detecting toxic content and warn contributors before they interact with it.
   • Screening contributor's responses and offer suggestions for improvement, or else directly applying fixes
4. Model-Annotation
   • Generating pre-annotations that can be refined by contributors

How to configure the tool

Step 1: Ensure you have models configured for your team, instructions can be found here.

Step 2: In your Job Design page click on "Manage Language Models"

Step 3: Enable the specific model[s] you want to use in your Job. Take note of the Model ID #.

Step 4: Switch to Code Editor and configure the cml:model tag:

<cml:model name="NAME" model-id="MODEL ID" prompt="Prompt to send to the model." /> 

For example, the below snippet would send the prompt "What is the capital of the USA?" to the Model with the ID "115".

 Step 5: Choose how to display the model's response in the job.

For example, you can choose to display the response in one of the editable text tools on the platform (i.e. cml:smart_text, cml:text or cml:text_area), with the addition of a model-annotation parameter that references the name attribute from the cml:model tag.

Or you may decide to just display the response on the page, using liquid syntax to reference the name attribute within an HTML element such a <div> or a <p>. 

Step 6: Save, preview and launch!

Example:

<h2>What is the capital of the USA?</h2>
<cml:model name="model1" model-id="115" prompt="What is the capital of the USA? Respond with only the capital. Don't include any other context or explanation."/>
<cml:text label="Capital:" validates="required" model-annotation="model1" />

Preview:

Parameters

The cml:model tag is used with the following parameters:

• name (required)
  • this is reference of your model. If you want to present your model response in an html element or a text element, you would use the name attribute
• model-id (required)
  • • this is the model ID of the model you are using. The model ID is assigned when you configure your model and which you see when you enable a model in your job
• prompt (required)

  • This parameter allows you to customize the prompt you want to send to your model. You can include a standard string, or add references to different parts of your CML:

 

Ref Type	Description
Source Data: {{column_name}}	This is the prompt you will be sending within the request to your model. This is highly customizable. You can use references to your data using liquid syntax:
Element Content: {'element_name'}	If you want to use what a contributor selected in a form element, wrote in a text element, or uploaded through the cml:file_upload tool and send that information with the model request, you can reference the name of the element within your cml:model prompt. The information that is referenced to the model is the label parameter. The following elements supported: checkbox checkboxes radios ratings drop down selection text textarea smart text file upload Contributors can upload images through the cml:file_upload tool and we can send that image to a model that supports image processing (Refer to the “Image Processing” section of this article). To use file upload with cml:model, you must also add a column in your uploaded dataset called _secure_directory. This allows access to CDS refs that are generated on the fly within the job. For example: allow-extensions only accepts 'JPEG' or 'PNG' image types when using it with cml:model Example: Within the job:
CML code: {'#element_id'} OR {'.element_class'}	If you want to send a reference to part of your CML, you can add an ID or Class to the CML you would like to send to the model, and then reference this ID or Class in your prompt. Example: Within the job:

 

• trigger-limit (optional, defaults to unlimited)
  • trigger-limit can only be used with trigger and allows you to specify the maximum number of times a contributor can click the associated button to send the same request to the model
  • example syntax: trigger-limit="5", contributors can click the button and send model requests five times.
  • if an error occurs, contributors will be able to click the button to send another request until they receive the number of successful responses specified by the limit
• model-output (optional, defaults to "true")
  • when model-output="true", a column is appended to the output containing the model's response
  • in cases where model-output="true" and trigger-limit="false", only the final response is output.
  • the label of the column in the output corresponds to the name attribute you specified in the tag, e.g. "model1".
• image (optional)
  • when you are integrating a model in order to process image data, you will need to include the image parameter along with the prompt within cml:model.
  • the image attribute currently accepts one image at a time in the following formats:
    • image URL
    • PNG (.png)
    • JPEG (.jpeg)
    • WEBP (.webp)
    • non-animated GIF (.gif)
• validates (optional)
  • validates="required" 

    • Defines whether or not the element is required to be answered

    • Defaults to not required if not present

Image Generation

In order to generate images using a model:

1. In the Model Configuration section, set up a model that enables image generation.

2. If you require your image to be in base64, ensure that the following is included in your input schema:

response_format: "b64_json"

3. You can only display your image in <p>, <div> or <cml:smart_text> tags.

Image Processing

To ensure your model can effectively process images:

1. Include the image attribute.
   • Make sure to include the image attribute when calling the model. This attribute specifies the image you intend to send to the model.
2. Update model configuration. Incorporate the following into your input and output schema.

Input schema:

{
  "payload": {
    "model": "gpt-4-turbo",
    "messages": "${message_items}",
    "temperature": 0.7
  },
  "message_item": {
    "role": "user",
    "content": [
    {
      "type": "text", 
      "text": "${message}"
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "${role}"
        }
      }
    ]
  }
}

Output Schema:

{
    "type: "multi-result",
    "results": "/choices",
    "text": "/message/content",
    "role": "/message/role"
}