Follow

Guide To: Running a Job with Ontology Attributes

Overview

Our ontology attributes feature allows users to collect additional metadata on jobs using the cml:shapes tool. 

Important Note: As this tool is currently in BETA, it must be enabled for your team for access; please contact your Customer Success Manager or Account Executive for more information.

Building a Job 

CML 

The ontology attributes feature can be used in any cml:shapes type job. Please see our other articles detailing how to for run jobs for specific cml:shapes types:  

Ontology Manager 

Creating an Ontology Via the UI 

When enabled, ontology attributes can be accessed via the Ontology Manager page of a job and configured per class by clicking the pencil icon next to the desired class. 

blobid0.png

Fig. 1: Selecting a class to edit via the Ontology Manager

After selecting the pencil icon, a modal will appear with the ontology attributes section to configure questions to collect attribute information. The graphical editor follows the graphical editor used for creating questions in job designs (please see Guide to: The Graphical Editor).  

When creating questions for attribute collection, keep in mind there is a maximum of 5 and 50 options for multiple choice and dropdown items, respectively, to ensure forms are streamlined for contributors.  

blobid1.png 

Fig. 2: Ontology Attributes section and question type options

blobid2.png

Fig. 3: Example question set-up with Ontology Attributes

Creating an Ontology via JSON

An ontology can also be created by uploaded a JSON file. We recommend creating your first ontology via the graphical interface, then downloading the JSON to understand structure and continue building on.  

Here is an example JSON file that corresponds to the question created above in the graphical editor.

For classes with ontology attributes, the following variables must be included unless stated otherwise: 

  • id: Platform generated id for the question 
  • Questions: This array hold the JSON objects that describe the attributes added to the class 
  • customUserId (optional)Accepts a string for a customer provided unique identifier to map to an internal ontology schema that can be applied to questions and answers.  
    • Note: This currently can only be updated via JSON upload and appears in the output.
  • Type: The type of question used for the attribute (i.e. Multiple Choice, Checkbox Group, etc.) 
  • Data: a JSON object containing information on the attribute. The variables may change depending on the question type chosen. Standard variables include:
    • questionTextAccepts a string for the question label that appears to the contributor 
    • questionChoicesAccepts a string for the options to a question (i.e. for Multiple Choice and Checkbox Group) 
      • label: Accepts a string for the option label shown to the contributor 
      • value: Accepts a string for the option value stored in the output reports 
    • defaultsCheckedAccepts a boolean value for Checkbox questions to default a checkbox as marked 
    • isRequired: Accepts a boolean value for if a question is required 
    • tipsHints:  Accepts a string for a hint to give to the contributor 
    • resultsHeader: Accepts a string for the output name of the question provided in the output reports 

Please see the Ontology Manger article for information on standard ontology formats. 

Peer Review

In BETA, test questions are not supported, and peer review is a method to ensure quality. To create a peer review job, follow the guidelines in our Guide to: Running a Shapes Peer Review Job article. 

Monitoring and Reviewing Results

As this is a BETA feature, aggregation is not supported. Jobs should be run either to a trusted partner or in a peer review workflow (see above). 

Results

  • The Ontology Attributes feature will result in a JSON array per row with information on the shape annotation and its metadata. Within the metadata object per shape, there are shapeAnswers and shapeQuestions arrays to represent the attributes labeled for an object and information on the ontology available to the contributor. 
  • Example output from an ontology attributes job: 
    {
    "id":"2069ebb8-3e3e-4b57-b056-00eddd1a7c78",
    "class":"Car",
    "type":"box",
    "coordinates":{
    "x":298,
    "y":450,
    "w":410,
    "h":293
    },
    "metadata":{
    "shapeAnswers":[
    {
    "type":"Multiple Choice",
    "customUserId":"",
    "name":"occlusion",
    "answer":{
    "values":"yes",
    "customUserId":"122"
    }
    }
    ],
    "shapeQuestions":[
    {
    "id":"02906416-37be-42ca-97dd-40f81b9f88ef",
    "customUserId":"",
    "type":"Multiple Choice",
    "data":{
    "questionText":"What is the level of occlusion?",
    "questionChoices":[
    {
    "id":"a06aacd5-24bf-458f-9300-1af4666f5739",
    "customUserId":"122",
    "label":"Yes",
    "value":"yes"
    },
    {
    "id":"175bdde7-fc5b-4f6a-bfa5-788024a522d9",
    "customUserId":"133",
    "label":"No",
    "value":"no"
    }
    ],
    "isRequired":true,
    "tipsHints":"",
    "resultsHeader":"occlusion"
    }
    }
    ]
    }
    }

     

Each variable can be defined as the following:

Variable Description
id UUID assigned by the platform, can be applied to a shape annotation, question, or question response
class Ontology category assigned to the shape
type The type of shape used. Can be box, dot, line, or polygon. 
coordinates

The coordinates of the shape.

metadata An object containing information on the attribute answers and questions available in the job.

 

The metadata object includes the following:

 

Metadata variable Description
shapeAnswers

Object within metadata object containing answer information for specific attributes. Variables include:

  • type:Question form used (I.e. multiple choice or checkbox group)
  • customUserId: ID provided in ontology manager file
  • name: Value inputted for resultsHeader in ontology manager file
  • answer: Object containing information on the answer to the attribute
    • value: Value of chosen answer
    • custID: ID that corresponds to the value from the ontology manager file
shapeQuestions

Object within metadata object containing information on the attributes questions and values. Additional variables not defined above include:

  • data: JSON object containing information on the question text and choices
    • questionText: Question label shown to contributors during annotation.
    • questionChoices: An array of the available choices to the question
  • isRequired: Boolean value that determines if a question is required or not
  • tipsHints: An optional hint provided to contributor during annotation
  • resultsHeader: Value inputted for resultsHeader in ontology manager file.
  •  

Was this article helpful?
0 out of 0 found this helpful


Have more questions? Submit a request
Powered by Zendesk