Guide to cml:group and multiple=“true” Aggregation

Overview

cml:group paired with multiple=”true aggregation allows requesters to aggregate data across multiple text fields and submissions within a cml:group.

The feature generates a new column in the aggregate report to display linked entities in a cml:group and the confidence of each linked response.

Building a Job

The following CML contains the possible parameters for a cml:group with multiple="true" aggregation job:

<cml:group multiple=“true” group_name=“group_1” aggregation=“true”> 

<cml:text name=“first_name” label=”Enter First Name:” /> 

<cml:text name=“last_name” label=“Enter Last Name:” /> 

</cml:group> 

Important Note: This feature is applicable as a possible aggregation method within text fields (cml:text) in aggregated reports only. It does not apply to test question units.

Parameters

Below are the required parameters to enable cml:group and multiple =“true” aggregation: 

• multiple
  • When set to "true" contributors can input a dynamic number of entries of the group. 
• group_name
  • The name of the output aggregation column.
• aggregation
  • When set to "true", an aggregated output is generated.

Aggregation

Aggregation for cml:group and multiple="true" functions as follows:

• Aggregation for cml:group with multiple="true" enabled aggregates linked instances, defined as cml:text fields within a cml:group, and provides confidences per instance.
• Confidence is calculated by dividing the total number of unique submissions from a linked instance by the total number of judgments on the unit.
  • Example: Given the example CML above, if one out of two contributors submitted "Johnny" and "Appleseed" for first_name and last_name respectively, the confidence would be 0.5

Reviewing Results

The output of the aggregated column is an array of JSON objects containing the linked entities and confidences ordered by decreasing confidence.

For example, if two contributors submitted the following responses:

Contributor 1:

Contributor 2:

The aggregated output for that row in the aggregated report would be:

[{"first_name":"Foo","last_name":"Bar","confidence":1.0},
{"first_name":"John","last_name":"Apple","confidence":0.5},
{"first_name":"Johnny","last_name":"Apple","confidence":0.5}] 

where:

• "first_name" and "last_name" are columns of the text fields in the group
• "confidence" is the level of agreement of the linked instances