Overview
In addition to the project-level downloadable dataset available as a .csv on the DATASET page (see this article), and reports for specific features (see Other Reports, below), there are additional project-level reports that can be downloaded from "Reports (Downloadable)", under the DASHBOARDS tab.
The Reports (Downloadable) tab contains the following reports, detailed below.
Note
As with Dashboards you can use the questions settings button to include/exclude particular form & text question metrics from appearing in your reports.
Job Daily Report
The Job Daily Report provides the throughput and quality dashboard information for each day, over the life of the project broken down by jobs.
Job Daily Report Details
- DATE: Date of the metrics. Metrics on each row will reflect activity for the row date only. The report is sorted from most recent activity to last.
- FLOW_ID: The ID of leading and following jobs. E.g. where a Leading Work Job is connected to a following QA job the flow ID will be the same for each job.
- JOB_ID: The unique job ID.
- JOB_TITLE: The job name.
- JOB_TYPE: WORK or QA.
- TOTAL_EFFECTIVE_DURATION_HOURS: Duration of audio submitted (if using audio_transcription)
- TOTAL_WORKING_HOURS: Calculated across all contributors for that job, for that date.
- TOTAL_SUBMITTED_UNITS: Calculated across all contributors and judgments for that job, for that date. Note for jobs collecting multiple judgments: if 3 judgments were collected on 5 units, the total submitted units will be 15.
- TOTAL_SUBMITTED_TIMES: Number of times units were submitted, will usually be equal to total submitted units unless you have quality settings that send a unit back to the original contributor to be reworked.
- UNITS_APPROVED_QA: Number of units accepted in QA. Will only be populated for QA jobs. Will always display 0 for Work jobs.
- UNITS_MODIFIED_QA: Number of units modified in QA. Will only be populated for QA jobs. Will always display 0 for Work jobs.
- UNITS_REJECTED_QA: Number of units rejected in QA. Will only be populated for QA jobs. Will always display 0 for Work jobs.
- QAd_UNITS_LEADING: Number of units sent to QA from the leading Work Job, this should equal the TOTAL_SUBMITTED_UNITS for the following QA job with the same flow ID.
- UNITS_APPROVED_LEADING: Number of units accepted in QA. Will only be populated for Work jobs. this should equal the UNITS_APPROVED_QA in the associated QA job.
- UNITS_MODIFIED_LEADING: Number of units modified in QA. Will only be populated for Work jobs. this should equal the UNITS_MODIFIED_QA in the associated QA job.
- UNITS_REJECTED_LEADING: Number of units rejected in QA. Will only be populated for Work jobs. this should equal the UNITS_REJECTED_QA in the associated QA job.
- SUM_REJECTED_TIMES: number of times units were rejected if sent back for rework
- ACCURACY/ACCEPTANCE_RATE: Total number of accepted units divided by total number of QAd units (for that date). Will only be populated for leading jobs.
- depending on which tools are being used and the quality settings of the jobs additional columns may be included with review reasons or more detailed metrics, such as word, question or tag error rates.
Contributor Daily Report
The Contributor Daily Report provides the throughput and quality dashboard information for each day, broken down by contributor.
Contributor Daily Report Details
- DATE: Date of the metrics. Metrics on each row will reflect activity for the row date only. The report is sorted from most recent activity to last.
- FLOW_ID: The ID of leading and following jobs. E.g. where a Leading Work Job is connected to a following QA job the flow ID will be the same for each job.
- CONTRIBUTOR_ID: ADAP contributor ID.
- JOB_ID: The unique job ID.
- JOB_TITLE: The job name.
- JOB_TYPE: WORK or QA.
- TOTAL_EFFECTIVE_DURATION_HOURS: Duration of audio submitted by the contributor for that date (if using audio_transcription)
- TOTAL_WORKING_HOURS: Calculated for that contributor for that job, for that date.
- TOTAL_SUBMITTED_UNITS: Calculated for that contributor, for that date. Note for jobs collecting multiple judgments this will mean collected judgments.
- TOTAL_SUBMITTED_TIMES: Number of times units were submitted, will usually be equal to total submitted units unless you have quality settings that send a unit back to the original contributor to be reworked.
- QAd_UNITS_LEADING: Number of units sent to QA from the leading Work Job, this should equal the TOTAL_SUBMITTED_UNITS for the following QA job with the same flow ID.
- UNITS_APPROVED_LEADING: Number of units accepted in QA. Will only be populated for Work jobs.
- UNITS_MODIFIED_LEADING: Number of units modified in QA. Will only be populated for Work jobs.
- UNITS_REJECTED_LEADING: Number of units rejected in QA. Will only be populated for Work jobs.
- SUM_REJECTED_TIMES: number of times units were rejected if sent back for rework
- ACCURACY/ACCEPTANCE_RATE: Total number of accepted units divided by total number of QAd units (for that date for that contributor). Will only be populated for leading jobs.
- depending on which tools are being used and the quality settings of the jobs additional columns may be included with review reasons or more detailed metrics, such as word, question or tag error rates.
Json
The Json report allows you to download the full dataset report (all columns) in JSON format.
Source
The Source report allows you to download your original source file.
Other Reports
In addition to the reports outlined above, there are other feature-specific reports that can be downloaded:
- Test Questions report: see Guide to: Test Questions in Quality Flow
- Job level aggregate & agreement reports: see Guide to: Multiple Judgments in Quality Flow Projects
- Error breakdown reports: some tools output detailed error metrics after QA, these can be downloaded via the Dashboards page, see Dashboards in Quality Flow Projects