Follow

Guide To Enterprise Analytics

Enterprise Analytics provides in-platform reporting on the organization's usage and spending. By enabling the Organization and Team Admins to view in-depth analytics, this empowers them to make data-driven decisions about allocation, resourcing and ROI.

If you already have access to Enterprise Analytics you can access it in the following ways:

  1. Select this option on the menu bar on the left-hand side of the screen on ADAP:

    ent_analytics.png

  2. Go to this link directly by pasting it on the URL bar of the browser: https://client.appen.com/analytics/dashboard

If you'd like to request access to this feature, please contact your Customer Success Manager or Account Executive. 

Note: For Dedicated customers, this feature is currently not available On-Premises and only available via the cloud multi-tenant Appen Data Annotation Platform.

Enterprise Analytics consists of four different pages 

  • Overview 
  • User spend report
  • Job insights 
  • Contributor Stats

Each of these pages presents a different set of information about the team or organization, with a date picker that can customize the timeframe to retrieve information. 

Note: the Contributor Stats page is not date-specific, as it contains information for a single job). 

 

Overview

The Overview page covers the following data points: 

  • Contributor Spend
  • Activity Level 
  • Rows Finalized
  • Judgments Collected

Contributor Spend 

The contributor spends section shows the total amount an Organization or Team has paid contributors for working on its jobs over the given timeframe along with a line chart to show historical trends. 

  • Organization Admins have the ability to break this down by teams within their organization, or by projects the jobs are associated with. 
  • Team admin will only see this information for their own team, so the amount automatically broken down by the project. In addition to the breakdown, there is also a line chart to show historical trends. 

Screen_Shot_2019-02-21_at_10.30.13_AM.png

Fig 1. Contributor Spend Page

 

Activity Level - Active Jobs

  • The Rows Finalized section shows how many rows were finalized in the given timeframe
    • Note: A row is finalized when it has collected the requested number of trusted judgments

Activity Level - Judgments collected

  • The Judgments collected section shows how many judgments were submitted by contributors across all jobs and does not include test question judgments. 

Screen_Shot_2019-02-21_at_10.41.24_AM.png

Fig 2. Judgments Collected Page

 

User Spend Report

  • The User Spend Report covers how much team members have paid contributors over the given timeframe, along with their total lifetime spend 
    • Note: if a user has not spent any money on jobs over the given timeframe, they won't appear in the report even if they have a lifetime spend amount. 

Screen_Shot_2019-02-21_at_10.43.44_AM.png

Fig 3. User Spend Report page

 

Job Insights 

The Job Insights page provides a detailed breakdown of each job run over the chosen timeframe. It contains the following information per job: 

  • Title:  this is the title of your job
  • Job Owner: this is the email of the user who owns the job
  • Cost Over Period: this is the amount paid to contributors over the given timeframe
  • Status: Current state of the job
  • Team: if you are a Team Admin, this will be the same for all jobs
  • Project: the project in which the job is attached to
  • Tags: A list of tags that were added to the job
  • Total Cost: this is the total amount paid to contributors for the job over all time
  • First Order: the date of the first order placed on the job (when the first set of rows was launched)
  • Last Order: the date of the last order placed on the job. In jobs that have only one order, this will be the same as first order.
  • Cost Per Row: this is the actual amount paid per finalized row, including costs for tainted judgments and test questions
  • Total Rows: the number of rows uploaded to the job
  • Launched Over Period and Total Launched Rows: the number of rows ordered over the given timeframe and the total number of rows ordered in the job for all time, respectively
  • Finalized Rows Over Period and Total Finalized Rows: the number of rows finalized over the given timeframe, then the total number of rows finalized for all time, respectively
  • Total Judgments Over Period: the total number of judgments over the given timeframe, including untrusted judgments, but not including test question judgments
  • Trusted Judgments Over Period: the number of trusted judgments over the given timeframe, not including test question judgments
  • Untrusted Judgments Over Period: the number of untrusted judgments over the given timeframe, not including test question judgments
  • Total Judgments, Total Trusted Judgments, and Total Untrusted Judgments: the total number of each type of judgment for the job for all time

Screen_Shot_2019-02-21_at_10.46.12_AM.png

Fig 4. Job Insight page

 

Contributor Stats

The Contributor Stats page on the job dashboard provides a detailed view of how contributors are performing in the job and the work they're doing. The information provided is as follows: 

  • Trusted Judgments: The total number of  trusted judgments on non-test question rows 
  • Average Trust Score: The average test question accuracy of contributors in the job
  • Total Hours Active: The cumulative amount of time contributors have spent working on the job. This is not just the amount of time the job has been running. 
  • Seconds per Trusted Judgment: This is the average time in seconds it takes contributors to judge a row of data. 
  • Contributor ID and Channel: The contributor's account ID and the channel they're working through 
  • Earning: How much money is in USD the contributor has earned working in the job 
  • Total Judgments: The number of judgments the contributor has submitted, excluding test questions 
  • Hours Active: The amount of time the contributor has spent working in the job
  • Time per Judgment: the average time the contributor has taken to judge a row (in hours)
  • TQ Accuracy: The contributor's test question accuracy in the job 
  • TQs seen: The number of test questions the contributor has judged
  • Missed TQs: The number of test questions the contributor has answered incorrectly
  • Forgiven Count: THe number of missed test questions the 

Screen_Shot_2019-02-21_at_11.02.27_AM.png

Fig 5. Contributor Stats Page


Was this article helpful?
7 out of 7 found this helpful


Have more questions? Submit a request
Powered by Zendesk