TQE Calculator

Overview

NB: A responsible TQE contains much more than what is on this page. Please refer to the Tranquality website for a full example of a possible TQE system.

Below is a calculator for TQE based on the scoring model parameters in Serge et al. 2024. These parameters are found in TQE metric files, which are XML files that contain information critical to TQE. Each of the three sections of the tool are dependent on information that can be input manually or auto-populated by uploading a metric file right at the start.

Part 1 of this tool, the Error Count Table, includes the selection of error types, along with parts of the scoring model: error weights and severity level penalty multipliers. This is also where you can upload a metric XML file. After the error table is configured (manually or by upload), you can also upload TEI (.xml) file from the TRG Annotation tool to auto-populate the error counts. Part 2 uses the data in the Error Count Table to calculate the Absolute Penalty Total of an annotated text. Part 3 takes in the rest of the scoring model as well as a threshold value or cutscore to deliver a pass/fail decision on the translation's acceptability.

There is also the option to download a summary of all the generated information as an Excel sheet, which itself contains all the formulas so that you can continue the TQE offline.

Future work includes adding the option to use a non-linear scoring model for the accurate evaluation of translations of larger texts.

1. Error Count Table

The below table represents all the errors found in the translation to be evaluated. On the left side are the names of all the error types to be considered, as well as the weights they are to be multiplied by. There is also a button to add more rows (for more error types). On the right side are boxes for count the errors of each type, separated into 4 severity levels for each type. Each severity level has a multiplier (not to be confused with the weights, which belong to error types).

Upload a metric XML file to auto-populate the Error Count Table with error types, error weights, and severity level multipliers. An example metric file can be downloaded here.

Once you have set up the error types, error weights, and severity level multipliers, input the number of errors at each severity level for each error type in the appropriate row. After all of the error counts are inputed, proceed to Part 2.

Metric upload:

Scorecard TEI output upload:

	Neutral	Minor	Major	Critical
Severity Level Penalty Multipliers:
× weight:
× weight:

2. Absolute Penalty Total (APT) Calculator

Break down the APT formula used to calculate the APT.

3. Quality Score Calculator (Linear Scoring Model)

Currently using: Raw Linear Scoring Model

APT:

EWC:

PT:

Model Calibration Parameters

MSV:

RWC:

APP:

Generate an Excel spreadsheet based on the specific metric treated here.

Note: The dynamic calculations in these downloadable spreadsheets only work if Enable Editing is selected upon opening the spreadsheet.

Coming soon: Non-linear Scoring Models

Loading