Welcome to MQM Tools

This site is intended to provide tools as a public service to facilitate usage of Multidimensional Quality Metrics (MQM) in Translation Quality Evaluation (TQE). It does not purport to be a source of standards. Read more about MQM here.

Below is the first of many tools — a calculator for TQE based on the parameters in Serge et al. 2024. These parameters are found in TQE metric files, which are XML files that contain information critical to TQE. Each of the three sections of the tool are dependent on information that can be input manually or auto-populated by uploading a metric file right at the start. The parameters fulfilled by metric files are detailed in WK46396:

4.5
A TQE metric shall consist of (1) an appropriate selection of error types, (2) a scoring model, and (3) a mechanism to determine whether the evaluated translation has passed or failed. In this standard practice all three components are based on the MQM framework.

Part 1 of this tool, the Error Count Table, includes the selection of error types, along with parts of the scoring model: error weights and severity level penalty multipliers. Part 2 uses the data in the Error Count Table to calculate the Absolute Penalty Total of an annotated text. Part 3 takes in the rest of the scoring model as well as a cutscore to deliver a verdict on whether the translation passes or not.

There is also the option to download a summary of all the generated information as an Excel sheet, which itself contains all the formulas so that you can continue the TQE offline.

Future work includes adding the option to use a non-linear scoring model for the accurate evaluation of translations of larger texts.