[Elsnet-list] [Mt-list] ACT: a free, open-source MT Evaluation Metric
najeh.hajlaoui at idiap.ch
Mon Mar 25 14:01:30 CET 2013
We have just made available a new metric ACT "Accuracy of Connectives Translation", a MT metric specific for connectives translation. This is a free, open-source tool under GPL v3 license (available on github). It was built using Perl, so it runs on any platform:
ACT is a reference-based metric to measure the accuracy of discourse connective translation. The metric relies on automatic word-level alignment (using GIZA++, or not ) between a source sentence and respectively the reference and candidate translations, along with other heuristics for comparing translations of discourse connectives. Using a dictionary of equivalents, the translations are scored automatically, or, for more accuracy, semi-automatically. The accuracy of the ACT metric was assessed by human judges on sample data for English/French, English/Arabic, English/Italian and English/German translations; the ACT scores are within 2-4% of human scores.
We would like to maintain and further develop the ACT metric porting it to other language pairs and to other linguistic phenomena (verb tenses, pronouns, etc) that still pose problems for current SMT systems, so if your are interested to extend it to other new language, let us know.
Idiap Research Institute | tel: (41 27) 721 7761
Centre du Parc, CP 592 | fax: (41 27) 721 7712
CH-1920 Martigny | Najeh.Hajlaoui at idiap.ch
Switzerland | http://people.idiap.ch/nhajlaoui
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Elsnet-list