Martin Kroon, Sjef Barbiers & Stéphanie van der Pas. (2019, February 2). A filter for syntactically comparable parallel sentences [Conference presentation]. Grote Taaldag 2019, Utrecht University. https://www.universiteitleiden.nl/binaries/content/assets/customsites/avt/programmaboekje-2019-algemeen.pdf
This study investigates methods and measures to filter out parallel sentence pairs that are syntactically too different to be used in comparative-syntactic research. We explore four filtering methods that use the sentence-length ratio, the Damerau-Levenshtein distance, the graph-edit distance between dependency parses, and sentence-vector similarity, and compare them to the Levenshtein distance on part-of-speech tags. For all approaches we experiment with ignoring specific functional material. We also experiment with tf-idf weighting in the sentence-vector approach and with including morphological features in the dependency-parses approach. Furthermore, we investigate whether combining filters, either in parallel or in sequence, will render better results. For the evaluation, we use a manually labelled dataset of parallel English-Dutch-German aligned sentences extracted from the Europarl corpus (Koehn 2005). Our preliminary results on English-Dutch sentence pairs suggest that the dependency-parse approach achieves as high results as the Levenshtein distance and we expect it to have a potential to be more robust in its performance between other language pairs.