Teaching Data to Recognise “Gipuzkoa” and “Gipuzcoa” as the Same Place
This data-matching method was developed as part of FEDORA Deliverable D3.1 – Integrated Cross-Sectoral Data Space.
Anyone who has worked with real-world, multi-source data knows the problem: the same place, organisation or entity can appear under several different names. A Basque province might be spelled “Gipuzkoa” in one dataset and “Gipuzcoa” in another. A city might be abbreviated as “S. Cristóbal” in one source and written out fully as “San Cristóbal” in the next. And in genuinely multilingual regions, the same place might carry two entirely different names in two different languages — “Pamplona” in Spanish, “Iruña” in Basque — with no obvious textual overlap at all.
For a data space pulling in datasets from dozens of organisations and regions, this isn’t a cosmetic issue — it silently creates duplicate entries, breaks joins between datasets, and undermines analysis that depends on knowing two records actually refer to the same thing.
FEDORA’s solution combines three distinct matching techniques, each suited to a different kind of naming variation. The first, EditDistance, measures how many single-character changes — insertions, deletions or substitutions — it would take to turn one string into another, making it effective for catching typos and minor misspellings. The second, TF-IDF, breaks text into fragments and compares their statistical overlap, which makes it well suited to catching abbreviations. The third, and most sophisticated, uses text embeddings — numerical representations that capture the underlying meaning of a word rather than just its spelling — allowing the system to recognise that two completely different-looking strings, in two different languages, refer to the same concept.
Run together through the open-source PolyFuzz framework, these three models let the system automatically group name variants and flag likely duplicates for human review, rather than requiring every dataset to be manually reconciled by hand. In one worked example from the deliverable, the system correctly grouped “Hondarribia Gipuzkoa” with its misspelled variant “Hondarribia Gipuzcoa,” each match carrying a confidence score to support the final human check.
It’s a small step in a larger data-cleaning pipeline, but it’s foundational: before mobility data from six different European regions can be meaningfully compared, combined, or analysed together, it first has to agree on what to call the places it’s describing.
Learn more: The full methodology and worked examples are available in FEDORA Deliverable D3.1 – Integrated Cross-Sectoral Data Space, Section 5.1.1.