Breaking the Language Barrier in Mobility Data Catalogues

How the FEDORA Catalogue Learned to Read Every Language in Europe

The auto-translation pipeline described here was developed as part of FEDORA Deliverable D3.1 – Integrated Cross-Sectoral Data Space.

Open mobility data in Europe is scattered across hundreds of national and regional portals — and each one speaks its own language. A traffic dataset published by Austria’s data.gv.at portal might describe itself entirely in German; a French or Spanish regional portal will do the same in its own language. For anyone trying to search across all of them at once, that’s a problem: a catalogue that mixes languages becomes fragmented and hard to navigate, and keyword search quietly breaks down.

FEDORA’s answer is a purpose-built translation layer sitting inside its data catalogue, which is built on the open-source CKAN platform. Whenever a dataset is harvested or updated, a custom plugin automatically intercepts the incoming metadata — the title, description and tags — and routes it through a translation pipeline before it is indexed for search.

What makes the approach notable is how it translates. Rather than relying on a commercial cloud translation API, the plugin connects to a self-hosted instance of LibreTranslate, an open-source machine translation engine. Running it locally serves three purposes at once: it keeps sensitive metadata inside FEDORA’s own infrastructure, removes dependence on external services, and avoids hitting rate limits when harvesting large batches of datasets at once — a real risk when pulling in metadata from dozens of sources in a single run.

The translation itself is non-destructive. The English version is written into the catalogue’s main searchable fields, so every dataset becomes discoverable through a single, unified English-language search. But the original text isn’t discarded — it’s preserved alongside the translation, so a dataset originally published in Basque, German or Danish remains fully readable and searchable in its native language too. Users get both: one consistent search experience, and the original context intact underneath it.

The team has also been transparent about the pipeline’s limits. Based on observational checks, the underlying language-detection component performs at around 90% accuracy, with confidence dropping — and translations sometimes being skipped — for very short text strings. Built-in fallback logic ensures the system displays the original text rather than failing outright when something goes wrong, and edge cases are caught through manual review of the catalogue.

Looking ahead, the team is exploring the integration of a locally hosted large language model to strengthen the pipeline further — either to resolve the low-confidence detection cases or to replace the current translation engine altogether — all while keeping the same commitment to data sovereignty that shaped the original design.

It’s a small piece of infrastructure with an outsized effect: it’s what allows a researcher in Vienna, an engineer in Reggio Emilia, and a transport authority in Nicosia to search the same catalogue and actually find each other’s data.

Learn more: Read the full technical description in FEDORA Deliverable D3.1 – Integrated Cross-Sectoral Data Space, Section 4.3.

Inside the FEDORA Public Catalogue: One Front Door to Europe’s Mobility Data

One Search Box for Hundreds of Mobility Data Sources

The FEDORA Public Catalogue was developed as part of FEDORA Deliverable  D3.1 – Integrated Cross-Sectoral Data Space.

Europe’s mobility data landscape is vast and fragmented: government open-data portals, transport authorities, research institutes, private operators and geospatial agencies each publish datasets through their own systems, in their own formats, and often in their own language. For anyone trying to find relevant traffic, transit or logistics data, the practical starting point is usually a frustrating one — checking dozens of separate websites one by one.

The FEDORA Public Catalogue is designed to remove that friction. Built on CKAN, a widely used open-source data cataloguing platform, it acts as a single, publicly accessible entry point that lists what data is available across a wide range of external sources — without needing to centralise the data itself. Behind that simple front door is a genuinely substantial piece of integration engineering.

Datasets can be added manually, but the catalogue’s real strength is automated harvesting: a set of CKAN-based extensions that periodically pull in metadata from external portals, on a schedule ranging from continuous updates to monthly refreshes. To make that possible across such a fragmented landscape, the catalogue supports a whole family of metadata standards and protocols — from DCAT and DCAT-AP (including national variants used in Germany, Italy, Denmark, Spain, Norway, Sweden, Belgium and the Czech Republic) to the OGC standards used by geo-data portals, and APIs used by platforms like ArcGIS, Socrata and DKAN. Where a source doesn’t fit any of these standard patterns — as was the case with data.gv.at, Austria’s national open data portal — the team built a dedicated custom harvesting plugin to bridge the gap.

Managing dozens of these harvesting connections individually through a web interface doesn’t scale, so the team also built a command-line tool that lets administrators create, update or remove entire batches of harvesting sources from a single CSV file — turning what would be a slow, error-prone manual process into a single, auditable command.

Once data is in the catalogue, finding it is designed to be flexible: a simple keyword search for casual users, and an advanced Solr-powered search syntax for those who need it — supporting field-specific queries, wildcards, proximity matching, and fuzzy matching that tolerates typos and spelling variants. Results can be filtered by organisation, group, tag, format or licence, and individual datasets can include built-in visualisations — maps, charts, tables, or direct previews — depending on the type of data attached.

The catalogue is intentionally the visible tip of a larger system: behind the search box sits the harvesting infrastructure, the automatic translation pipeline, and the data quality tools that keep what’s listed trustworthy. Its job is deceptively simple — make it possible to type one query and find relevant mobility data, wherever in Europe it happens to live.

Learn more: The full technical description of the catalogue’s harvesting, search and visualisation capabilities is available in FEDORA Deliverable D3.1 – Integrated Cross-Sectoral Data Space, Chapter 4.