feat: parse wfs for meta and layers infos by abulte · Pull Request #385 · datagouv/hydra

abulte · 2026-02-03T16:30:21Z

This add support for WFS services 🗺️ to hydra 🐙.

If a WFS is url-detected, it will fetch general info (WFS version, output formats) and layers info (names and supported projections). It will also compute detected_layer_name if a valid layer name is found in URL or resource title (synching title to catalog table is needed for that).

The general workflow is respected (this can be challenged) : WFS is handled as any other resource type, and then augmented by the scraped infos.

The scraped infos are stored in checks.ogc_metadata as JSONB. They are sent is to udata in analysis:parsing:ogc_metadata for every resource. I went with a full object since the layer structure has to be complex anyway (ie more than a list of strings).

There's a bit of future-proofing here: ogc_metadata is ready for other formats if needed (eg WMS).

owslib is used to parse the WFS XML: lib seems mature and it's always nice to avoid parsing XMLs manually 😬.

I have tested end-to-end with a local udata instance.

Bonus: a bit of refactoring in cli.py with _find_check helper.

❯ uv run udata-hydra analyse-ogc --url 'https://geobretagne.fr/geoserver/hlc/wfs?REQUEST=GetCapabilities&SERVICE=WFS&typename=qualite_eaux_baignade_station'
2026-02-10 15:59:12 dev.local owslib.feature.wfs100[39210] WARNING pyproj not installed
2026-02-10 15:59:13 dev.local udata-hydra[39210] ERROR Could not find a check linked to the specified URL
2026-02-10 15:59:13 dev.local udata-hydra[39210] DEBUG Starting OGC analysis for https://geobretagne.fr/geoserver/hlc/wfs?REQUEST=GetCapabilities&SERVICE=WFS&typename=qualite_eaux_baignade_station
2026-02-10 15:59:13 dev.local udata-hydra[39210] DEBUG OGC analysis complete for https://geobretagne.fr/geoserver/hlc/wfs?REQUEST=GetCapabilities&SERVICE=WFS&typename=qualite_eaux_baignade_station: 10 layers found
2026-02-10 15:59:13 dev.local udata-hydra[39210] INFO OGC analysis completed successfully.
2026-02-10 15:59:13 dev.local udata-hydra[39210] DEBUG {
  "format": "wfs",
  "version": "2.0.0",
  "layers": [
    {
      "name": "hlc:classement_voirie_po",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:classement_voirie_li",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:dechetteries_pt",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:lieuxcovoiturage_hlc_pt",
      "default_crs": "EPSG:2154"
    },
    {
      "name": "hlc:mediatheque_hlc_pt",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:pav_zoneinfluence_parcelle_po",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:dechetpav_pt",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:qualite_eaux_baignade_station",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:voirie_statistiques_par_ville",
      "default_crs": "EPSG:3948"
    },
    {
      "name": "hlc:sd_observation_terrain_flore_rare_hlc_pt",
      "default_crs": "EPSG:3948"
    }
  ],
  "output_formats": [
    "application/gml+xml; version=3.2",
    "GML2",
    "KML",
    "SHAPE-ZIP",
    "application/geopackage+sqlite3",
    "application/json",
    "application/vnd.google-earth.kml xml",
    "application/vnd.google-earth.kml+xml",
    "application/vnd.ogc.fg+json",
    "application/x-gpkg",
    "csv",
    "dxf",
    "geopackage",
    "geopkg",
    "gml3",
    "gml32",
    "gpkg",
    "gpx",
    "json",
    "mif",
    "ods",
    "tab",
    "text/csv",
    "text/xml; subtype=gml/2.1.2",
    "text/xml; subtype=gml/3.1.1",
    "text/xml; subtype=gml/3.2",
    "xlsx"
  ],
  "detected_layer_name": "hlc:qualite_eaux_baignade_station"
}

Fix ecolabdata/ecospheres#892
Related ecolabdata/ecospheres#846

abulte · 2026-02-03T16:44:04Z

TODO

final review of tests
~~should we keep urn:ogc:def:crs:EPSG::3948 for projections or a more standard EPSG::3948?~~ Short codes are probably better for our usages (frontend libs and QGIS), and it's supported by owslib.
use more hints for detection (resource.format)
future-proof naming (ogc vs wfs)

maudetes

Nice! It works nicely 🎉
Main main comment is on the scope of the hydra detection 😛

udata_hydra/cli.py

udata_hydra/config_default.toml

udata_hydra/analysis/ogc.py

udata_hydra/config_default.toml

udata_hydra/analysis/helpers.py

udata_hydra/analysis/resource.py

udata_hydra/utils/ogc.py

maudetes

So cool!

For future reference, working examples on layer detection:

non-working examples:

https://www.data.gouv.fr/datasets/zones-de-repartition-des-eaux-zre-metropole?resource_id=3b1ab5f0-0fb0-4fe5-be77-2eaa6ecbe3ba
almost every WFS on the first page of https://www.data.gouv.fr/datasets/search?format=wfs (that don't have any layer information)

maudetes · 2026-02-10T10:04:50Z

udata_hydra/analysis/ogc.py

+                metadata["output_formats"] = list(output_formats.get("values") or [])
+
+            # Extract feature type information
+            for name, layer in wfs.contents.items():


Somehow, the feature type names get prefixed with a namespace? Ex for this resource, the layer name, parsed here is sa:ZRE_FXX, which doesn't match with the title.
But it isn't prefixed in the layer name in WMS GetCapabilities.

Interesting! I added a match on detected layer without namespace (if and only if one match, to avoid namespace collision) 0052167.

The detected layer name includes the namespace, it will probably work better when queried from frontend.

maudetes · 2026-02-10T10:13:34Z

udata_hydra/cli.py

Can you add title in _insert_resource_into_catalog

abulte · 2026-02-10T15:02:25Z

@maudetes I changed the wording from "feature" to "layer" since layer is more generic if we ever add support for other OGC services e30c416.

feat: parse wfs for meta and layers infos

a6ced49

abulte added 5 commits February 3, 2026 17:49

use proj short codes

10f2be7

review tests/test_analysis/test_wfs.py

d8a606d

useless comments

ef2caca

use resource.format to detect wfs

42463aa

rename wfs to ogc

4d24f24

abulte marked this pull request as ready for review February 4, 2026 15:57

maudetes reviewed Feb 5, 2026

View reviewed changes

abulte added 3 commits February 5, 2026 10:14

review fixes

710167c

feat: detected_feature_type_name

a6580d9

remove other_crs

4476367

maudetes reviewed Feb 10, 2026

View reviewed changes

abulte added 3 commits February 10, 2026 15:25

future-proof: layers instead of features

e30c416

review fix: _insert_resource_into_catalog

1e79493

review fix: match layer name w/o namespace

0052167

Comments

Conversation

abulte commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abulte commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maudetes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maudetes left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maudetes Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

abulte Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

abulte Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

maudetes Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

abulte Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

abulte commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

abulte commented Feb 3, 2026 •

edited

Loading

abulte commented Feb 3, 2026 •

edited

Loading

maudetes left a comment •

edited

Loading