Harvesting

other languages [de / es / fr]

The catalogues that are currently harvested are listed in the European Data Catalogues Overview.

Status Overview

Status Overview

 

You can see which ones are harmonised, pending to harmonised, pending to be harvested and which ones cannot be harvested.

Select Harvesting Status

Select Harvesting Status

 

Here it is also explained why these catalogues cannot be harvested. At the top of the page you can also search for a specific catalogue. In order to harvest the metadata from an open data catalogue, the catalogue needs to be registered first. This can be done in the catalogue registration panel.

Harvest Source

Harvest Source

In the catalogue registration panel you first have to enter whether the platform is based on CKAN or Socrata, or otherwise it is to be treated via HTML scraping. Thereupon a few details about the catalogue need to be provided, like its URL, country, language and source type. This information suffices for CKAN or Socrata based catalogues.

Harvesting Details

Harvesting Details

 

For HTML catalogues some additional information are necessary, because the harvester must learn how fields are described based on an exemplary dataset. Once the harvester has understood how the fields are described in this specific catalogue, it can transfer these rules to all datasets in the catalogue.

A registration needs to be approved by an administrator to ensure consistency in the ODM workflow. When the registration has been approved the catalogue’s metadata will be harvested.

*Please mind that the shown data may have changed in the meantime.

Video

Click on the subtitle icon at the bottom of the video to switch between French, German and Spanish subtitles.