OpenData – NATO STO CMRE OpenData Portal

NATO CMRE has been acquiring a tremendous amount of highly valuable environmental data throughout the years during its sea trials. Most of this data is already available for download via the GEOS3 system using HTTP and FTP links on a per sea trial basis, however a comprehensive search & download portal which would allow authorised users to discover content by performing searching and predefined metadata at global level is missing; users need to more or less be familiar with the directory structure of CMRE sea trials website to be able to download the data they might be looking for. In addition no or little information about provenance, quality and similar information is available prior to download.

The goal of this project is to lay down a plan for:

• creating proper metadata following international as well as NATO standards  described the resources acquired during an entire sea trial
• deploying an IT infrastructure able to ingest, index and perform searches on the created metadata, both via a user-friendly graphical user interface (GUI) as well as via standard protocols for machine-to-machine interaction
• maintaining and growing over time the base of described data via the above mentioned metadata with the long term objective to put online in a human-friendly searchable manner information about the whole catalog of data available on CMRE GEOS3 system

CKAN at work

Particular care is being taken for protecting the information that will be put together for feeding the system. At a minimum only authorised users via login will be allowed to perform queries on the system. Existing access policy will still be enforced as far as access to real data will be attempted.

The architecture offers distinct endpoints for different external access types (as shown in the images on the side):

• Metadata queries via OGC CSW protocol will be handled by GeoNetwork directly
• Metadata search and presentation via a user interface will be handled by CKAN
• Secure access to data files will be performed by Apache HTTPd directly

As part of this project we have deployed an instance of GeoNetwork 2.10 in order to ingest, manage and disseminate (both directly via CSW as well as indirectly via CKAN UI) metadata about the data collected during CMRE sea trials. In terms of metadata standards we have customized GeoNetwork to support the NATO GeoSpatial Metadata Profile (NGMP) which is a specialization of the ISO19139 metadata standard adopted by NATO.  In order to allow direct download via HTTP of the data being described with metadata files, that latter contains proper elements (e.g. OnlineResource elements) indicating the links on GEOS3 to where such data is available. Standard CMRE access policy enforcements kick-in in once such links will be followed upon.

Data detail page

CKAN 2.2 has been deployed in order to act as the Open Data portal for allowing users to perform visual searches to discover the data they are looking for. CKAN will be customized in order to allow it to harvest metadata from GeoNetwork via the CSW protocol periodically. Proper metadata will be indexed for later search and transformed once ingested in order to allow a nicer presentation to users without redundant information (we are talking about complex XML files). Download links will be extracted from the original metadata files and presented to the users in a comprehensive manner via the GUI of CKAN (as shown in the images below).

Existing fine grain access policies will be enforced once users will follow the download links to the existing sea trials pages. Only authenticated users will get access to the system in order to perform searches via the User Interface or via the CSW protocols. An authenticated user will be able to search for any metadata in the system as we do not foresee a closer integration with CMRE authentication and authorization system at this stage. Forcing an authorization filter on metadata according to the groups the user belongs to is not part of this work but could be accomplished by customizing GeoNetwork and CKAN.

We are proud to work with organizations like NATO STO CMRE in such challenging projects where we actually see our work impacting important use cases and being able to make a difference.

If you are interested in learning about how we can help you achieving your goals with open source products like GeoServer, MapStore, GeoNde and GeoNetwork through our Enterprise Support Services Subscription Services or Professional Training Services offer, feel free to contact us!

The GeoSolutions Team,