Support #2809

UK - Ordnance Survey: Question about validation report

Added by Angelo Quaglia about 4 years ago. Updated almost 4 years ago.

Status:FeedbackStart date:13 Jul 2016
Priority:NormalDue date:
Assignee:Angelo Quaglia% Done:

50%

Category:Validation outcome
Target version:-
Submitting Organisation:UK - Ordnance Survey Knowledge-Base relevant?:
Proactive:No Keyword #1:
Country:UK - United Kingdom Keyword #2:
Originating UI: Keyword #3:

Description

-----Original Message-----
From: Peter Parslow [mailto:Peter.Parslow@os.uk]
Sent: 13 July 2016 11:22
To: angelo.quaglia@ext.jrc.ec.europa.eu
Subject: Favour: interpreting a metadata validation error

 

Angelo,

As you might guess, I've been looking at the really useful proxy browser, running a quick check of the records that my employer has provided. Most of the warnings & errors are obvious enough, and I can encourage the relevant people to fix them.

 

But there are two errors given against one record, which I can't understand.

 

The record is http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-f89f4772-05f5-11e1-b7de-52540004b857_20160713-060348/services/1/PullResults/41-60/services/15/resourceLocator2/download/services/1/downloadDatasets/1/downloadDatasetMetadataUrl/downloaded.xml

 

(I can give a shorter URL to the original)

 

The two validation errors are:

 

" *The URL "" is not a valid download link for a supported media type for the dataset or dataset part."

 

And

 

" *In content of element <SpatialDataSetDownloadLink>: The content is incomplete. The following elements would be valid here, all in namespace http://inspire.ec.europa.eu/schemas/geoportal/1.0: SpatialDataSetPartDownloadResourceLocator, MultiPartDownloadDescriptionLocator. at column 42, line 175"

 

My problems are that I can't find any empty URL, and I can't match the issue to line 175, which is part way through a keyword Citation. So presumably the validator counts lines differently to me. I can guess that "element <SpatialDataSetDownloadLink>" actually implies a problem with one of the "download" resource locators. Perhaps it's because one of them includes a > sign in the gmd:name/gco:CharacterString? That character doesn't need to be escaped in XML, but perhaps something about the way you are processing it makes it an issue.

 

Or perhaps I have the wrong idea, and the problem lies elsewhere. If you could help, that would be nice.

 

Peter

 

 

Peter Parslow

Principal Geographic Information Architect Products & Innovation, Ordnance Survey

 

 

History

#1 Updated by Angelo Quaglia about 4 years ago

-----Original Message-----
From: Angelo Quaglia [mailto:angelo.quaglia@ext.jrc.ec.europa.eu]
Sent: 13 July 2016 11:37
To: 'Peter Parslow' <Peter.Parslow@os.uk>
Subject: RE: Favour: interpreting a metadata validation error

 

Peter,

 

I will reply using the Collaboration Environment where I logged a new ticket for you:

https://ies-svn.jrc.ec.europa.eu/issues/2809

 

You should have already received a notification and you should be able to update it.

 

Best regards,

Angelo

 

#2 Updated by Angelo Quaglia about 4 years ago

  • Status changed from Assigned to Feedback

Dear Peter,

the errors you mention refer to the individual download datasets.

I suggest we address first the issues at the service level:

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-f89f4772-05f5-11e1-b7de-52540004b857_20160713-060348/services/1/PullResults/41-60/services/15/resourceLocator2/download/services/1/resourceReport/

 

It seems that the links to the metadata do not work.

The first entry says:

<link rel="alternate" type="application/atom+xml" href="http://os.uk/xml/atom/OSOpenMapLocal.xml" />
<link rel="describedby" type="application/vnd.iso.19139+xml" href="http://os.uk/xml/products/OSOpenMapLocal.xml" />
 

The link http://os.uk/xml/products/OSOpenMapLocal.xml is supposed to point to the dataset metadata but it does not work for me.

 

Download Service Data Sets Needing Review

 

Inspire Validation Issues
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
  • A link to the subfeed could not be found in the entry with title "OS MasterMap Integrated Transport Network Layer" or the link to it ("http://www.ordnancesurvey.co.uk/oswebsite/xml/atom/ITN.xml") could not be resolved successfully because of the following error: "Could not interpret document: "
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
 
Expert documentation
 
  • For one or more datasets the INSPIRE validation failed
Expert documentation
 
  • The metadata element "Response Language" is missing, empty or incomplete but it is required. Hint: ""
Expert documentation
 
  • The metadata element "Supported Languages" is missing, empty or incomplete but it is required. Hint: ""
Expert documentation
 
  • The metadata element "Get Describe Spatial Dataset Operation Metadata" is missing, empty or incomplete but it is required. Hint: ""
Expert documentation
 
  • Ensure that the links to the Spatial Data Set metadata are correctly specified and they are working
Expert documentation
 
Inspire Validation Warnings

#3 Updated by Angelo Quaglia about 4 years ago

Coming to the specific error you metioned, I need to inspect the subfeed.

 

#4 Updated by Angelo Quaglia about 4 years ago

The top feed is:

http://os.uk/xml/atom/LiveProductFeed.xml

It declares the following 32 datasets

OS Open Map - Local
OS Open Map Names
OS Open Roads
OS Open Rivers
1:10 000 Scale Black and White Raster
1:10 000 Scale Colour Raster
1:250 000 Scale Colour Raster
1:25 000 Scale Colour Raster
1:50 000 Scale Colour Raster
1:50 000 Scale Gazetteer
Address Layer 2
Address Layer
AddressBase
AddressBase Plus
AddressBase Premium
Boundary-Line
Code-Point
Code-Point Open
Code-Point with Polygons
OS MasterMap Imagery Layer
OS MasterMap Integrated Transport Network Layer
Meridian 2
OS Locator
OS StreetView
Land-form PANORAMA Contours
Land-Form PANORAMA DTM
Land-Form Profile Contours
Land-Form Profile DTM
Strategi
OS MasterMap Topography Layer
OS VectorMap District
OS VectorMap Local

 

I now focus on OS Open Names (the second entry in the top feed)

The feed entry is:

  <entry>
    <title>OS Open Map Names</title>
    <link rel="alternate" type="application/atom+xml" href="http://os.uk/xml/atom/OSOpenNames.xml"/>
    <link rel="describedby" type="application/vnd.iso.19139+xml" href="http://os.uk/xml/products/OSOpenNames.xml"/>
    <id>http://os.uk/xml/atom/OSOpenNames.xml</id>
    <rights>No limitations</rights>
    <updated>2015-03-24T09:39:00Z</updated>
    <summary>OS Open Names launched 23 March 2015. Open data gazetteer of settlements, roads, and post codes.</summary>
    <category term="http://www.opengis.net/def/crs/EPSG/0/27700" label="British National Grid"/>
    <georss:polygon>49.86 -8.45, 49.86 1.78, 60.86 1.78, 60.86 -8.45, 49.86 -8.45</georss:polygon>
    <inspire_dls:spatial_dataset_identifier_code>OS Open Names</inspire_dls:spatial_dataset_identifier_code>
    <inspire_dls:spatial_dataset_identifier_namespace>os.uk</inspire_dls:spatial_dataset_identifier_namespace>
  </entry>

 

 

The subfeed is http://os.uk/xml/atom/OSOpenNames.xml

<?xml version="1.0" encoding="utf-8"?>

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss">
  <title>OS Open Names</title>
  <subtitle>Gazetteer of Great Britain</subtitle>
  <link href="http://inspire.ec.europa.eu/featureconcept/NamedPlace/" rel="describedby" type="text/html"/>
  <link href="http://os.uk/xml/atom/OSOpenNames.xml" rel="self" type="application/atom+xml" hreflang="en" title="This document"/>
  <link href="http://os.uk/xml/atom/LiveProductFeed.xml" rel="up" type="application/atom+xml" hreflang="en" title="The parent service feed document"/>
  <id>http://os.uk/xml/atom/OSOpenNames.xml</id>
  <rights>No limitations</rights>
  <updated>2016-05-03T00:00:00Z</updated>
  <author>
    <name>Ordnance Survey</name>
    <email>customerservices@os.uk</email>
  </author>
  <entry>
    <title>OS Open Names in ETRS-89 (zipped GML)</title>
    <link rel="alternate" length="339259475" href="http://os.uk/xml/datasets/OSOpenNamesETRS-89.zip" type="application/x-gmz" title="OS Open Names in ETRS-89 as a single GB set; over 330Mb"/>
    <id>http://os.uk/xml/atom/OSOpenNames.xml_ETRS-89_epoch2</id>
    <rights>attribution required</rights>
    <updated>2016-05-03T16:26:00Z</updated>
    <summary type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p>April 2016 - OS Open Names ETRS-89 epoch 2 published, containing updated data and a bug fix with non-persistent identifiers</p>
        <p>The ETRS-89 variant is only available as a single national (GB) set. Even zipped it is more than 330 Mb; expands to almost 10Gb.</p>
        <p>To order British National Grid, by National Grid Reference squares, or as a full set on DVD, see https://os.uk/opendatadownload/products.html</p>
      </div>
    </summary>
    <category term="http://www.opengis.net/def/crs/EPSG/0/4258" label="ETRS89"/>
    <georss:polygon>49.86 -8.45, 49.86 1.78, 60.86 1.78, 60.86 -8.45, 49.86 -8.45</georss:polygon>
  </entry>
  <entry>
    <title>OS Open Names in ETRS-89 (zipped GML)</title>
    <link rel="alternate" length="339259475" href="http://os.uk/xml/datasets/OSOpenNamesETRS-89.zip" type="application/x-gmz" title="OS Open Names in ETRS-89 as a single GB set; over 330Mb"/>
    <id>http://os.uk/xml/atom/OSOpenNames.xml_ETRS-89</id>
    <rights>attribution required</rights>
    <updated>2015-08-21T11:56:00Z</updated>
    <summary type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p>August 2015 - OS Open Names now available in ETRS-89</p>
        <p>The ETRS-89 variant is only available as a single national (GB) set. Even zipped it is more than 330 Mb; expands to almost 10Gb.</p>
        <p>To order British National Grid, by National Grid Reference squares, or as a full set on DVD, see https://os.uk/opendatadownload/products.html</p>
      </div>
    </summary>
    <category term="http://www.opengis.net/def/crs/EPSG/0/4258" label="ETRS89"/>
    <georss:polygon>49.86 -8.45, 49.86 1.78, 60.86 1.78, 60.86 -8.45, 49.86 -8.45</georss:polygon>
  </entry>
  <entry>
    <title>OS Open Names in British national grid</title>
    <link href="https://os.uk/opendatadownload/products.html" title="OS Open Names in BNG; order by grid square"/>
    <id>http://os.uk/xml/atom/OSOpenNames.xml_BNG</id>
    <rights>attribution required</rights>
    <updated>2015-08-21T11:55:00Z</updated>
    <summary type="xhtml">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p>23 March 2015 - New Ordnance Survey gazetteer launched: OS Open Names</p>
      </div>
    </summary>
    <link rel="describedby" type="application/vnd.iso.19139+xml" href="http://os.uk/xml/products/OSOpenNames.xml"/>
    <link href="http://os.uk/docs/user-guides/os-open-names-user-guide.pdf" rel="describedby" type="application/pdf"/>
    <category term="http://www.opengis.net/def/crs/EPSG/0/27700" label="British National Grid"/>
    <georss:polygon>49.86 -8.45, 49.86 1.78, 60.86 1.78, 60.86 -8.45, 49.86 -8.45</georss:polygon>
  </entry>

</feed>

 

The error message is:

The URL "" is not a valid download link for a supported media type for the dataset or dataset part.

So, there is a problem with the download links, i.e. the link elements where rel="alternate" (or rel="section" for multipart downloads but this is not the case)

The third entry has no such link.

 

 

 

#5 Updated by Angelo Quaglia about 4 years ago

Peter,

I have modified the error message so that now is:

Inside entry with id:"http://os.uk/xml/atom/OSOpenNames.xml_BNG", no valid download link for a supported media type for the dataset or dataset part coud be found. The URL was "no suitable link element could be found"

 

#6 Updated by Angelo Quaglia about 4 years ago

There is a sample atom based download service with a code inspector here:

http://inspire-geoportal.ec.europa.eu/demos/ccm/codeview.html

#7 Updated by Peter Parslow about 4 years ago

Angelo,

thank you. I have been getting the update notifications; it's just that I had three days intensely on a different project.

I had not guessed that the error report related to the Atom feed; the change you've made to the report text will make that clear in future.

You should note that rel="alternate" is the default in an Atom feed. https://tools.ietf.org/html/rfc4287#section-4.2.7 4.2.7.2 states "If the "rel" attribute is not present, the link element MUST be interpreted as if the link relation type is "alternate"."

In the light of this, I feel that the <entry> is valid both to the Atom spec and to the requirements in TG Download.

However, we cannot satisfy the description in the first sentences at 5.2.3 "The link element of an entry is also used to provide a link which resolves to the pre-defined dataset. This shall be a direct link to the dataset (e.g. to a file)". In many cases the link is to an e-commerce / ordering system, as allowed in the INSPIRE Directive. As often mentioned, the technical guidance provides no guidance for these cases.

Reading the Atom spec further, I suspect that rel="payment" may be a better value in these circumstances - but that's a comment to the TG.

The OS Open Names case is the only one where we have an entry which does provide a direct download, as well as the alternative which goes via the ordering service. We could get better "compliance" by hiding this option, but would prefer to continue to advertise it through the discovery service.

Peter

#8 Updated by Angelo Quaglia almost 4 years ago

  • Category set to Validation outcome

#9 Updated by Angelo Quaglia almost 4 years ago

  • Tracker changed from Bug to Support

#10 Updated by Angelo Quaglia almost 4 years ago

  • Country set to UK - United Kingdom

#11 Updated by Peter Parslow almost 4 years ago

  • % Done changed from 0 to 50

In addition, I have now realised the cause of the error at our end: the broken links relate to products that we've withdrawn. For example, we withdrew our long standing "50K Gazetteer" when we replaced it with the much richer INSPIRE compliant one.

As part of the withdrawal process, we deleted the product-specific Atom files.

But we forgot to delete the entries for them in the overarching 'service' Atom file. Now I've spotted the pattern, it will be easy to fix!

#12 Updated by Angelo Quaglia almost 4 years ago

  • Proactive set to No

That is good news, thanks.

Also available in: Atom PDF