Support #3679

NL: harvesting results INSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857

Added by Ine de Visser 10 months ago. Updated about 1 month ago.

Status:ClosedStart date:12 Sep 2019
Priority:NormalDue date:
Assignee:Angelo Quaglia% Done:

0%

Category:-
Target version:-
Submitting Organisation: Knowledge-Base relevant?:No
Proactive:No Keyword #1:
Country: Keyword #2:
Originating UI: Keyword #3:

Description

Dear Angelo and INSPIRE team,

 

Like Denmark, we have also this week very bad harvesting results. We also tried harvesting several  times again (monday, tuesday, today). The connected download services are dropped from 128 to 57.

Is something changed in the geoportal?

Our  Service resource id: INSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857

Can you have a look at what is going wrong/ we are doing wrong?

 

Best regards Ine

 

 

 

 

History

#1 Updated by Angelo Quaglia 10 months ago

Dear Ine,

I will have a look at it immediately.

Best regards,
Angelo

 

#3 Updated by Angelo Quaglia 10 months ago

Another serious problem is that a weird way to raise an exception when requestign the data:

 

http://services.inspire-provincies.nl/AreaManagement/services/download_AM?request=GetFeature&Language=dut&DataSetIdNamespace=071a5652-c513-4722-bb45-bf522bb4349f&service=WFS&count=10&STOREDQUERY_ID=http://inspire.ec.europa.eu/operation/download/GetSpatialDataset&version=2.0.0&DataSetIdCode=071a5652-c513-4722-bb45-bf522bb4349f

Returns some data and an exception embedded in it:

<?xml version='1.0' encoding='UTF-8'?>
<wfs:FeatureCollection xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd http://www.opengis.net/gml/3.2 http://schemas.opengis.net/gml/3.2.1/gml.xsd http://inspire.ec.europa.eu/schemas/am/3.0rc2 http://services.inspire-provincies.nl/AreaManagement/services/download_AM?SERVICE=WFS&amp;VERSION=2.0.0&amp;REQUEST=DescribeFeatureType&amp;OUTPUTFORMAT=application%2Fgml%2Bxml%3B+version%3D3.2&amp;TYPENAME=am:ManagementRestrictionOrRegulationZone&amp;NAMESPACES=xmlns(am,http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fam%2F3.0rc2)" xmlns:wfs="http://www.opengis.net/wfs/2.0" timeStamp="2019-09-12T12:47:33Z" xmlns:gml="http://www.opengis.net/gml/3.2" numberMatched="unknown" numberReturned="0">
  <!--NOTE: numberReturned attribute should be 'unknown' as well, but this would not validate against the current version of the WFS 2.0 schema (change upcoming). See change request (CR 144): https://portal.opengeospatial.org/files?artifact_id=43925.-->
  <wfs:member>
    <am:ManagementRestrictionOrRegulationZone xmlns:am="http://inspire.ec.europa.eu/schemas/am/3.0rc2" gml:id="AM_MANAGEMENTRESTRICTIONORREGULATIONZONE_Zwemwaterlocatie.fid--390903e8_16d22685ac9_6aef">
      <am:inspireId>
        <base:Identifier xmlns:base="urn:x-inspire:specification:gmlas:BaseTypes:3.2">
          <base:localId>1009</base:localId>
          <base:namespace>NL.zwrloc</base:namespace>
        </base:Identifier>
      </am:inspireId>
      <am:downloadableDataset>
        <inspire_common:Code xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0">071a5652-c513-4722-bb45-bf522bb4349f</inspire_common:Code>
        <inspire_common:Namespace xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0">071a5652-c513-4722-bb45-bf522bb4349f</inspire_common:Namespace>
      </am:downloadableDataset>
      <am:name>
        <gn:GeographicalName xmlns:gn="urn:x-inspire:specification:gmlas:GeographicalNames:3.0">
          <gn:language>nl_nl</gn:language>
          <gn:nativeness xsi:nil="true" nilReason="UNKNOWN"/>
          <gn:nameStatus xsi:nil="true" nilReason="UNKNOWN"/>
          <gn:sourceOfName xsi:nil="true" nilReason="UNKNOWN"/>
          <gn:pronunciation xsi:nil="true" nilReason="UNKNOWN"/>
          <gn:spelling>
            <gn:SpellingOfName>
              <gn:text>Het Lageveld; Zwemvijver</gn:text>
              <gn:script>Latn</gn:script>
            </gn:SpellingOfName>
          </gn:spelling>
        </gn:GeographicalName>
      </am:name>
      <am:geometry>
        
          <ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd" version="2.0.0">
            <ows:Exception exceptionCode="NoApplicableCode">
              <ows:ExceptionText>Illegal input: comment content has embedded -- in it (index 79)</ows:ExceptionText>
            </ows:Exception>
          </ows:ExceptionReport>

#4 Updated by Ine de Visser 10 months ago

Hi Angelo,

Thanx for your respond. I will contact the dataproviders with this information.

Are these problems causing the total drop of 70 downloadable datasets? Are they blocking other services? or have all the services not available anymore now probleems.

That is strange because they were before last week available in the geoportal.

Best regards Ine

 

 

#5 Updated by Angelo Quaglia 10 months ago

See next comment.

#6 Updated by Angelo Quaglia 10 months ago

 

 

As I wrote to Michael:

 

 Updated by Angelo Quaglia 1 day ago

Comment Edit

Dear Michael,

I agree with you.

The problem is that I keep receving conflicting requirements: one is to only present to the user download links that actually work, i.e. that actually return some meaningful content.

I have already explained many times that in INSPIRE, download links are allowed to be access-protected , but then, the INSPIRE Geoportal cannot possibly check that the links actually work.

As a result, the DOWNLOAD_MATCHING_DATA_IS_AVAILABLE is correctly awarded but the DATA_DOWNLOAD_LINK_IS_AVAILABLE is not.

Angelo

 

#7 Updated by Angelo Quaglia 10 months ago

Dear Ine,

with the latest development version of the code, the number of downloadable datasets is 89.

I am publishing the changes now.

Angelo

 

#8 Updated by Angelo Quaglia 10 months ago

I have published the modifications and submitted a new harvesting.

#9 Updated by Angelo Quaglia 10 months ago

Dear Ine,

could you please tell me which is the service metadata document describing this WFS?

https://geoportaal.fryslan.nl/arcgis/services/ProvinciaalGeoRegister/PGR2/MapServer/WFSServer?service=wfs&request=GetCapabilities

Best regards,

Angelo

#10 Updated by Ine de Visser 10 months ago

Dear Angelo,

The service without metadata you are pointing to, was a problem of one of our dataproviders, they marked around 100 dataset metadata as INSPIRE, what is not the case. This problem is solved.

Also the performance problem, you pointed out is solved.

We have the difficult situation that our Dutch georegister also changed last Friday. We are aware things are not working well and also buzy to solve the issues with our discovery service.

It is very helpfull if you can tell what is changed, which requirement is added, in the EU geoportal last week, so we can identify the problems. We have no access protected services. I read in the Swedish issues that a check is added and the iNSPIRE Geoportal now checks that the download links are actually working.

Can you tell me where I can find the documentation of this check what is exactly tested? How do you check if download links are actually working? Because the number off not working services are much higher then iI expect.

Best regards Ine

#11 Updated by Daniele Francioli 10 months ago

  • Assignee set to Angelo Quaglia

#12 Updated by Angelo Quaglia 10 months ago

Dear Ine,

for example:

https://inspire-geoportal.ec.europa.eu/sandbox/resources/INSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857_20190917-143253/services/1/PullResults/21-40/services/3/resourceLocator1/download/services/1/resourceReport/

 

 

Indeed, if I try the request:

https://geodata.nationaalgeoregister.nl/inspire/bu/wfs?request=GetFeature&Language=dut&CRS=http://www.opengis.net/def/crs/EPSG/0/4258&DataSetIdNamespace=&service=WFS&count=10&STOREDQUERY_ID=urn:x-inspire:storedQuery:bu:FullDataset&version=2.0.0&DataSetIdCode=80b2d12c-32be-54fg-b6d3%E2%80%9434f4ffc06cf3

I get an error:

And that was the very first I checked.

You can open the Resource Browser to check all those Data Sets for which a Dowload Service was found but a download link could not be established:

https://inspire-geoportal.ec.europa.eu/sandbox/proxybrowser/?msdsResourceLocator=/INSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857_20190917-143253/services/1/PullResults&msdsCountryCode=NL#fq=id%3A%5C%2FINSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857_20190917-143253%2Fservices%2F1%2FPullResults*&fq=missingInteroperabilityAspect%3ADATA_DOWNLOAD_LINK_IS_AVAILABLE&fq=interoperabilityAspect%3ADOWNLOAD_MATCHING_DATA_IS_AVAILABLE&fq=sourceMetadataResourceLocator%3A%5C%2F*&q=*%3A*

The are 60 of them.

 

To see the problems you can open the Resource Browser on the Download Service Data Sets for wichi a matching Data Set metadata has been found but no download link could be tested successfully:

https://inspire-geoportal.ec.europa.eu/sandbox/proxybrowser/?msdsResourceLocator=/INSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857_20190917-143253/services/1/PullResults&msdsCountryCode=NL#fq=id%3A%5C%2FINSPIRE-8c93a17a-05f4-11e1-b7de-52540004b857_20190917-143253%2Fservices%2F1%2FPullResults*&fq=interoperabilityAspect%3ADATA_MATCHING_DOWNLOAD_IS_AVAILABLE&fq=missingInteroperabilityAspect%3ADATA_DOWNLOAD_LINK_IS_AVAILABLE&q=*%3A*

 

 

Then click on "Click here to see the issues with the Get Network Service Metadata Response"

I was asked by Robert Tomas to have the INSPIRE Geoportal check that the download links are actually working in order to avoid that INSPIRE Geoportal users are presented download links that do not work.

Robert told me this will be discussed in Prague.

Best regards,

Angelo

 

 

#13 Updated by Ine de Visser 10 months ago

Dear INSPIRE team,

I understand and support that the INSPIRE Geoportal now checks that the download links are actually working. As long as the tests are conform the INSPIRE directive, I have no comments on that.

This change without communication beforehand and limited answers on questions, causes lot of frustration and loss of resources.

Angelo informed us that we can discuss this change during the MIG-T, but do we have to wait weeks to get the information on what is actually changed and tested?

The Netherlands is like other member states hard working to be ready for the monitoring on december 15TH. Each delay in getting the actual working of the geoportal clear, reduces the change we will be ready in time.

I really appreciate it if we get this week the information and explanation we needed and asked for two weeks ago. Based on that information, we can have discussions in the MIG-T and MIG if the new tests are fair and conform the directive. That doesn't seem to be on all aspects the case.

Best regards Ine

 

#14 Updated by Emanuela Epure 2 months ago

Dear Ine,

We are going through the issues that are still not addressed.

We would linke to understand if this issue is still valid for you. Can we close it?

Thank you in advance,

Emanuela on behalf of the JRC INSPIRE Support team

#15 Updated by Ine de Visser about 1 month ago

Dear Emanuela,

The main point in this issue,  that it is not clear to the member states when and what is being adjusted in the geoportal is still standing.

A release strategy and transparency about the adjustments, especially in the field of data service coupling, would be very welcome. This  should be accessible to a wide audience, for example through the geoportal itself.

Best regards Ine

#16 Updated by Emanuela Epure about 1 month ago

Dear Ine,


Thank you for your feedback. We will keep it in consideration for our planning and release processes within the INSPIRE Geoprtal.


Thank you,

Emanuela on behalf of the JRC INSPIRE Support team

#17 Updated by Emanuela Epure about 1 month ago

  • Status changed from New to Closed

Also available in: Atom PDF