Support #2888

FI - NLSFI: GeoNetwork 3.2.0 - "OGC:Not" filter returns no records (was: Finnish National Discovery Service not responding)

Added by Angelo Quaglia over 3 years ago. Updated over 3 years ago.

Status:FeedbackStart date:25 Aug 2016
Priority:NormalDue date:
Assignee:Angelo Quaglia% Done:

0%

Category:Harvesting process
Target version:-
Submitting Organisation:FI - NLSFI Knowledge-Base relevant?:
Proactive:Yes Keyword #1:
Country:FI - Finland Keyword #2:
Originating UI: Keyword #3:

Description

Dear Kai,

the Finnish National Discovery Service was registered indicating the following endpoint:

http://www.paikkatietohakemisto.fi/geonetwork/srv/csw?service=CSW&request=GetCapabilities&version=2.0.2

 

 

 

It used to work fine but since  Dec 14 20:47 it is responding with an html page:

 

I have noticed that this URL is working:

http://www.paikkatietohakemisto.fi/geonetwork/srv/fin/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetCapabilities

The difference with the registered one is the fin/ path.

You can either modify the URL in the registration or restore the original URL.

Best regards,

Angelo

 


Related issues

Copied from Geoportal Helpdesk - Support #2832: FI - NLSFI: Inspire discovery service service not responding Closed 25 Aug 2016

History

#1 Updated by Angelo Quaglia over 3 years ago

  • Status changed from New to Assigned

#2 Updated by Angelo Quaglia over 3 years ago

You probably have updated GeoNetwork.

#3 Updated by Angelo Quaglia over 3 years ago

By the way, at https://ies-svn.jrc.ec.europa.eu/issues/2869

I am collecting here a list of OGC CSW 2.0.2 Catalogue software implementation in use in National Geoportals

I would appreciate if you could update that issue or simply tell me what version you have upgrade GeoNetwork to.

#4 Updated by Angelo Quaglia over 3 years ago

  • Priority changed from High to Normal
On 17 Dec 2016, at 10:06, Angelo Quaglia <angelo.quaglia@ext.jrc.ec.europa.eu> wrote:
Dear Kai,
I see you have two users:
 
Login First name Last name Email Administrator Created Last connection  
<958077cc017ca10cb91f4f1203883d40.png>kai.koistinen@maanmittauslaitos.fi Kai Koistinen   03 Mar 2014 06:47 am Lock Delete
<dcbf5650c4cbba7a0acb9f251206e981.png>kai.koistinen@nls.fi Kai Koistinen   29 Nov 2016 01:40 pm Lock Delete
 
I have enabled both but do you need really the two of them?
 
 
Best regards,
Angelo

#5 Updated by Angelo Quaglia over 3 years ago

I have set your other user as a watcher, too, so you should be receiving notifications.

#6 Updated by Angelo Quaglia over 3 years ago

  • Status changed from Assigned to Feedback

Dear Kai, I confirm you the service endpoint 

http://www.paikkatietohakemisto.fi/geonetwork/srv/csw?service=CSW&request=GetCapabilities&version=2.0.2

is not responding as a OGC CSW.

 

 

#7 Updated by Angelo Quaglia over 3 years ago

I see you have registered a new endpoint:

 

 

#8 Updated by Angelo Quaglia over 3 years ago

Unfortunately, the filter specified in the registration does not return any record of the 857 that are available:

<?xml version="1.0" encoding="UTF-8"?>

<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" service="CSW" version="2.0.2" maxRecords="20" startPosition="1" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" outputFormat="application/xml">
  <csw:Query typeNames="gmd:MD_Metadata">
    <csw:ElementSetName>full</csw:ElementSetName>
    <csw:Constraint version="1.1.0">
      <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
        <ogc:PropertyIsNotEqualTo>
          <ogc:PropertyName>Subject</ogc:PropertyName>
          <ogc:Literal>Ei-Inspire</ogc:Literal>
        </ogc:PropertyIsNotEqualTo>
      </ogc:Filter>

    </csw:Constraint>
    <ogc:SortBy>
      <ogc:SortProperty>
        <ogc:PropertyName>apiso:Identifier</ogc:PropertyName>
        <ogc:SortOrder>ASC</ogc:SortOrder>
      </ogc:SortProperty>
    </ogc:SortBy>
  </csw:Query>
</csw:GetRecords>

 

 

 

 

 

#9 Updated by Angelo Quaglia over 3 years ago

From: Angelo Quaglia [mailto:angelo.quaglia@ext.jrc.ec.europa.eu]
Sent: 19 December 2016 10:22
To: 'Koistinen Kai (MML)' <kai.koistinen@maanmittauslaitos.fi>
Subject: RE: [Geoportal Helpdesk - Support #2888] FI - NLSFI: Finnish National Discovery Service not responding

 

Dear Kai,

I hope you can now connect to the Helpdesk:

 

https://ies-svn.jrc.ec.europa.eu/issues/2888

 

No records are returned.

 

I am keeping the old records on the inspire geoportal for now.

 

Best regards,

Angelo

#10 Updated by Angelo Quaglia over 3 years ago

Dear Kai,

I gather "Ei-Inspire" means non-INSPIRE.

I think this is the problem:

No filter returns 857 records:

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" service="CSW" version="2.0.2" maxRecords="20" startPosition="1" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" outputFormat="application/xml">
  <csw:Query typeNames="gmd:MD_Metadata">
    <csw:ElementSetName>full</csw:ElementSetName>
  </csw:Query>
</csw:GetRecords>

 

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
   <csw:SearchStatus timestamp="2016-12-19T19:33:41" />
   <csw:SearchResults numberOfRecordsMatched="857" numberOfRecordsReturned="20" elementSet="full" nextRecord="21">

 

Get the records that are non-INSPIRE and you get 271 records:

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" service="CSW" version="2.0.2" maxRecords="20" startPosition="1" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" outputFormat="application/xml">
  <csw:Query typeNames="gmd:MD_Metadata">
    <csw:ElementSetName>full</csw:ElementSetName>
    <csw:Constraint version="1.1.0">
      <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
        <ogc:PropertyIsEqualTo>
          <ogc:PropertyName>Subject</ogc:PropertyName>
          <ogc:Literal>Ei-Inspire</ogc:Literal>
        </ogc:PropertyIsEqualTo>
      </ogc:Filter>
    </csw:Constraint>
  </csw:Query>
</csw:GetRecords>

 

 

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
    <csw:SearchStatus timestamp="2016-12-19T19:36:05" />
    <csw:SearchResults numberOfRecordsMatched="271" numberOfRecordsReturned="20" elementSet="full" nextRecord="21">

 

 

Get the records that aren't non-Inspire, i.e. they do not contain the "Ei-Inspire" keyword, you get 0 records, instead of 857-271 = 586

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" service="CSW" version="2.0.2" maxRecords="20" startPosition="1" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" outputFormat="application/xml">
  <csw:Query typeNames="gmd:MD_Metadata">
    <csw:ElementSetName>full</csw:ElementSetName>
    <csw:Constraint version="1.1.0">
      <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
          <ogc:Not>
          <ogc:PropertyIsEqualTo>
            <ogc:PropertyName>Subject</ogc:PropertyName>
            <ogc:Literal>Ei-Inspire</ogc:Literal>
          </ogc:PropertyIsEqualTo>
        </ogc:Not>
      </ogc:Filter>
    </csw:Constraint>
  </csw:Query>
</csw:GetRecords>

 

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
    <csw:SearchStatus timestamp="2016-12-19T19:38:17" />
    <csw:SearchResults numberOfRecordsMatched="0" numberOfRecordsReturned="0" elementSet="full" nextRecord="0" />
</csw:GetRecordsResponse>

 

 

 

 

#11 Updated by Kai Koistinen over 3 years ago

Dear Angelo,

 

We updated to Geonetwork version 3.2.0 on 5th of December. It seems that the not-filter is not working in the new Geonetwork version. Has there been similar issues with other countries that have updated to Geonetwork 3?

 

Please keep the old records available in the Geoportal until we find the way to fix this issue.

 

About my login information: it's fine for me to use the kai.koistinen@nls.fi account but I'm not sure if the @maanmittauslaitos.fi account is used somewhere. If it is not active anywhere it is ok to deactivate that one. 

 

Best regards,

Kai

 

#12 Updated by Kai Koistinen over 3 years ago

I think I succeeded to make a workaround to the CSW issue. I created a virtual CSW endpoint on http://www.paikkatietohakemisto.fi/geonetwork/srv/fin/csw-inspire-resources?SERVICE=CSW&REQUEST=GetCapabilities. It should return the right records with no filter. Is it working now?

Regards,

Kai

#13 Updated by Kai Koistinen over 3 years ago

The number of returned records with this workaround was <300 so I think it returned those records that it should have not returned :( I'll try find what is wrong with the CSW now.

#14 Updated by Angelo Quaglia over 3 years ago

Dear Kai,

I confirm you that the new virtual endpoint you registered returns only 271 records, those which have the "Ei-Inspire" keyword, so the non-Inspire ones:

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-24f97be1-c61d-11e6-b68e-52540023a883_20161219-200120/services/1/PullResults/

I have removed the records from the geoportal online cache and disabled the reharvesting but it is not an ideal situation for the production environment.

 

I have also restored the latest good havresting which returned 573 records, the INSPIRE ones:

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-88351fbe-05f3-11e1-b7de-52540004b857_20161114-085509/services/1/PullResults/

 

Best regards,

Angelo

#15 Updated by Angelo Quaglia over 3 years ago

  • Subject changed from FI - NLSFI: Finnish National Discovery Service not responding to FI - NLSFI: GeoNetwork 3.2.0 - "OGC:Not" filter returns no records (was: Finnish National Discovery Service not responding)

Dear Kai,

Looking at Issue #2869 Austria, Belgium and The Netherlands are working or plan to upgrade to GeoNetwork 3.2.

I have added as watchers of this issue the relevant contacts hoping they can help.

I have also changed the title of this issue according to what seems to be the actual problem (i.e. "GeoNetwork 3.2.0 - "OGC:Not" filter returns no records").

Best regards,

Angelo

 

#16 Updated by Angelo Quaglia over 3 years ago

The situation in production is:

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-88351fbe-05f3-11e1-b7de-52540004b857_20161114-085509/services/1/PullResults/

573 records cached in the geoportal.

The registration has been deactivated but the records are kept until the issues with the new endpoint are solved.

Harvesting is disabled as the endpoint is not responding any more.

 

 

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-24f97be1-c61d-11e6-b68e-52540023a883_20161219-200120/services/1/PullResults/

271 records NOT cached in the geoportal as they are the "EI-INSPIRE" records that should have instead been filtered out.

The registration is active and for the following virtual endpoint:

http://www.paikkatietohakemisto.fi/geonetwork/srv/fin/csw-inspire-resources?SERVICE=CSW&REQUEST=GetCapabilities&version=2.0.2

Harvesting is disabled because now the virtual endpoint has been modified and would return 863 records (i.e. INSPIRE + "EI-INSPIRE")

 

#17 Updated by Paul van Genuchten over 3 years ago

This issue has been picked up by fxp at https://github.com/geonetwork/core-geonetwork/issues/1827, a patch is available for both 3.0.x and 3.2.x

Let us know if the patch resolves your issue

 

#18 Updated by Angelo Quaglia over 3 years ago

Dear Kai,

any news about this issue?

Best regards,

Angelo

 

#19 Updated by Angelo Quaglia over 3 years ago

rom: Koistinen Kai (MML) [kai.koistinen@maanmittauslaitos.fi]
Sent: 31 January 2017 06:58
To: Angelo Quaglia
Subject: VS: any news regarding Finland Discovery Service?
 
 

Dear Angelo,

 

We have the Geonetwork batch that should fix the issue but I haven’t had time to test the batch yet. I’ll let you know when we have installed the batch.

 

Best regards,

Kai

#20 Updated by Kai Koistinen over 3 years ago

  • Assignee changed from Angelo Quaglia to Paul van Genuchten

Paul van Genuchten wrote:

This issue has been picked up by fxp at https://github.com/geonetwork/core-geonetwork/issues/1827, a patch is available for both 3.0.x and 3.2.x Let us know if the patch resolves your issue  

 

Dear Paul, 

I installed the Geonetwork patch in our demo environment at http://demo.paikkatietohakemisto.fi/ (CSW is at http://demo.paikkatietohakemisto.fi/geonetwork/srv/eng/csw) but I still get 0 records with query:

<?xml version="1.0" encoding="UTF-8"?>
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" service="CSW" version="2.0.2" maxRecords="20" startPosition="1" resultType="results" outputSchema="http://www.isotc211.org/2005/gmd" outputFormat="application/xml">
  <csw:Query typeNames="gmd:MD_Metadata">
    <csw:ElementSetName>full</csw:ElementSetName>
    <csw:Constraint version="1.1.0">
      <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
          <ogc:PropertyIsNotEqualTo>
            <ogc:PropertyName>Subject</ogc:PropertyName>
            <ogc:Literal>Ei-Inspire</ogc:Literal>
          </ogc:PropertyIsNotEqualTo>
      </ogc:Filter>
    </csw:Constraint>
  </csw:Query>
</csw:GetRecords>

 

With PropertyIsEqualTo I get the correct results. The index was rebuilt after reinstalling the patch. Do you have some idea what might be wrong now?

Best regards,

Kai Koistinen

#21 Updated by Kai Koistinen over 3 years ago

  • Assignee changed from Paul van Genuchten to Angelo Quaglia

Dear Angelo and Paul,

The ogc:PropertyIsNotEqualTo problem has been solved in Geonetwork 3.2.1 which we have now installed. 

Discovery service information and filter should be ok now in http://inspire-geoportal.ec.europa.eu/INSPIRERegistry/. If not, the service URL should be http://www.paikkatietohakemisto.fi/geonetwork/srv/fin/csw?SERVICE=CSW&REQUEST=GetCapabilities&version=2.0.2 and the filter <ogc:Filter xmlns:ogc="http://www.opengis.net/ogc"><ogc:PropertyIsEqualTo><ogc:PropertyName>Subject</ogc:PropertyName><ogc:Literal>Ei-Inspire</ogc:Literal></ogc:PropertyIsEqualTo></ogc:Filter>

Can you try the harvesting now, Angelo?

Best regards

Kai

 

 

#22 Updated by Angelo Quaglia over 3 years ago

Thank you Kai, it seems OK to me:

Result of the interaction with the Discovery Service

Resources available for discovery: 273, Expected Resource Count: 273, Actual Resource Count : 273

The validation report is here:

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-f670705f-f4e9-11e6-81e4-52540023a883_20170217-100622/services/1/PullResults/

 

The quality of the metadata is very low, though, less than 84%.

You can find the issues in the validation report.

You also seem to have lost all links from the WMS layers to the dataset metadata, which is a real pity.

 

Best regards,

Angelo

#23 Updated by Kai Koistinen over 3 years ago

I'm very sorry, the filter was still wrong. It was propertyisequalto but should have been propertyisnotequalto. I made this change now to the filter. With this filter you should get over 600 records and the quality of metadata should be better also. Please rerun the harvester.

Best regards,

Kai

#24 Updated by Angelo Quaglia over 3 years ago

No problem.

That's much better, thanks!

 
Result of the interaction with the Discovery Service

Resources available for discovery: 629, Expected Resource Count: 629, Actual Resource Count : 629

 

http://inspire-geoportal.ec.europa.eu/resources/INSPIRE-f670705f-f4e9-11e6-81e4-52540023a883_20170217-112659/services/1/PullResults/

 

Best regards,

Angelo

Also available in: Atom PDF