Support #2405

FR - PNEIG: Several requests coming from the INSPIRE Geoportal

Added by Angelo Quaglia over 5 years ago. Updated over 2 years ago.

Status:ClosedStart date:07 Apr 2015
Priority:NormalDue date:
Assignee:Angelo Quaglia% Done:

100%

Category:Harvesting processEstimated time:1.00 hour
Target version:-
Submitting Organisation:FR - PNEIG Knowledge-Base relevant?:
Proactive:No Keyword #1:
Country:FR - France Keyword #2:
Originating UI: Keyword #3:

Description

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [mailto:Robert.Riviere@developpement-durable.gouv.fr]
Sent: 07 April 2015 17:23
To: angelo.quaglia@ext.jrc.ec.europa.eu; "RIVIERE Robert - CETE Méditerr./DI/ETER"
Subject: from @PNEIG

 

Hello,

 

thanks for your answer to my message on twitter.

 

First let me introduce myself : I'm the project manager of "Geo-IDE", a french ministery of transports project for building a SDI.

 

Our catalog http://catalogue.geo-ide.developpement-durable.gouv.fr/ has been online for a year. It is based on "Geosource" product http://www.geosource.fr/ a french adaptation of Geonetwork, made by the BRGM agency http://www.brgm.fr/

 

We've recently experienced two crashes of our catalog, the first on March 31th morning, and the second this morning.

These crashes seem to be highly correlated with a robot hitting at a high rate the URL :

http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=...

IDs vary to match our catalog's metadata IDs.

 

The source IP address is 139.191.1.236

User-agent is  "Apache-HttpClient/4.3.3 (java 1.5)"

There is no Referrer

 

Could you please investigate what are these requests about ?

 

FYI, our catalog content is already harvested by the french Geocatalogue, http://www.geocatalogue.fr/, which AFAIK is the entry point for Inspire in our Country.

 

Thanks in advance,

 

Robert Rivière


Related issues

Related to Geoportal Helpdesk - Support #2836: FR - BRGM: Some batches fail Feedback

History

#1 Updated by Angelo Quaglia over 5 years ago

From: Angelo Quaglia [mailto:angelo.quaglia@ext.jrc.ec.europa.eu]
Sent: 07 April 2015 18:33
To: 'RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG'
Subject: RE: from @PNEIG

 

Dear Robert,

 

The INSPIRE Geoportal does not harvest the service on http://catalogue.geo-ide.developpement-durable.gouv.fr/

It does instead harvest the following service on a weekly basis:

http://www.geocatalogue.fr/api-public/inspire/servicesRest?version=2.0.2&service=CSW&REQUEST=GetCapabilities

 

Coming to the requests you mention, which take the form of http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=...

those may certainly come from IP address 139.191.1.236 which is the one of the INSPIRE Geoportal.

But they do not relate to direct harvesting by the INSPIRE Geoportal as it performs the harvesting exclusively via GetRecords requests.

 

URLs like those you exemplified, are typically contained in metadata like the one I have attached to this email.

 

Please have a look at the following section, where metadata resources coupled with the service are listed.

The geoportal simply takes each URL and tries to retrieve the resource, one by one.

How this can possibly take your service down, is unclear to me.

 

                    <srv:operatesOn uuidref="fr-120066022-jdd-a3cc793a-0897-40c9-abbe-72c42aeb843d" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-a3cc793a-0897-40c9-abbe-72c42aeb843d" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-54ffae81-88a7-4bdc-a9f4-719e6e8cfa89" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-54ffae81-88a7-4bdc-a9f4-719e6e8cfa89" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-65240e05-9a74-4a79-aa6d-6e32abae8f95" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-65240e05-9a74-4a79-aa6d-6e32abae8f95" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-054d2183-709a-4887-b64a-1b5643391281" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-054d2183-709a-4887-b64a-1b5643391281" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5c9cd900-0147-454c-bec1-2b5196be2eed" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5c9cd900-0147-454c-bec1-2b5196be2eed" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-bc97ad54-e2c3-4a7a-8f6c-f3abe4058173" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-bc97ad54-e2c3-4a7a-8f6c-f3abe4058173" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-7dbfb8c9-5a87-422f-aa0a-028d93da8423" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-7dbfb8c9-5a87-422f-aa0a-028d93da8423" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ab89d3ae-f0f1-4c87-a13a-d0d06b53d3a2" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ab89d3ae-f0f1-4c87-a13a-d0d06b53d3a2" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ddc39908-8514-4276-a6dd-91a337aed50f" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ddc39908-8514-4276-a6dd-91a337aed50f" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-253d549c-518b-4b2c-8cfe-96239961d253" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-253d549c-518b-4b2c-8cfe-96239961d253" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e8b01147-9d19-42f0-a436-382060cae4be" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e8b01147-9d19-42f0-a436-382060cae4be" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-53cf80b3-9ee0-4fc7-b650-916a72db43ff" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-53cf80b3-9ee0-4fc7-b650-916a72db43ff" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-6c707a48-cbab-4618-883c-b9a22a14ca09" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-6c707a48-cbab-4618-883c-b9a22a14ca09" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-05d72b01-30dc-4cd6-8c3b-8ff341b6ba21" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-05d72b01-30dc-4cd6-8c3b-8ff341b6ba21" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5201fe9d-eb56-49f8-8471-61aa40cf1b32" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5201fe9d-eb56-49f8-8471-61aa40cf1b32" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5af6c6e7-1a48-438b-b359-559ffc9fa913" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5af6c6e7-1a48-438b-b359-559ffc9fa913" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-34422ade-a25a-4225-aeec-dba5ca8f98c7" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-34422ade-a25a-4225-aeec-dba5ca8f98c7" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-c4fc71eb-d640-4b72-a0f4-4e1d45172c09" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-c4fc71eb-d640-4b72-a0f4-4e1d45172c09" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-64247983-c2bf-4083-ac73-62c7fa9367ba" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-64247983-c2bf-4083-ac73-62c7fa9367ba" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-2f302dda-6625-4856-a600-d33ecea6665b" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-2f302dda-6625-4856-a600-d33ecea6665b" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-fcb9ec9b-d525-41be-8ab6-5c60292da024" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-fcb9ec9b-d525-41be-8ab6-5c60292da024" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-feb1c502-cef3-463b-b15f-b8f2ba825a11" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-feb1c502-cef3-463b-b15f-b8f2ba825a11" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-a81e8cb7-bf70-4c39-b2ba-f306da0a4b25" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-a81e8cb7-bf70-4c39-b2ba-f306da0a4b25" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-c4269aec-b0c6-4be4-98e9-c28b0afff1e8" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-c4269aec-b0c6-4be4-98e9-c28b0afff1e8" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-2b61b545-5ff6-497e-a5ee-ae6b55eb97f8" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-2b61b545-5ff6-497e-a5ee-ae6b55eb97f8" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-06282491-db03-4d10-a172-c65f49299448" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-06282491-db03-4d10-a172-c65f49299448" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-0eb914eb-b2c8-41ff-ab01-b1a5b517e82b" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-0eb914eb-b2c8-41ff-ab01-b1a5b517e82b" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-c0cb4513-583b-4e20-aaae-0a0d968606be" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-c0cb4513-583b-4e20-aaae-0a0d968606be" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-d58d0321-27e0-4e78-922e-fd485b01e9e5" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-d58d0321-27e0-4e78-922e-fd485b01e9e5" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-6949e056-3e53-4abd-a03c-e995cf4ff6fb" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-6949e056-3e53-4abd-a03c-e995cf4ff6fb" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-057ec1b4-04f2-4714-bb51-784b9c9cefa2" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-057ec1b4-04f2-4714-bb51-784b9c9cefa2" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-0b4f0dce-76a3-49a5-95a3-35f46ad35467" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-0b4f0dce-76a3-49a5-95a3-35f46ad35467" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-391ea790-3981-4678-9587-dd73d4ad29f3" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-391ea790-3981-4678-9587-dd73d4ad29f3" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e4706649-f852-46de-a399-63e874e37cd2" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e4706649-f852-46de-a399-63e874e37cd2" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-3fb8046c-765c-40f3-9273-a21d5b7d0a42" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-3fb8046c-765c-40f3-9273-a21d5b7d0a42" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-bbf0240d-6484-447b-b17c-cf6fa28a34ff" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-bbf0240d-6484-447b-b17c-cf6fa28a34ff" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-fc216f9d-7e38-447f-9f6a-90da6276c2cd" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-fc216f9d-7e38-447f-9f6a-90da6276c2cd" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-a95eb37d-fa95-492a-b80b-2b03ec69c897" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-a95eb37d-fa95-492a-b80b-2b03ec69c897" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-4535e4ed-582a-4881-a61d-d7d8cb06f4f3" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-4535e4ed-582a-4881-a61d-d7d8cb06f4f3" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-9fc8d6e9-a397-4532-a584-ab7dfb97a971" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-9fc8d6e9-a397-4532-a584-ab7dfb97a971" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5fedeff4-9782-4d93-b0d4-063b2cc55c7d" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5fedeff4-9782-4d93-b0d4-063b2cc55c7d" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-8f3cda27-fd36-437b-beb1-cd4fdab35726" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-8f3cda27-fd36-437b-beb1-cd4fdab35726" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-7bccbf72-d239-4d5f-9d8d-a727594c8bb3" xlink:href="null" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ab702088-23c0-4b2f-9f80-185abebbe3e3" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ab702088-23c0-4b2f-9f80-185abebbe3e3" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-4ac737ea-966a-4b0b-b24c-ce736d1d73a0" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-4ac737ea-966a-4b0b-b24c-ce736d1d73a0" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-c604adda-f2a9-476d-8ab9-fc0a7434e549" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-c604adda-f2a9-476d-8ab9-fc0a7434e549" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-0d6b8e36-e86e-4fbd-af99-3d32ad6958fe" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-0d6b8e36-e86e-4fbd-af99-3d32ad6958fe" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e4fac012-5437-4950-b11e-b9be0d50615f" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e4fac012-5437-4950-b11e-b9be0d50615f" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-cf0a5d89-f6d4-4447-a7e3-aa8a71103fa6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-cf0a5d89-f6d4-4447-a7e3-aa8a71103fa6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5e02a51f-25e4-4222-9604-6608c9978a95" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5e02a51f-25e4-4222-9604-6608c9978a95" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ad684928-b9d9-4d21-ab05-8058ab155347" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ad684928-b9d9-4d21-ab05-8058ab155347" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-dbedc1c5-e9c1-4793-bf00-e5fb0cd4bb52" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-dbedc1c5-e9c1-4793-bf00-e5fb0cd4bb52" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-3570f7e8-20b3-473b-b5f5-faacd136a2f3" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-3570f7e8-20b3-473b-b5f5-faacd136a2f3" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-b5976239-6ef1-47d9-958b-a21f95ca97f6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-b5976239-6ef1-47d9-958b-a21f95ca97f6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-99f97825-da8e-4dcc-bce1-8ea5403250a9" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-99f97825-da8e-4dcc-bce1-8ea5403250a9" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ea27e469-2a80-40cb-b04f-8c172f42f558" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ea27e469-2a80-40cb-b04f-8c172f42f558" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-84faf656-9fa9-4ca8-8704-c1181d9eee27" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-84faf656-9fa9-4ca8-8704-c1181d9eee27" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-eb086039-3c4c-4141-8b39-258c75df7da9" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-eb086039-3c4c-4141-8b39-258c75df7da9" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e78805bf-10d6-41d2-906f-5c07ce77950e" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e78805bf-10d6-41d2-906f-5c07ce77950e" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-76114352-2118-462b-af73-091854d4533c" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-76114352-2118-462b-af73-091854d4533c" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ad85c91c-9ca6-4b51-b39a-aba0bc02c1da" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ad85c91c-9ca6-4b51-b39a-aba0bc02c1da" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e70804eb-1017-4984-b124-92fac8298b48" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e70804eb-1017-4984-b124-92fac8298b48" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e00ec55a-fe62-4e57-ab8b-5671cf4d328a" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e00ec55a-fe62-4e57-ab8b-5671cf4d328a" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-05e1a207-77a8-47e5-a2ef-8db8722f0ed7" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-05e1a207-77a8-47e5-a2ef-8db8722f0ed7" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-8d44dd38-9800-43d0-bfe6-8407b8971d5c" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-8d44dd38-9800-43d0-bfe6-8407b8971d5c" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-1bb0fbc9-e740-4a07-83a1-e3ad2b03ca67" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-1bb0fbc9-e740-4a07-83a1-e3ad2b03ca67" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-97121739-fffe-4796-89bb-bcf7868975a6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-97121739-fffe-4796-89bb-bcf7868975a6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-05724f21-3554-416a-8ae6-2e38d9e66c5e" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-05724f21-3554-416a-8ae6-2e38d9e66c5e" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-14f38415-0fbb-4324-99e6-2f0d09cc9b19" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-14f38415-0fbb-4324-99e6-2f0d09cc9b19" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-be0edc75-a0a1-4fa4-ad57-776a31b81a77" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-be0edc75-a0a1-4fa4-ad57-776a31b81a77" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-7f2fcab4-6935-415d-acf4-cec709e4d7e7" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-7f2fcab4-6935-415d-acf4-cec709e4d7e7" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e4c5ea72-04c5-4d32-b33a-98d3cc92be9d" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e4c5ea72-04c5-4d32-b33a-98d3cc92be9d" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5dcbc44a-5266-4a2f-91e5-af99586909a0" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5dcbc44a-5266-4a2f-91e5-af99586909a0" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-64e154d0-d360-4b2e-b7ae-54394bab4c81" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-64e154d0-d360-4b2e-b7ae-54394bab4c81" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-29d528f6-5542-4eae-b559-741301288607" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-29d528f6-5542-4eae-b559-741301288607" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e15a09cd-531e-47db-9f0a-fb65db3e6c36" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e15a09cd-531e-47db-9f0a-fb65db3e6c36" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-0f133c85-a1f3-4c4f-85cb-d8e037f8bd2f" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-0f133c85-a1f3-4c4f-85cb-d8e037f8bd2f" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-2fc1dd55-e57d-453e-9c2f-72e09c74e3b1" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-2fc1dd55-e57d-453e-9c2f-72e09c74e3b1" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-f63cee4c-9583-421f-9636-369c1ae93de2" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-f63cee4c-9583-421f-9636-369c1ae93de2" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-230be158-eac1-4030-8ae0-209ab3b2f9fa" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-230be158-eac1-4030-8ae0-209ab3b2f9fa" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-e694721b-53ee-4019-b938-f84c85b43147" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-e694721b-53ee-4019-b938-f84c85b43147" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-3356620c-441d-41e5-b2ba-9855ea172326" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-3356620c-441d-41e5-b2ba-9855ea172326" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-8b81c2be-844a-4a5f-adaa-969abad8f358" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-8b81c2be-844a-4a5f-adaa-969abad8f358" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-ba997868-4792-4646-8a56-79568789aec9" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-ba997868-4792-4646-8a56-79568789aec9" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-df2b863b-b4d1-4713-9550-b70289e50bff" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-df2b863b-b4d1-4713-9550-b70289e50bff" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-1828e691-f501-4a77-9d17-6e61d24ec799" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-1828e691-f501-4a77-9d17-6e61d24ec799" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-bf00b75f-a9d3-4f00-8613-9f465a15ceb8" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-bf00b75f-a9d3-4f00-8613-9f465a15ceb8" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-9924a2b6-eae9-4a04-834c-a96f1d7010ca" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-9924a2b6-eae9-4a04-834c-a96f1d7010ca" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-38ecdf5f-d500-454c-be99-ec359fb5d675" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-38ecdf5f-d500-454c-be99-ec359fb5d675" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-9e260753-2926-4ef5-92a4-e959e21da4d6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-9e260753-2926-4ef5-92a4-e959e21da4d6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-188b9b05-6c17-4ef7-8e08-56f72e836a56" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-188b9b05-6c17-4ef7-8e08-56f72e836a56" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-7a296448-9dce-472e-922c-71983e9a1cd0" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-7a296448-9dce-472e-922c-71983e9a1cd0" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-67bb8647-c5e1-46c9-b439-76cfca20c108" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-67bb8647-c5e1-46c9-b439-76cfca20c108" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-de576992-3573-4486-a13b-0d065bbf5e15" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-de576992-3573-4486-a13b-0d065bbf5e15" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-55d8176a-b3a4-4e29-b6b4-ba4366e3a77a" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-55d8176a-b3a4-4e29-b6b4-ba4366e3a77a" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-838b24fd-50a9-4d5e-9367-db2c3485ab41" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-838b24fd-50a9-4d5e-9367-db2c3485ab41" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-a8259aa8-0a81-48db-a63b-612a0be41fb0" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-a8259aa8-0a81-48db-a63b-612a0be41fb0" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-3e139d32-dd66-4d0b-8d87-902c466bf313" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-3e139d32-dd66-4d0b-8d87-902c466bf313" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-80bed9e7-b3ce-4b48-a456-b0a59d1c7511" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-80bed9e7-b3ce-4b48-a456-b0a59d1c7511" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-2a54285f-e3dd-4528-ae8a-50c48c4ee142" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-2a54285f-e3dd-4528-ae8a-50c48c4ee142" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-cfec6cdc-3809-4497-8bae-261e590d10ca" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-cfec6cdc-3809-4497-8bae-261e590d10ca" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-33b5119e-c15f-457b-8c4b-6f3985127ae6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-33b5119e-c15f-457b-8c4b-6f3985127ae6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-c2e52ba0-a122-47f7-b2a6-4dd4e8a97dc6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-c2e52ba0-a122-47f7-b2a6-4dd4e8a97dc6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5e32d727-534c-4146-8031-762fd7c710bc" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5e32d727-534c-4146-8031-762fd7c710bc" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-d8042dbd-7a15-4d34-bf52-01e4180019c6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-d8042dbd-7a15-4d34-bf52-01e4180019c6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-5312d5b4-382b-4c4d-93fa-efef56802ee6" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-5312d5b4-382b-4c4d-93fa-efef56802ee6" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-4c49dc2b-086c-43dd-8e05-d9850f0c52c2" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-4c49dc2b-086c-43dd-8e05-d9850f0c52c2" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-4e5f2ce4-7a8c-40c8-92ee-96720d3f7f11" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-4e5f2ce4-7a8c-40c8-92ee-96720d3f7f11" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-58d68581-0a9a-440d-b42b-a7540158985b" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-58d68581-0a9a-440d-b42b-a7540158985b" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-120066022-jdd-4baeccea-3a79-4605-8a6d-7e69e500aa28" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-4baeccea-3a79-4605-8a6d-7e69e500aa28" xlink:type="simple"/>

                    <srv:operatesOn uuidref="fr-12006

#2 Updated by Angelo Quaglia over 5 years ago

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [mailto:Robert.Riviere@developpement-durable.gouv.fr]
Sent: 08 April 2015 11:07
To: Angelo Quaglia
Cc: "RIVIERE Robert - CETE Méditerr./DI/ETER"
Subject: Re: from @PNEIG

 

Hello Angelo,

 

thanks for your explanations.

 

 > The geoportal simply takes each URL and tries to retrieve the resource,

 > one by one.

 >

 > How this can possibly take your service down, is unclear to me.

 

Unfortunately, I guess the problem is here. From the logs, I can see

that our server gets about 8,000 - 10,000 requests, with a rate of one

every 2 or 3 seconds. If there is some activity during this period, it

breaks.

 

Would it be possible for you to slow down the rate of the requests ? Or

use some kind of caching system (some URLs are called 4 to 6 times) ?

 

Best regards

 

Robert Rivière

#3 Updated by Angelo Quaglia over 5 years ago

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [mailto:Robert.Riviere@developpement-durable.gouv.fr]
Sent: 09 April 2015 11:28
To: Angelo Quaglia
Cc: "RIVIERE Robert - CETE Méditerr./DI/ETER"
Subject: Re: from @PNEIG

 

Hello Angelo,

 

FYI I just asked our sysadmins to blacklist IP 139.191.1.236

 

Our server has crashed  this night and once again now.

 

It's now clear the portal robot is the cause.

 

We can see that requests were more frequent than in previous days, we

now have 3 to 5 *per* seconds !

And the robot doesn't even check the http code returned, 70% of them are

code 503. Please at least consider to stop when the server itself tells

you it's down.

 

These crashes have also seriously compromised some maintenance

operations scheduled this night.

 

And finally the MD requested are probably already in the portal, since

they are harvested by geocatalogue.fr

 

R Rivière

 

 

#4 Updated by Angelo Quaglia over 5 years ago

From: Angelo Quaglia [mailto:angelo.quaglia@ext.jrc.ec.europa.eu]
Sent: 10 April 2015 17:18
To: 'RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG'
Subject: RE: from @PNEIG

 

Dear Robert,

 

I am sorry for the inconvenience caused. I am about to notify the French National Contact Point for INSPIRE about this issue.

 

The Geoportal harvests only the French National catalogue but metadata documents and service capabilities documents read from the National Discovery Service contain links which point to  http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139

 

The Geoportal sends requests with a maximum rate of 5 simultaneous requests per seconds, which is 6 times less than the minimum required by the Network Service Regulation:

 

See: http://inspire-geoportal.ec.europa.eu/documentation/legal-and-tgs/ns/CELEX_02009R0976-20101228_EN_TXT/?zoomToText=%20simultaneous%20requests

 

 

About the HTTP 503 code, we need to add the handling of that situation. Does your server set the Retry-After header?

 

 

Best regards,

Angelo

 

 

Ing. Angelo Quaglia

External Consultant

European Commission, DG Joint Research Centre

Institute for Environment and Sustainability

Digital Earth and Reference Data Unit, T.P. 262

Via E. Fermi, 2749.

I-21027 Ispra (VA)

Italy

Tel: +39 347 78 88 492

Fax: +39 0332 78 6325

e-mail: mailto:angelo.quaglia@ext.jrc.ec.europa.eu

URL: http://ies.jrc.ec.europa.eu/SDI/sdi-about-us/staff-profiles/angelo-quaglia.html

 

The views expressed are purely those of the writer and may not in any circumstances be regarded as stating an official position of the European Commission.

 

#5 Updated by Angelo Quaglia over 5 years ago

From: LEOBET Marc - CGDD/DRI/MIG [mailto:Marc.Leobet@developpement-durable.gouv.fr]
Sent: 13 April 2015 09:52
To: Angelo Quaglia
Cc: 'Taffoureau Etienne'; 'Michel Millot'; 'Michael Lutz'; 'Freddy Fierens'; roberto.sgnaolin@jrc.ec.europa.eu; RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG; DAVID Benoît - CGDD/DRI/MIG
Subject: Re: french geocatalogue ne peut etre moissone par le INSPIRE geoportal

 

Dear Angelo,

I confirm that the harvesting from Geoportal EU crashed last week our ministry SDI. The problem increased with hours and the ministry decided to blacklist the Geoportal IP as it caused a major issue in the French SDI. It could led to some instability in the French harvesting, too.

The French contact for the ministry SDI is Robert Riviere, in copy. Please contact him to solve this issue as soon as possible and let us know the result.

Regards
Marc Leobet
Spatial Data Office
Directorate of Research and Innovation
Department of the Commissioner-General for Sustainable Development
Ministry of Ecology, Sustainable Development and Energy
France
 

#6 Updated by Marc Leobet over 5 years ago

Dear Angelo,

for French point of view, to check a link found in an "OperatesOn" field is out of the scope of the Network Service Regulation and we did not designed our ministry SDI for that.

Furthermore, we do not understand why you send so many request by second : what is the objective, please? it could be costly for the Member States infrastructure as it is an artificial flow which could skew statistics and could lead to increase the investment.

Best regards

 

LBT

#7 Updated by Angelo Quaglia over 5 years ago

Dear Marc,

the links are found in the metadata that the INSPIRE Geoportal harvests from the Discovery Service registered as the French National Discovery Service for INSPIRE:

Here is an example of those links:

<srv:operatesOn uuidref="fr-120066022-jdd-a3cc793a-0897-40c9-abbe-72c42aeb843d" xlink:href="http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/srv/fre/xml_iso19139?uuid=fr-120066022-jdd-a3cc793a-0897-40c9-abbe-72c42aeb843d"xlink:type="simple"/>

 

That corresponds to the implementation that was agreed upon by the IOCTF concerning the implementation of Coupled Resources:

 

Since it finds such links, the Geoportal assumes that the metadata document is following the Metadata Technical Guidelines.

Consequently, the link is supposed to "point to the metadata record of the data on which the service operates".

It is unexpected that those links to get the metadata from, actually point to a service which is not the National Discovery Service for France.

Therefore, the Geoportal expects the service serving the metadata records to support the minimum performance requirements of Discovery Services described in the Network Service Regulation.

The Geoportal sends requests with a maximum rate of 5 simultaneous requests per seconds, which is 6 times less than the minimum required by the Network Service Regulation.

The number 5 comes from an agreement with Etienne:

 

From: Taffoureau Etienne [mailto:e.taffoureau@brgm.fr]
Sent: 03 September 2014 14:58
To: Angelo Quaglia
Cc: inspire-geoportal@jrc.ec.europa.eu; Marc.Leobet@developpement-durable.gouv.fr; 'Freddy Fierens'; 'Michael Lutz'; 'Michel Millot'
Subject: RE: french geocatalogue ne peut etre moissone par le INSPIRE geoportal

OK, nous allons regarder. Je veux bien que tu modifies pour moi le paramètre en question.

Je laisse Marc faire la demande à Michael pour le changement de contact.

Cordialement,

Etienne Taffoureau

 

There are quite a number of different solutions to this problem but we need to agree on the approach.

 

Best regards,

Angelo

 

#8 Updated by Marc Leobet over 5 years ago

Dear Angelo,

we agree that "we need to agree on the approach." And I join your comment saying that the issue is not only technical. So we would like to understand why the JRC lanches so huge amount of request toward MS's servors. Could you explain why you do that ? From our point of view, consuming 17 % of the allowed capacity to answer requests without users would lead to increase the costs without understandable benefits.

You write that « "It is unexpected that those links to get the metadata from, actually point
to a service which is not the National Discovery Service for France." ». May be we have to clarify our organization. We have not a National discovery service : we have a national focal point, an unifying catalog which only federates the regional or thematic catalogs, themselves often unifying catalogs of lower level. The answer of Etienne Taffoureau is just for the national Geocatalog.

By the way, in France the national Geocatalog does not test all the day the harvested catalogs.

In such organisation, the assumption that the French organizations follow the JRC's technical guidelines cannot be the only hypothesis and, in fact could be true only for a minority, having in mind that there is many ways to understand and implement them.

So the Geoportal would have to manage all the possible implementations let opened by the regulation, and not only one among many.

Best regards

LBT

 

#9 Updated by Michael Lutz over 5 years ago

Dear Marc, 

the discussion around this issue has raised a number of interesting technical and organisational points. We think it would be best if we discussed these in a telecon some time next week. This way it will be easier to agree on a way forward that is acceptable for all of us.

We would be available on Tuesday AM, Wednesday PM or Thursday PM. Please let us know when you would be available for a call (30-45 min should be enough I hope).

Best regards,

Michael, Freddy and Angelo

#10 Updated by Marc Leobet over 5 years ago

Sorry, I take some holydays (even this morning!) and I did not check my e-mails. We are of course open to a talk. We have just to take at least Benoit David and may be Robert Riviere and Etienne Taffoureau. We come back to you next week.

 

Best regards

 

#11 Updated by Angelo Quaglia over 5 years ago

From: LEOBET Marc - CGDD/DRI/MIG [mailto:Marc.Leobet@developpement-durable.gouv.fr]
Sent: 05 May 2015 17:05
To: Michael Lutz; angelo.quaglia@ext.jrc.ec.europa.eu; 'freddy.fierens@jrc.ec.europa.eu'
Cc: DAVID Benoît - CGDD/DRI/MIG
Subject: [Geoportal Helpdesk - Support #2405] French Ministry of Transports: Several requests coming from the INSPIRE Geoportal

 

Dear Michael,

We propose a doodle http://doodle.com/siq8dzk97s7dpw3h to be able to build a common understanding.

I have planified 1 hour, hoping it will be sufficient.

Best regards

LBT
 

#12 Updated by Freddy Fierens about 5 years ago

  • % Done changed from 0 to 100
  • Estimated time set to 1.00

Dear colleagues,

thank you for that swift minutes, and that good agreement. We just would like to add some details (in bold blue).
Please note that the ministry of Transport is actually a part of the ministry of ecology, sustainable development ans energy (ours) and we would like to keep the official name (that's the reason of its huge part in the French SDI).
>>>>>>>>>>>>>>>>

4 agenda points

 

1.       Understanding of the data flow that creates the problem

-          See slides in attachment for an explanation

-          Problems for resolving the problem at the geoportal side:

                                                               i.      Discard all link checks EU-wide will have an impact on the overall quality of the INSPIRE SDI

                                                             ii.      Customized validation behaviour per specific CSW difficult to justify

-          Solution at the national CSW side might be to replace the Operates on URL’s by URL’s pointing to the metadata record in the national catalogue.

2.       Understanding of the problems on the side of the French Ministry of Sustainable development CSW

-          With +/- 5000 metadata records, this service represents one quarter of the French INSPIRE SDI.

-          Is there a real need for INSPIRE metadata to be validated repeatedly (geoportal, EEA dashboard, other applications)? This has a big impact on the resources of the systems that are being (mis)used only for checking things.

-          Marc points to (lack of) legal status of the Technical Guidelines and the value to offer to end-users even non-TG compliant informations, Michael to the resource implications for adapting to services that deviate from these guidelines. A pragmatic middle ground needs to be found.

- Marc & Benoit announce that the FR ministerial implementation seems to be original and will be studied to become more resilient.

3.       MIG-T is proposing to use the UUID as reference to a metadata record instead of the xlink:href. This would allow to rechannel the link checks to the national INSPIRE discovery service instead of to the local CSW where the URL points to. Currently however, this is only in proposal phase.

4.       Specificities of the French implementation

-          Not discussed due to time constraints. 

 

Proposed (temporary) solution to the problem of the French Ministry of Sustainable development CSW:

Further reflection shows that:

-          Suspending the EU harvester/validator checking of the links can only be done per national INSPIRE discovery endpoint (so for all French services, not only for the Ministry of Transport CSW)

-          keep the blacklisting active would limit the effect of suspending the checks to the Ministry of Sustainable development CSW.

 

So we would suggest to go for the second option, i.e. maintain the blacklisting of the Geoportal rather than adapt the software to suspend the checks for France.

 
 Furthermore : a result of MIWP-5 would be to help to clarify this type of issue.
After the summer, FR will propose to the MIG-T to study the understanding of criteria (performance/capacity) of the NS regulation through EU.

 >>>>>>>>>>>>>>>>>


Best regards
Marc Leobet
Spatial Data Office
Directorate of Research and Innovation
Department of the Commissioner-General for Sustainable Development
Ministry of Ecology, Sustainable Development and Energy
France

 

#13 Updated by Angelo Quaglia almost 4 years ago

  • Category set to Harvesting process

#14 Updated by Angelo Quaglia almost 4 years ago

  • Subject changed from French Ministry of Transports: Several requests coming from the INSPIRE Geoportal to FR - PNEIG: Several requests coming from the INSPIRE Geoportal
  • Submitting Organisation changed from French Ministry of Transports to FR - PNEIG

#15 Updated by Angelo Quaglia almost 4 years ago

  • Country set to FR - France

#16 Updated by Angelo Quaglia over 3 years ago

  • Proactive set to No

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [Robert.Riviere@developpement-durable.gouv.fr]
Sent: 24 February 2017 16:28
To: Angelo Quaglia
Cc: "RIVIERE Robert - CETE Méditerr./DI/ETER"
Subject: Re-opening Geo-IDE metadata service to inspire geoportal

Hello Angelo,

we exchanged some mails almost 2 years ago about some problems we had in
Geo-IDE application (*) when the inspire geoportal requested numerous
metadata from it (for details you can see below).

We have now made some changes in our application and we believe it
should now answer correctly.

But before definitively opening back the network service, we'd like to
do some tests with you.
Would it be possible to agree on a period of time where :
- we would allow the robot to request
- you start the robot for a resource retrieval session
- when terminated, we close back the service, and analyse the logs.

If everything goes well, we'll then be able to re-open the service.

I guess a couple of days of tests should be enough. Does this seem
possible for you ? Could we plan this for the beginning of March ?

Thanks in advance,

Robert Rivière
MEEM, Centre de Prestations et d'Ingéniérie Informatique
Aix en Provence

(*) Geo-IDE : French Ministery of Sustainable Development SDI

#17 Updated by Angelo Quaglia over 3 years ago

From: QUAGLIA Angelo (JRC-ISPRA-EXT)
Sent: 27 February 2017 09:46
To: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG
Subject: RE: Re-opening Geo-IDE metadata service to inspire geoportal

Dear Robert,

I am glad to hear that.

This week is going to be very busy for me.
Would next week suit you?


Best regards,
Angelo

#18 Updated by Angelo Quaglia over 3 years ago

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [Robert.Riviere@developpement-durable.gouv.fr]
Sent: 27 February 2017 11:24
To: QUAGLIA Angelo (JRC-ISPRA-EXT)
Subject: Re: Re-opening Geo-IDE metadata service to inspire geoportal

Hello,

yes, no problem.

Around March 8th would be ok ?

Greetings,

Robert

#19 Updated by Angelo Quaglia over 3 years ago

From: QUAGLIA Angelo (JRC-ISPRA-EXT)
Sent: 27 February 2017 12:12
To: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG
Subject: RE: Re-opening Geo-IDE metadata service to inspire geoportal

March 8th will be fine for me, thanks.

Cheers,
Angelo

 

#20 Updated by Angelo Quaglia over 3 years ago

From: QUAGLIA Angelo (JRC-ISPRA-EXT)
Sent: 07 March 2017 09:14
To: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG
Subject: RE: Re-opening Geo-IDE metadata service to inspire geoportal

Dear Robert,
when you wish I can kick off the harvesting and you should then be seeing the requests coming.

If you could take the time to create an account here:
https://webgate.ec.europa.eu/cas/eim/external/register.cgi

I could enable you on https://ies-svn.jrc.ec.europa.eu/issues/2405
so that I do not have to copy and paste from emails any more?


Best regards,
Angelo

#21 Updated by Angelo Quaglia over 3 years ago

From: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG [Robert.Riviere@developpement-durable.gouv.fr]
Sent: 08 March 2017 11:35
To: QUAGLIA Angelo (JRC-ISPRA-EXT)
Subject: Re: Re-opening Geo-IDE metadata service to inspire geoportal

Hello Angelo,

well, could you launch a test at 2:00 PM (french time, 01:00 PM UTC) ?
Everything should be ready.

 > If you could take the time to create an account here:
 > https://webgate.ec.europa.eu/cas/eim/external/register.cgi

Done, my uid is Robert.Riviere@developpement-durable.gouv.fr

Thanks,

Robert

#22 Updated by Angelo Quaglia over 3 years ago

From: QUAGLIA Angelo (JRC-ISPRA-EXT)
Sent: 08 March 2017 11:54
To: RIVIERE Robert (Responsable PNE IG) - SG/SPSSI/CPII/DOM/ET/PNE IG
Subject: RE: Re-opening Geo-IDE metadata service to inspire geoportal

Hi Robert,
that sounds good.
I will kick it off at 2pm unless you tell me otherwise.
I have created your user and you should be able to update the issue and receive notifications:
https://ies-svn.jrc.ec.europa.eu/issues/2902

Best regards,

Angelo

#23 Updated by Angelo Quaglia over 3 years ago

It is running.

#24 Updated by Robert Riviere over 3 years ago

Ok I can see requests coming. So far everything's going fine. 

#25 Updated by Robert Riviere over 3 years ago

Job seems to have ended at about 4:05 PM. 

25123 hits with user-agent "Apache-HttpClient/4.4.1 (Java/1.8.0_45)" (I guess it's your robot (*)), around 100 hits/min.

Our server's still running fine with its nominal response time.

So everything went well from our side. Is it OK for you ?

(*) BTW it would be nice if the robot's requests had a specific User-Agent (like "Inspire Geoportal" or whatever)

Thanks

#26 Updated by Angelo Quaglia over 3 years ago

That is great news, congratulations!

Indeed, the last request I see, for the time being, is: 

08 Mar 2017 16:05:54,427 (OGCSRequest.java:543) -  INFO pool-1329581-thread-1 eu.europa.ec.inspire.geoportal.ogc.OGCSRequest - built request URL: http://catalogue.geo-ide.developpement-durable.gouv.fr/catalogue/apps/search/?uuid=fr-120066022-srv-b90000a5-d626-48f0-9c3a-91604f9743f0

 

I wrote "for the time being" because the process is still running.

It has processed about 8500 out of the 33800 metadata made available by http://www.geocatalogue.fr/api-public/inspire/servicesRest (Geocatalogue Catalogue Server INSPIRE of BRGM).

Not all metadata documents describe resources that end up referencing your service and it really depends on the order in which the metadata are received from BRGM.

So, I cannot guarantee your service will not be contacted again.

Thierry Vilmus (in copy) of BRGM might have greater insight on this.

I will notify you as soon as the process has finished.

Best regards,

Angelo

 

 

#27 Updated by Angelo Quaglia over 3 years ago

A new wave started at 08 Mar 2017 17:31:02,399 until 08 Mar 2017 17:52:55,588

#28 Updated by Angelo Quaglia over 3 years ago

Other bursts of activity:

08 Mar 2017 19:08:22,958 - 08 Mar 2017 19:34:23,790

08 Mar 2017 21:15:51,844 - 08 Mar 2017 21:16:25,456

08 Mar 2017 21:52:12,806 - 08 Mar 2017 21:52:45,091

08 Mar 2017 21:59:36,730 - 08 Mar 2017 22:31:23,374

08 Mar 2017 22:58:40,752 - 08 Mar 2017 23:59:24,556

 

 

#29 Updated by Robert Riviere over 3 years ago

Okay, I can also see some activity till 4:30 AM. The ATOM and OGC servers have also been requested.

Nevertheless, we can't see any impact on our servers' wealth.  

I'm going to keep an eye on them for a few days, but I think we'll soon be able to close this so old ticket :-)

Greetings

Robert

#31 Updated by Robert Riviere almost 3 years ago

Hi Angelo,

sorry but we are facing problems since a recent upgrade.

We expect to recover and resume operations next week

Best regards,

Robert

#32 Updated by Robert Riviere almost 3 years ago

Hello,

Geo-IDE CSW server is back online.

Greetings,

Robert

#33 Updated by Angelo Quaglia almost 3 years ago

Dear Robert,

many thanks and best regards,

Angelo

#34 Updated by Angelo Quaglia over 2 years ago

  • Status changed from Feedback to Resolved

#35 Updated by Angelo Quaglia over 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF