Task #2215

MIWP-8 (E-1) Integrate Theme specific metadata

Added by Michael Östling almost 6 years ago. Updated about 5 years ago.

Status:SubmittedStart date:17 Sep 2014
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Proposed change or action:

Description

This section has now been locked for editing and a draft version have been moved into the draft document.
See page:
https://ies-svn.jrc.ec.europa.eu/projects/metadata/wiki/DraftVersionsMIWP-8
Further discussions and comments should be done on the draft

*WIKI:* https://ies-svn.jrc.ec.europa.eu/projects/metadata/wiki/MIWP-8_(E-1)_Integrate_Theme_specific_metadata Right now there are number of optional metadata elements spread out over a number of INSPIRE Data Specifications. The TG Metadata will be updated to include those “optional” metadata elements within and updated TG because some data providers are producing theme specific data services already. This will allow those organizations that wish to use the optional elements to have appropriate 19115 elements in the metadata for this purpose. An overview of what optional elements that are defined in each dataspecifcation would also be beneficial. Then it must decided how should such an overview should published and managed.

2015-01-22_metadata_theme_specific_INSPIRE_19115-IdV.xlsx (42.2 KB) Ine de Visser, 30 Jan 2015 01:27 pm

History

#1 Updated by Michael Östling over 5 years ago

  • Description updated (diff)

#2 Updated by Peter Kochmann over 5 years ago

  • Description updated (diff)

In issue D (metadata for interoperability) I uploaded file is a draft of a mapping between metadata elements for interoperability focussed by INSPIRE (named in current release of TG metadata) and existing ISO 19115 (19157 FDIS 2011 for the last element). I tried to have all involved elements nearby the INSPIRE metadata element to show precisely what is needed to register. Therefore I looked up the data specifications again (current release).

If no one else started a similar work I can do this for the theme specific metadata as well.

#3 Updated by Peter Kochmann over 5 years ago

My thoughts concerning the Italian proposal of considering additional elements:

  • maintenanceAndUpdateFrequency is already involved by "Maintenance Information", which is optional. In my opinion there is no other way than using maintenanceAndUpdateFrequency. It will get along with updateScope and maintenanceNote.
  • PositionalAccuracy is one of the possible forming of dataQuality. So it's up to us to consider all possibilities without naming single ones explicitly.
  • processStep and source (via LI_Lineage) are definitely in! They are used at least in elevation, orthoimagery and buildings.

#4 Updated by Michael Östling over 5 years ago

When discussing this ticket can we define the possible ways forward and what we suggest from group.
I can see the scenarious below. Are there other ways we think we should go ?
 
A
An appendix is added to TG listing the optional elements defined in various dataspecification similar to what is attached to ticket wiki right now.
 

B
We add A as mentioned above but we also suggest to add some elements new elements from dataspecifications. These elements are optional but we suggest to add them to TG Metadata so that
IF they are used they are handled in a similar for each theme using the element(s)  and we get better interoperability for the elements.
 

Peter: Just so I get your  query.
Regarding the various DQ_Elements:  " So it's up to us to consider all possibilities without naming single ones explicitly."
Did you mean that we add a general clause on DQ_reports and how to write these but do not specificially list all elements?

 

I would agree to add maintenanceAndUpdateFrequency  (together with updateScope and maintenenceScope). We already have this in our national profile in Sweden.
Also processStep and lineage would be useful.
PositionalAccuracy is good. But we (in Sweden) would need more general guidelines on how to use the DQ_reports and not just for this DQ-Element.

#5 Updated by Peter Kochmann over 5 years ago

I guess it's necessary at least to get the given information out of data specifications (A). When matching it with ISO 19115 we will find some more elements not clearly named by data specification, but maybe useful. We should prove every particular case and maybe consider single elements in addition (B).

I started to build up a new table matching data specifications with ISO 19115 considering all elements that can be found in ISO when following the given structures. The first example is Maintenance information: From data specification you will find that it is matched with resourceMaintenance which uses MD_MaintenanceInformation. It's mentioned that "at least the following elements should be used". These are maintenanceAndUpdateFrequency, updateScope and maintenanceNote. But MD_MaintenanceInformation itself has some more optional elements: dateOfNextUpdate, userDefinedMaintenanceFrequency, updateScopeDescription and contact. It's on us to decide whether we consider them too!

Please have a look at this first draft. I can continue this work fur discussing (especially the yellow marked cells!) if you all agree.

Concerning dataQuality: I'm not familiar with ISO 19157. As far as I know there are several ways to report dataQuality: quantitative results or descriptive results. Both ways mentioned in data specifications and I propose to do the same (not going into details). But there is a mix with references to ISO 19115 for reporting dataQuality as well in single data specifications (e.g. Positional Accuracy in theme orthoimagery). It's clearly named to use DQ_GriddedDataPositionalAccuracy from ISO 19115. In this case we have to consider that certain way too.

 

#6 Updated by Pierluigi Cara over 5 years ago

I definitely agree with option A. I find interesting option B and the method proposed by Peter is very good, but It's challenging: are we able to do all this work in our short time? Maybe we should take a more realistic decision. To help us in making decision, I propose to adopt another point of view (functional approach): we could try to realize option B taking into account only the optional metadata coming from DS metadata already present in current processes. With current processes I mean the national profile implemented by the EU State Members. Another example of process could be the (operative) Copernicus Project: which metadata could be useful for that project? For example the Italian Civil Protection Department (where i actually work) has activated serveral times the Emergency section of Copernicus and I've never seen any metadata in the final delivered datasets.

#7 Updated by Marc Leobet over 5 years ago

We have a different approach in France, following the first huge feedback from public authorities. First, they think too complex and too costly to build metadata - and I talk only of mandatory ones. Secondly, the mandatory fields are not well filled, and sometimes important information (lineage...) misses.

In our point of view, we do not need to heavily update European TG before to be sure than public authorities use them properly. That's not the case, typically for important value as Lineage, Resolution, access and use...

We decided (in FR) to have TG explaining how to get INSPIRE MD compliant with ISO 19115. For those who want to go further, they would have to take ISO standards, and that is the job of ISO to draft TG for them. It was just a way to be able to focus on core metadata.

A word about Data quality : I check who is using ISO 19157 in France : except national mapping agencies, I found nobody. What is the value to draft very smart chapters on that subject? Since Istanbul Workshop, we know that this is an issue for rechearch, not for implementation. That's the reason for B priority in the ToR.

I keep in mind that the best MD are those we are able to really get. The example of Emergency section of Copernicus shows that, instead to be more detailled, we need effectiveness, even if it could lead to drop smartness.  

#8 Updated by Michael Östling over 5 years ago

In this ticket we are not planning to add any mandatory tickets. The discussion is to possibly add optional elements that exists in multiple data specifications
so that their use are described in a common way. It will not force anyone to add these elements. But if they use them there will be unified guidlines for them.

Regarding the ISO 19157. The current task within MIWP-8 is not primarily to include ISO 19157 into the tecnical guidelines. But we need to describe the current
situation since ISO 19157 is implemented in Inspire Data specfications. See comments on:https://ies-svn.jrc.ec.europa.eu/projects/metadata/wiki/MIWP-8_(E-2)_Integrate_Theme_specific_metadata

 

#9 Updated by Ine de Visser over 5 years ago

The example by Peter is clear and let us see where the discussions are. I think we need such an overview of all metadata elements mentioned in the dataspecs and find a way to present them in an annex of te TG MD.  We can start work on  the examples based on the number of specifications they are mentioned, metadata elements mentioned in a lot of specifications first. I prefer recommandation of specific optional elements, but i wonder if it is feasable. We can decide on that if we have an overview. 

#10 Updated by Peter Kochmann over 5 years ago

I continued my work on the Excel sheet already and finished it for all theme specific metadata elements regarding ISO 19115 (see uploaded file from today). There are some issues left:

We have to discuss all yellow fields and decide which of the optional elements are needed.

In theme geology there are some facts mentioned concerning details on lineage. But they are not mentioned under the topic of "theme specific metadata". Do you agree that we do not have to consider this?

I'm still not sure how to deal with the metadata elements concerning data quality and focussing on ISO 19157. What solution or recommendation can be given in Technical guidance metadata? Is anyone familiar with ISO 19157?

#11 Updated by Pierluigi Cara over 5 years ago

Great job Peter, it's really helpful.

About yellow fields it's difficult to decide what to do. We probably have to set up some uniform criteria.

About the elements not mentioned under the topic "theme specific metadata" it's seems to me too restrictive. One possible criteria could be to take into account all the indication regarding metadata contained into "Reccomendation *".

About ISO 19157, I'm not familiar too (I also haven't got any copy of that document), I agree with Michael that is not primarily to include ISO 19157 into the tecnical guidelines.

#12 Updated by Peter Kochmann over 5 years ago

I'm sorry I've detected another question:

in theme Protected sites there's a table of mandatory and conditional theme-specific metadata in chapter 8 (table 10). It names the four well-known elements Coordinate reference system, Temporal reference system, Encoding and Character encoding (they are handled in issue D: metadata for interoperability).

But scrolling down to the details you can find five of them: in addition there's named Application schema which is given mandatory!

There's no other theme that considers this element. So do we have to consider it as theme-specific metadata?

#13 Updated by Darja Lihteneger over 5 years ago

Dear Peter,

I was looking into the INSPIRE PS Technical Guidelines and found this metadata element (Application schema, page 57) in the technical guidelines version 3.1. (http://inspire.ec.europa.eu/documents/Data_Specifications/INSPIRE_DataSpecification_PS_v3.1.pdf ). This technical guidelines was updated and the latest version 3.2 (http://inspire.ec.europa.eu/documents/Data_Specifications/INSPIRE_DataSpecification_PS_v3.2.pdf ) doesn't include this metadata element. Could we clarify if this is correct?

The purpose of metadata element Application schema in INSPIRE PS 3.1 was to provide the application schema that the data set complies with. Currently, PS includes only one IR application schema, this is Simple application schema.

In INSPIRE PS 3.2, the conformity metadata element (Conformity, page 35) can be used to document the conformance with the data specification as a whole, with a specific conformance class defined in ATS (for example: Application schema conformance class) or with another specification. I suppose this approach replaced the previously used metadata element Application schema. Could we clarify if this is the case?

#14 Updated by Peter Kochmann over 5 years ago

Oops! Thanks a lot for giving that hint! I didn't realize there was an update of annex I themes at all! I'll check this and update my excel sheets ...

#15 Updated by Peter Kochmann over 5 years ago

Darja Lihteneger schrieb:

The purpose of metadata element Application schema in INSPIRE PS 3.1 was to provide the application schema that the data set complies with. Currently, PS includes only one IR application schema, this is Simple application schema.
In INSPIRE PS 3.2, the conformity metadata element (Conformity, page 35) can be used to document the conformance with the data specification as a whole, with a specific conformance class defined in ATS (for example: Application schema conformance class) or with another specification. I suppose this approach replaced the previously used metadata element Application schema. Could we clarify if this is the case?

I guess the intention is the same: to have an opportunity for documenting accordance with a certain schema. In DQ_ConformanceResult it's possible to have a citation of a schema. In MD_ApplicationSchema there's also a chance to have the schema file included. Maybe the Drafting team PS didn't see that as needed. I can't speak for them.

Anyway: My proposal is to consider the new PS data specification only, where nothing can be read of MD_ApplicationSchema.

#16 Updated by Peter Kochmann over 5 years ago

Pierluigi Cara schrieb:

About yellow fields it's difficult to decide what to do. We probably have to set up some uniform criteria.

I agree to "uniform criteria". This could mean the existence of all elements being mentioned in any data specifications (maybe just one of them) and all elements required by ISO, right?

This would cover all differences and give a common theme specific element, which does not defer from one theme to another. E.g. "Digital transfer options information" occurs in themes Elevation, Orthoimagery and Hydrography. Both Elevation and Orthoimagery name "unitsOfDistribution" and "offLine" as ISO-elements of MD_DigitalTransferOptions to be used. Hydrography names "transferSize" instead. In TG we would consider all three of them.

#17 Updated by Peter Kochmann over 5 years ago

Peter Kochmann schrieb:

Oops! Thanks a lot for giving that hint! I didn't realize there was an update of annex I themes at all! I'll check this and update my excel sheets ...

There's nothing new in it to be considered here. The clearing in theme protected sites (application schema is out now) seems to be one of the most relevant changes.

One mistake (in my opinion) is still in. In theme addresses there's an element given named "Spatial Representation type" (8.3.3 in table 5, page 70). The details on page 72 give ISO 19115 No. 12 (spatialRepresentationInfo) and MD_SpatialRepresentation, which seems wrong to me, because they match it with spatialRepresentationType (37) later on. We already have considered that element for the metadata for interoperability in MIWP-8 (D)! This is a mandatory element for all annex themes! So this is redundant here!

#18 Updated by Peter Kochmann over 5 years ago

When looking at the Terms of Reference of our group I got aware that we will handle all theme specific metadata elements concerning IS0 19157 in package E-2 (I assume that applies to all elements given in data specifications in chapters 8.3.2)

@Michael: Could you please give the designated name to package E-2? At the moment it is similar to E-1!

So I looked up the data specifications again and found no element to add to the existing Excel file. I just fixed some smaller things.

For my understanding all other theme specific metadata elements will work only when using ISO 19157 (and 19115-1). So we do not have to find a matching with existing 19115 elements instead of 19157 here (as we have to in package D).

#19 Updated by Ine de Visser over 5 years ago

I not agree with your conclusion, not to add the theme specific metadata elements on data quality. All those elements, listed in the table at 8.3 are exsisting ISO 19115:2003 data quality elements. If we don't want to force people to use the ISO 19115:2014 in combination with ISO 19157 (not even possible because the ISO 19115-3 is not ready), we should provide some intructions.

#20 Updated by Peter Kochmann over 5 years ago

My intention was not to ignore data quality elements at all, but the perspective given in MIWP-8 (E-2) ("Review applicability of ISO 19157 for theme specific metadata) let me see the focus on elements not concerning data quality here.

Anyway: Trying to match the data quality elements means the same problem we have in MIWP-8 (D) with Topological Consistency: Only one of the two given ways for documenting data quality (quantitative results or descriptive results) can be done in 19115 as well (the quantitative one).

So for quantitative results there is no problem. All required structures and elements can also be found in existing ISO 19115.

For reporting descriptive results: There is no DQ_DescriptiveResult in ISO 19115 and I do not see any matching elements otherwise. So do we skip the descriptive way and for now focus on quantitative results only?

#21 Updated by Marc Leobet over 5 years ago

I support Peter's remark to focus on quantitative results only.

I added in Issue 2214 #34 the extract about FR TG Topoligal consistency, and I recall my post in #2202 #25 about inexistence of ISO19157 in the regulations.

About ToR, I assume that we have (only, if I dare) to "Review applicability of 19157", furthermore with a low priority. We could conclude to a non-applicability of ISO19157 in an user context (lack of maturity and lack of tool).

 

Best regards

#22 Updated by Ine de Visser over 5 years ago

Yes, that's a good solution, to focus on the quantitative results.

#23 Updated by Ine de Visser over 5 years ago

I found this recommandation in the dataspec on elevation;

Recommendation 39 Following the ISO/DIS 19157 Quality principles, if a data provider has a procedure for the quality management of their spatial data sets then the appropriate data quality elements and measures defined in ISO/DIS 19157 should be used to evaluate and report (in the metadata) the results. If not, the Lineage metadata element (defined in Regulation 1205/2008/EC) should be used to describe the overall quality of a spatial data set.

So the discriptive quality results should be placed in the lineage element

#24 Updated by Ine de Visser over 5 years ago

I added comments in the attached table

#25 Updated by Marc Leobet over 5 years ago

Dear Ine,

in your last post (the .xslx file), I am confuse as I found no reference to any of these "theme specific INSPIRE metadata elements" in the Regulations. Could you help me?

I agree with you for your n-1 post, about placing descriptive quality result in the lineage element. It is the way the Regulation 1089/2010 states for Elevation Requirements on data quality and consistency 

Thanks

#26 Updated by Ine de Visser over 5 years ago

The elements in the table made by Peter are the theme specific elements as described in each dataspecification at chapter 8.3

#27 Updated by Peter Kochmann over 5 years ago

Ine de Visser schrieb:

I found this recommandation in the dataspec on elevation; Recommendation 39 Following the ISO/DIS 19157 Quality principles, if a data provider has a procedure for the quality management of their spatial data sets then the appropriate data quality elements and measures defined in ISO/DIS 19157 should be used to evaluate and report (in the metadata) the results. If not, the Lineage metadata element (defined in Regulation 1205/2008/EC) should be used to describe the overall quality of a spatial data set. So the discriptive quality results should be placed in the lineage element

Yes, this recommendation can be found in every data specification in chapter "8.1.2 Lineage". It's a kind of reverse conclusion for our problem to match the quantitative and measured elements (designated in ISO 19157) to the existing ISO 19115 elements and put the descriptive ones (not matching ISO 19115 exactly) to the more general lineage element.

I will try to consider it in the Excel sheet in a general way for all mentioned data quality elements.

#28 Updated by Peter Kochmann over 5 years ago

The revised table considering lineage for the descriptive results of data quality elements is still in progress. Sorry for that.

Does anyone else have comments on the optional elements found in ISO (the yellow marked ones)? Otherwise I propose to consider the mandatory and optional ones only and of course the ones given in examples in data specifications.

According to that another issue: If data specifications give statements like "At least the following elements should be used ..." we take it as mandatory for the theme specific metadata though ISO doesn't see it that way, right?

#29 Updated by Peter Kochmann over 5 years ago

Peter Kochmann schrieb:

Otherwise I propose to consider the mandatory and optional ones only and of course the ones given in examples in data specifications.

Of course it has to be "the mandatory and conditional ones ..."

#30 Updated by Peter Kochmann over 5 years ago

As announced above I finished the Excel sheet and considered the mandatory and conditional elements of ISO 19115 only and in addition the elements given by implementing instructions or examples in chapter 8 in data specifications. Elements named in context of the statement "At least the following elements should be used ..." I considered as mandatory for theme specific metadata now.

I considered the data quality elements in general. The concept is the same for all of them.

I'm still not sure how to show the content of the Excel sheet in a suitable style for the TG draft. At least we could have a describing text and name the XPaths explicitly.

See Wiki page to find the uploaded file.

#31 Updated by Michael Östling over 5 years ago

I made two example pdf's of the current content of excel-file and attached them to wiki
https://ies-svn.jrc.ec.europa.eu/attachments/download/993/2015-03-09_metadata_theme_specific_INSPIRE_19115_2.pdf
https://ies-svn.jrc.ec.europa.eu/attachments/download/992/2015-03-09_metadata_theme_specific_INSPIRE_19115.pdf

This would be one way to display the contents of this table.
Either get an overview of the Implemention or get an overview on where it's used in Annnexes.

Xpaths would be needed I think and also XML-examples for each group  of elements would also generate a better understanding.
Can we manage to add also that?

#32 Updated by Peter Kochmann over 5 years ago

By now I didn't think of having the Excel table itself in the TG document. My intention was to have an overview of the required elements.

Of course we can have the XPaths in addition. But we have to create a text in TG style as well, right?

#33 Updated by Peter Kochmann over 5 years ago

What hasn't been documented by now in the Excel table is the different view on data quality depending on the several INSPIRE data themes. So the mapping for data quality still is incomplete! How do we document that the data quality concerning e.g. Geographical names focusses on four issues (Conceptual Consistency, Domain Consistency, Completeness – Omission, Absolute or External Accuracy) and e.g. Administrative units focusses on one more issue (Completeness – Commission)? Do we need an explicit mapping for this in TG?

#34 Updated by Peter Kochmann over 5 years ago

I've just uploaded a new Excel file that now includes a mapping for the several data quality issues and the concerned annex themes in a second table.

Can someone check it, please? I don't want to be the only one who had a view on it.

I focussed on the information given in chapter 8 in data specification. Sometimes there is given a hint on chapter 7 where data quality itself is described. I considered further information from chapter 7 only when the level for that information was dataset or series, not spatial object!

#35 Updated by Michael Östling over 5 years ago

I support the use of Lineage as an alternative for reporting descriptive results.
It could though be hard to get the structure if all is just a plain mass of text.
If reporting is needed on multiple DQ_Elements eg: Absolute or external accuracy, Temporal Consistency, Conceptual Consistency.

Would it be possible to recommend using some kind of prefix for text so its easier to extract info?

Absolute or external accuracy:
Lorem Ipsumm ...

Temporal Consistency:
Lorem Ipsumm ...

Conceptual Consistency:
Lorem Ipsumm ...
 

 

#36 Updated by Peter Kochmann over 5 years ago

Yes, I think this would be very helpful!

Lineage is used in chapter 2.7.1 anyway and it's mandatory. So an introductory prefix would allow a separation between

  • the content of lineage concerning article 5 (2) of INSPIRE regulation --> 2.7.1 in TG
  • the content of lineage alternatively used for descriptive results in metadata for interoperability, here: Topological consistency (future chapter 2.12.6 in TG)
  • the content of lineage alternatively used for descriptive results in theme-specific metadata (future chapter 2.13.x in TG)

I agree to have a recommendation here and in 2.12.6 as well!

#37 Updated by Michael Östling about 5 years ago

  • Description updated (diff)

Also available in: Atom PDF