A good validation test (suite)
- helps service and data providers in fulfilling a predefined, commonly agreed set of useful requirements,
- can be used to automatically and meticulously pinpoint errors that should be obvious to the careful expert eye,
- can be used as a meter for specification compliance,
- can be used for learning the details of the specification requirements (the hard way),
- cannot be used for fixing vague, unclear or ambiguous requirements,
- cannot be used for justifying the existence of the original requirements.
Thinking of implementing good tests for a selected set of requirements will help in clarifying and classifying them: Can this requirement be interpreted in more than one way? Is this statement a requirement for:
- an organization or a person,
- a software product,
- a data set or metadata item, or
- a service instance.
Meetings of the MIWP-5 sub-group¶
- 11th MIWP-5 meeting (virtual) / ETS Demo webinar, 3 November 2016
- 10th MIWP-5 meeting (virtual) / ATS comment resolution web-conference #2, 28 April 2016
- 9th MIWP-5 meeting (virtual) / ATS comment resolution web-conference, 15 April 2016
- 8th MIWP-5 meeting (virtual), 16 March 2016
- 7th Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 20 January 2016
- Joint MIWP-5-ARE3NA meeting on planning the development of the validator/ETS, 16 December 2015
- 6th Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 9 December 2015
- 5th Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 16 November 2015
- 4th Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 16 October 2015
- Status update on the ATS drafting contract (Ilkka Rinne)
- Physical informal meeting MIG temporary sub-group on Validation and conformity testing, 29 May 2015
- 3rd Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 24 March 2015
- 2nd Virtual Meeting MIG temporary sub-group on Validation and conformity testing, 15 January 2015
- Virtual kick-off Meeting MIG temporary sub-group on Validation and conformity testing, 12 December 2014
Workshops and webinars¶
- Workshops on ATSs for metadata and network services
- Status meeting 09.02.2015
- Status meeting 19.01.2015
- Status meeting 19.12.2014
- Virtual kick-off meeting for the preparation team, Skype, 20 November
- Workshop on validation of metadata and network services, Ispra, 2-3 December 2014
- INSPIRE Workshop on validation and conformity testing, 15-16 May 2014
MIWP-5 work packages and tasks¶
1.1 Define scope (ToR) - closed¶
1.2 Policies and procedures - active¶
During the adhoc meeting in Aalborg on 16th June I (Peter Parslow) took an action to draft a change control process for the validation suite, by the end of July, for an interested sub-group to consider. This work forms part of task 1.2 in the draft project plan. My notes suggest that we intend to produce a draft by September 2014 & a process by the end of December 2015. I remember suggesting we (WP5) should raise this with the MIG, as we consider change control to be important more widely than this WP. I can't find any formal notes of the meeting, and don't have a record of who was interested.
I have uploaded an initial draft: https://ies-svn.jrc.ec.europa.eu/documents/26
It is based on the IT Infrastructure Library change management process; ITIL is formalised as ISO 20000; it describes 'change management' as "The process responsible for controlling the lifecycle of all changes, enabling beneficial changes to be made with minimum disruption to IT services".
1.3 Compliance certification - closed¶
Outcome of the workshop in Lisbon at the INSPIRE conference 2015: currently no need of certification facilities and processes for INSPIRE compliance
1.4 Establish INSPIRE testing maintenance - open¶
to be specified (implementation of 1.2)
2.1 Define use cases for (a) commonly agreed validator(s) - active¶
#2394 - Documentation of use cases
#2448 - Draft use case of common validation
#2611 - User stories
List of the validation related use cases (Initially by Ilkka Dec 2015 ->)
2.2 Specification of (functional and non-functional) requirements based on use cases - active¶
This task is crucial to support the work of the JRC contractor that will develop the INSPIRE testing framework.
This page can be used to collect requirements for the testing framework.
2.3 Existing validation tools/platforms and approaches, including languages/approaches for documenting tests - done¶
2.4/2.5/3.x.3 Design & implementation of INSPIRE testing framework and ETS - active¶
This task will be carried out by a JRC contractor. See an extract of the draft technical annex to the request for offer [Request_for_offer_validation.pdf], which specifies the terms of reference for this work. This document also specifies how the MIWP-5 is expected to contribute to and review the work of the contractor.
See another wiki page for description/discussion of the INSPIRE Testing framework conceptual data model and the high-level validator software components.
2.6 Collect schema and schematron existing examples from Thematic Clusters and proposed in MIWP-14 - open¶
3.x.1 Develop ATS (MD, NS, SDS) - active¶
3.x.2 Compare existing implementation of IR requirements (MD, NS) - active?¶
3.x.3 Map requirements in IR to requirements in TG (and identify gaps and inconsistencies) (MD, NS, SDS) - active?¶
3.5 Investigate feasibility of testing INSPIRE data sets - active¶
Notes on using OGC CITE (cite.opengeospatial.org)¶
We (both Ordnance Survey GB & eENVPlus - Stefania Morrone) ran INSPIRE based GML application schema through the CITE GML3.2.1 test (r15). http://inspire.ec.europa.eu/schemas/base/3.2/BaseTypes.xsd failed, because the test didn't allow for a metadata property to use an anonymous type.
OGC CITE team have noted ((https://github.com/opengeospatial/ets-gml32/issues/1) 'although unusual this is permitted', and have/will fix it in r16 of the GML 3.2.1 test.