Re3gistry software v1.1 - Documentation

Index

  1. Background
  2. Information Model
  3. System description
  4. Serving the registry content
  5. Technical description
  6. System installation and first run
  7. Creating new modules

 

Quick installation guide

https://ies-svn.jrc.ec.europa.eu/projects/registry-development/wiki/Re3gistry_1_1_quick_install

Quick customisation guide

https://ies-svn.jrc.ec.europa.eu/projects/registry-development/wiki/Re3gistry_1_1_quick_customisation

 

Disclaimer

The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.

Acknowledgements

As part of the Interoperability Solutions for European Public Administrations (ISA) Programme[1], the European Commission’s (EC) Joint Research Centre (JRC) is establishing a Reusable INSPIRE Reference Platform  (ARE3NA) which is identifying and developing common components for the successful implementation of the INSPIRE Directive.

The work on the Re3gistry addresses a missing component of INSPIRE as an open source solution for use in other contexts, including those who want to manage multilingual code lists in various levels of public administration in Europe.

This software has been developed by the ISA founded developers Daniele Francioli and Emanuela Epure.

We are also grateful for the review of this document to Robin S. Smith.

 

http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-17action_en.htm
https://joinup.ec.europa.eu/community/are3na/description
Re3gistry software https://joinup.ec.europa.eu/software/re3gistry/home
INSPIRE registry service http://inspire.ec.europa.eu/registry/

 

To provide feedback on the software/service we kindly invite you to use this service: https://joinup.ec.europa.eu/software/re3gistry/home

 

Contact information

Michael Lutz

European Commission – Joint Research Centre

Institute for Environment and Sustainability

Spatial Data Infrastructures Unit

E-mail: inspire-registry-dev@jrc.ec.europa.eu

http://ies.jrc.ec.europa.eu

http://www.jrc.ec.europa.eu

 

Purpose

This document provides details about the Re3gistry, an open source tool for the management of codelists and other key terms within the scope of the Infrastructure for Spatial Information for Europe (INSPIRE) Directive 2007/2/EC. A background to this work is provided (Section 2), followed by a description of the information model (Section 3) and a description of the overall system (Section 4). Data module and Data Staticization (Section 4) functions are then described before focussing on the web service (Section 7). The last two sections list the technical components used in the Re3gistry (Section 8) and the installation instructions (Section 9).

Changes in this version

This is the v1.1 of the Re3gistry software. Changes in this version:

  • Update: data analyzer improved
  • Update: ability to re-start the staticization after the first data procedure
  • Bug fix: wrong error message in case of wrong software configuration
  • Bug fixes: various bugs in the error report system and minor bugs in the software.

 

1 Background

The European Union (EU) Member States are currently implementing the INSPIRE Directive and related regulations.  Technical guidelines for INSPIRE’s implementation, based on existing international standards, have been developed or are currently under development. Interoperability between systems is, however, being limited by varying ways of implementing standards, the regular evolution of standards and challenges in coordinating changes between standards, alongside varying choices in the technologies being adopted.

To further address these interoperability issues and provide support to the Member States, the platform will provide guidance, collaboration, sharing of best practices and approaches and a reference implementation of common components through the following activities:

  • Inventory of
    • Existing INSPIRE components from the Open Source community;
    • Components used within the Member States to implement INSPIRE;
    • Missing components;
  • Selection of other policies and initiatives from other sectors (such as INSPIRE, Water Framework Directive[2], Digital Agenda for Europe, open data, Shared Environmental Information System (SEIS)[3] etc.;) requiring exchange and sharing and maintenance of spatial data sets and services.
  • Selection of the missing components and/or functionalities. Multilingual support is envisioned where required;
  • Support Open Source projects to develop the missing items and produce the related documentation (installation guides and technical documentation in several languages);
  • Selection and development where required of conformance test suites;
  • Set up a collaborative platform to share and maintain the components

The outputs of this work will also appear on the ISA programme collaborative platform, JoinUp, to aid wide re-use.

Introduction

The Re3gistry software is designed to manage registers of items in different languages. This system loads the registry data from a data file, organizes them, and  exports these data in different (customisable) formats (e.g. html, xml, json, rdf). The exported files could then be used to set up a web service.

The following documentation explains the details related to the Re3gistry software, including the install instructions and the user guide. There is also a guide to extend the software by creating new modules.

 

2 Information Model

A simple representation of the system’s information model is shown below (fig. 1), with the following paragraphs describing each entity that can be found in the system. The complete information model can be found here:[4].

Figure 1 - Simplified system model

2.1 Registry

A registry is an information system on which a register is maintained.

 

2.2 Register

A register is a single controlled collection (list) of items. Each register is operated on behalf of some owner organisation which provides the authority for the collection.

The type of item that can be entered in a register is completely open. For example, anything which can be given a Uniform Resource Identifier (URI) can be registered.

The register items are called items. Each item has a specific item class that describes the type of the items.

 

2.3 Item Class

The item class defines a group of items contained in the register. The item class is a description of a class of information items specified in a technical standard.

For example, in a “theme register”, such as a register for an INSPIRE Theme, the item class of the items contained can be referred to as “theme items”.

Item class can have parent/child relations to identify if the items are part of a collection. For example, in the INSPIRE Registry, the “codelist register” has two item class: CodeList and CodeValue. The CodeList item class is the father of CodeValue. The CodeValue item class contains the items that are part of the “Code List” collection. This means that this collection relation is between items belonging to different item classes. The "collection" relation and the hierarchy between the item classes is used to handle the hierarchical registers.

 

2.4 Item

Items are elements that can be entered in a register. An item can contain other child items. For example, in the INSPIRE registry, the item class “CodeList” could contain other items (items with item class “CodeValue”). The contained item (the “codevalue item”), has an indication of the collection within the collection_id field (for detailed information please see the complete information model [4]).

Another type of relation between items of the same item class is the parent/child relation. This kind of relation is provided by the parent_id field in the CSV data file. This field represents the reference to the local id of the parent item.

Note: The parent item and the child item shall be in the same item class. If an item is part of a collection, it can have a parent contained in the same collection.

 

2.5 Status

Each item has a status as defined in ISO 19135-1 draft:

  • Valid: the item has been accepted, is recommended for use, and has not been superseded or retired (value to be used in the CSV import file: valid).
  • Invalid: a decision has been made that a previously valid register item contains a substantial error and is invalid, and will normally have been replaced by a corrected item (value to be used in the CSV import file: invalid).
  • Submitted: the item has been entered into the register, but the control body has not accepted the proposal to add it (value to be used in the CSV import file: submitted).
  • Superseded: the item has been superseded by another item and is no longer recommended for use (value to be used in the CSV import file: superseded).
  • Retired: A decision has been made that the item is no longer recommended for use. It has not been superseded by another item (value to be used in the CSV import file: retired).

 

2.6 Standard and Custom Attribute Fields

Each element has a standard list of attributes. Below there is a list of attributes for each object type represented in the system.

Registry

Id Identifier of the registry
Label Label of the registry
Content summary

Description of the purpose for which the registers and their items managed in the registry are made available to potential users.

Registry manager Person or organization responsible for the day-to-day management of the registry

Register

Id Identifier of the register
Label Label of the register
Content summary Description of the purpose for which items in the register are made available to potential users. It should also specify any limits to the scope of the register and identify the types of applications for which the items are intended.
Owner Organization that estabilishes a register
Register manager Organization to which management of a register has been delegated by the register owner
Control body Group of technical experts that makes decisions regarding the content of a register.
Submitter Organization authorized by a register owner to propose changes to the content of a register
Contact point Name or position of the person who serves as a point of contact for information about the register owner and the register
License The licence under which the register content is being made available

The Registry and Register attributes cannot be extended using custom attributes (see below).

The information related to the Registry and the Register can be configured in the system during the installation using the provided example .sql script (refer to chapter 6.1.2).

Item

Id Identifier of the item
Label Label of the item
Definition Definition of the item (precise statement of the nature, properties, scope or essential qualities of the item)
Description Description of the element (statement of the nature, properties, scope, or non-essential qualities of the item but are not specified by the definition)
Status Status of the item

The custom attributes are designed to extend the standard attributes of an item. By using a custom attribute, the item can be extended to include any item-specific attribute. The current standard list of attributes required by the input file format is described in paragraph 3.5.1, Table 1.

The system creates the items reading the data from the input file. Fore more information on the system flow, refer to chapter 3.

 

2.7 Language representations

The Re3gistry admin interface and the data produced by this tool have two different localization systems.

The Re3gistry admin interface uses a localization file to translate the GUI. This file is not linked with the translation of the data processed by the tool. Currently in the package there is only the English and Italian translation file for the Re3gistry admin interface.

The data produced by the Re3gistry admin interface have 2 different localizations:

The data localization (included in the zip data file);

The graphic user interface localization, included in a XML file.

After loading the data from the zip data file (the package contains an example of localized data file), the system exports the data it in different formats (customizable by the user).

For doing this, it uses a template file for each format. These template files, is then filled by the system with the data coming from the data file and the GUI localization information, coming from the GUI localization file.

The example zip data file provided in the package contains some example data in all the languages. The example template contains the INSPIRE registry GUI interface and the GUI localization files with all the languages.

Using the file contained in the package, you are able to load and export the example data in all the languages supported by the INSPIRE registry service.

 

 

3 System description

The Re3gistry software handles the entire data flow process for register items management, from data import and edit to the staticization of specific items in different customizable formats.

The system is composed by modules: there is a common module, implementing all the basic features, and several other modules implementing each specific functionality.

This package is bundled with the data management module (responsible for importing and editing data), the data staticization module (responsible for saving the data as static files in different formats) and the deployer module (responsible for deploying all the static files produced to the target production server).

The picture below (fig. 2) shows a schematic representation of the system.

Figure 2 - Schematic system description

 

The schema below gives a complete overview of the Re3gistry software and all the other components required in order to provide a RESTful web service (fig. 3).

Figure 3 - System description

The following paragraphs give an overview of the system. A detailed description of each module and how to extend or create a new module is provided at chapter 5 and chapter 7.

 

3.1 Registry core

The registry core module contains all the basic object model and database operation required by the system.

 

3.2 Data module

The data module is responsible for all the data operation that can be performed on the registers.

The input for this module is the zip data file (described at paragraph 3.4), containing one CSV file for each of the operations, for each register.

The supported operations are taken from the ISO19135-1[5] and are listed below (detailed at paragraph 2.5):

  • Addition
  • Clarification
  • Supersession
  • Retirement
  • Invalidation

The description of the CSV file endocing for each of these actions is provided at paragraph 3.5.

The data module performs different check to ensure the data consistency in the register.

The data module procedure steps are listed below:

  • Format check: a check on the csv files format (described at paragraph 3.5);
  • Data analysis: a semantic check trough the data, by verifying links between data and data consistency (described at paragraph 3.6)
  • Data store: save the data contained in the CSV files (described at paragraph 3.7).

An important concept is the “operating language”: this property identifies the language representation that shall be always available. You cannot load an item without this language representation.

For example, if the operating language is set to English, and a new item has to be added in the French language, the addition file shall also contain the item in the English language.

The operating language can be configured in the system following the documentation at chapter 6.

 

3.3 Common user interface

Since version 1.0 the Re3gistry software has a simplified single page interface. This interface has a single input point that starts all the required modules/procedures.

To start the procedure, the user needs only to upload the data file using the "Add data file" button and dialog. Only zip file can be uploaded.

By clicking the "Add data file" button, a dialog is opened: it requires choosing a data file, a short description for the procedure and the optional “Ignore warning” flag.

The procedure’s description field is mandatory and it is useful to later identify the procedure in the list of the procedure completed. The “Ignore warnings” checkbox force the system to perform the data store even if the analyzer finds any warnings in the data files. If the analyzer finds some errors, the procedure is locked even with the “force warnings” flag set. The difference between errors and warnings is explained at paragraph 3.6.

After filling all the required information, the procedure is tarted by clicking on the “Start procedure" button.

The user can always check the status of the procedure in the "procedure list" table. This table contains an history of the procedures performed togheter with the current running procedure. Once the procedure starts, it performs all the store & staticize operations.

Note: Only one procedure can run per time. If the user add a new data file while another procedure is running, the newly added data file is queued. The system will run the queued files once the current running procedure wil end.

The procedure may take a considerable amount of time, depending on the number of items to be processed.

Once the procedure has started, the browser can be closed: the system will continue to run. An email notification will be sent once the procedure ends; if it ends with some problems, the list of problems will be attached to the email.

Once the procedure ends succesfully, the static files export and the solr export can be started againg using the specific button in the table containing the list of procedures.

 

3.3.1 Procedure statuses

Checking data The system is checking the dafa file format and analyzing the data file consistency.
Storing data The data has been checked; the system is storing the data to the database
Writing files The data has been  stored to the database; the system is writing the static file to the file system
Completed The files has been written to the file system and they are ready for the deployment
Imported This status indicates that the data import has succesfully completed but the export to the static files has not been done (or the export went wrong). In this case the export can be started manually with the specific button in the UI.
Deployed The file has been deployed to the web server
Failed Somenthing went wrong during one of the data procedure step. In this case, a detailed error description is sent to the user's mail

 

Figure 4 - Re3gistry web interface

The Re3gistry web interface contains also 2 data file examples: one is an empty file containing only the structure for each register, the other file contains some sample INSPIRE Registry data. They could be a starting point for testing the system and getting familiar with it.

 

3.4 Data file structure

Since the first versions of the Re3gistry software, the data file had a XML file descriptor. Now, the system can understand all the information needed in order to perform the actions by analysing the ZIP folders and files.

The zip file structure is described below:

  • The root of the ZIP file contains one folder for each of the item class (for the definition of item class, refer to paragraph 2.3); the system is going to perform an action on each of these folders. The name of the folder has to be exactly the same name of the item class, including the case (the reference to the register is taken through the item class, which is linked to the register in the information model – chapter 2).
  • Each of these folders contains the CSV files related to the action to be performed on the specific item class. The file is named with the action’s name in lowercase.

An example ZIP structure:

Exmaple 1 - ZIP data file

import.zip   

     ApplicationSchema/addition.csv    

     ApplicationSchema/clarification.csv    

     Theme/supersession.csv 

     Theme/retirement.csv

     CodeList/invalidation.csv

 

The ZIP data files can contain only the files and folders needed to perform the related actions.

For example, the ZIP data file provided above (example 1) will perform:

  • addition(s) and clarification(s) for the ApplicationSchema item class;
  • supersession(s) and retirement(s) for the Theme item class;
  • invalidation for the CodeList item class.

3.5 Data actions and CSV formats

For each of the actions supported by the system, there is a specific CSV format for the data file (detailed below).

 

3.5.1 Addition

This action adds the items specified in the related register.

The CSV for this action is composed by two different types of fields: main fields and additional fields.

The main fields must be provided even if they have null values; if one or more of the main field values are not available, the specific field must be left blank (the csv separator without any character or space. Example: value1|||value2|value3|||value4||…).

There are some main fields that are required for each action. These fields are highlighted in table 1 and can be configured in the data module properties file.

The property to customize the required fields is: data.mantatoryfields.<action_name>=<field>.<field>. …

The example below shows an example of mandatory fields for the addition action.

Exmaple 2 - Mandatory field property

Data file header:

LocalId|ParentLocalId|CollectionLocalId|Language|Label|Definition|Description|Status|Comment|*Theme[t,f,f,t]|*UMLName[t,f,f,f]

 

Property in the properties file:

data.mantatoryfields.addition=0.3.4.7

 

List the index of the field as in the property above to set a common field mandatory.

The additional fields (called custom attributes) represent some extra fields that users can add in order to customize the data model. The properties of these fields are described in the table 1.

Table 1 - CSV fields for the addition data file

Type

Code

Description

Main fields

LocalId (mandatory)

The local identifier (id) is used to identify an item inside the register and to create the URI of the item. If an item has a local identifier such as “localId”, the URI will be composed as follow: “http://site.ext/register/localId”;

ParentLocalId

This is the reference to the parent item

CollectionLocalId

This is the reference to the collection parent. If this field is set then an item is part of a collection.

Language (mandatory)

The language of this item. If there is multiple language for each item, the row has to be duplicated, hanging the language related fields.

Label (mandatory)

This contains the label of the items

Definition

This contains the definition of the system

Description

This contains the description of the items

Status (mandatory)

This contains the status related to the current item. More details about the code to be used in this field can be found in Table 2.

Comment

A textual comment for the change log stored by the system for each action.

Additional fields

(custom attribute/ header descriptor)

The custom attribute headers define the properties of the additional fields.

 

Custom attributes must be added to the CSV file after the standard fields and have the following format:

*custom_attribute_name[required,multivalue,coded,foreignkey]

 

The value in the square parenthesis is a Boolean value, and can be “t” (for true) of “f” (for false): e.g. *extensibility[t,f,f,f].

The meaning for each of the properties set in the square parenthesis is described in the following rows.

Required

This indicates that the custom attributes is required. The whole data rows in the CSV must have this value in order to be imported.

Multivalue

This indicates that this custom attribute could have more than one value. The values are saparated by comma. E.g. ...|ab,cd,ef

Coded

If this property is set to true, the custom attribute takes its values from a list of values defined in a specific database table. More details about the information model can be found in Annex A (This feature is not implemented but is foreseen in the future).

Foreignkey

If this property is set to true, the value in this field is a localId of another item in the register. The name of the custom attribute has to be the name of the item class referenced.

Exmaple 3 - Addition CSV example

Import.zip -> ApplicationSchema/addition.csv

LocalId|ParentLocalId|CollectionLocalId|Language|Label|Definition|Description|Status|Comment|*Theme[t,f,f,t]|*UMLname[t,f,f,f]

sd|||en|Species Distribution|||valid||sd|SpeciesDistribution

sd|||fr|Répartition Des Espèces|||valid||sd|SpeciesDistribution

 

This example contains an addition of the item "sd" in the ApplicationSchema register. There is the mandatory header line and two rows (the "sd" item in two languages - one row for english language an the other for frrench).

Note: in the example above the custom attribute (name: Theme) is defined as foreign key. In this case, the name of the custom attribute (in the descriptor line – the first line of the csv) shall be exactly the name of the itemclass related to the item specified as value. The value of the custom attribute shall be the LocalId of the referenced element. Only items that are not part of a collection (with the CollectionId field empty) can be pointed by a custom attribute.

 

3.5.2 Clarification

This action allows editing the items. Every time a clarification action is performed, the version of the edited item is increased.

This action data file shall be filled only with the fields that need to be edited (plus the LocalId and the language to identify the item). Only the filled fields will be updated; the field left blank in the action file, will not be updated, and remain as is in the register.

The clarification action cannot change the collection of an item. Nevertheless the clarification CSV file contains the CollectionLocalId field in order to unambiguously identify the item. In fact, in the same data file, different items that are part of a collection may have the same LocalId but different CollectionLocalId. In the clarification (but also in the supersession, retirement and invalidation) the item has to be identified in the data file by its LocalId and CollectionLocalId (if available).

The clarification’s CSV file is like the addition CSV file but without the status field. The CSV format is described in table2 (for the detailed fields description, refer to table 1).

Table 2 - CSV fields for the clarification data file

Type

Code

Description

Main fields

LocalId (mandatory)

The local identifier (id) is used to identify the item to be updated

ParentLocalId

 

CollectionLocalId

 

Language (mandatory)

The local identifier (id) is used to identify the language representation of item to be updated

Label

 

Definition

 

Description

 

Status

 

Comment

 

Additional fields

(custom attribute/ header descriptor)

In the clarification data file, all the custom attribute related to the current item class shall be specified in the first row as the addition CSV.

Exmaple 4 - Clarification CSV example

Import.zip -> ApplicationSchema/clarification.csv

LocalId|ParentLocalId|CollectionLocalId|Language|Label|Definition|Description|Comment|*Theme[t,f,f,t]|*UMLname[t,f,f,f]

sd|||en|Species Distribution changed|||||changed custom attribute

sd|||fr|| Ajouter la définition en français ||||

 

This example contains a clarification CVS file for the ApplicationSchema item class (linked to the ApplicationSchema register). The file contains only the “sd”. Item, to be modified in the 2 different language:

  • English: the label and the “UMLname” custom attribute are edited;
  • French: the definition is added

 

3.5.3 Supersession

This action allows superseding an item. The supersession CSV file format is quite different from the previous two, because it only requires the reference to the item to be superseded and to the successor item(s).

The successor item(s) shall be available in the database or shall be present in the addition csv of same item class data file containing the supersession csv.

Table 3 - CSV fields for the supersession data file

Code

Description

SupersededLocalId (mandatory)

The local identifier (id) is used to identify the item to be superseded

SupersededCollectionLocalId

The optional collection local identifier (id) is used to identify the item to be superseded (to be provided if the item is part of a collection)

NewLocalId (mandatory)

The local identifier (id) is used to identify the successor item(s). The identifier can be a single identifier or multiple identifier separated by comma (e.g. ac,mf).

In case of multiple successors elements that are part of a collection, all of the specified successor shall belong to the same CollectionId.

If the successor belong from different Collection, one row for each collection shall be specified in the supersede file.

NewCollectionLocalId

The optional collection local identifier (id) is used to identify the successor item (to be provided if the item is part of a collection).

Comment

A textual comment for the change log stored by the system for each action.

Exmaple 5 - Supersession CSV example

Import.zip -> ApplicationSchema/supersession.csv

SupersededLocalId|SupersededCollectionLocalId|NewLocalId|NewCollectionLocalId|Comment

sd||ad||this is a comment

mf||ad,ac||this is a comment

This example contains a supersession CVS file for the ApplicationSchema item class (linked to the ApplicationSchema register). The file contains two supersessions:

  • the "sd" item, with successor item "ad".
  • the "mf" item, with successor items "ad","ac".

 

3.5.4 Invalidation

This action allows invalidating an item. The invalidation CSV file format is the same such as the supersession CSV file, in addition there is the recursive flag (explained in the  table 4), because it only requires the reference to the item to be superseded and to the successor item(s).

The successor item(s) shall be available in the database or shall be present in the addition csv of same item class data file containing the supersession csv.

Table 4 - CSV fields for the invalidation data file

Code

Description

LocalId (mandatory)

The local identifier (id) is used to identify the item to be invalidated

CollectionLocalId

The optional collection local identifier (id) is used to identify the item to be invalidated (to be provided if the item is part of a collection)

SuccessorLocalId (mandatory)

The local identifier (id) is used to identify the successor item(s). The identifier can be a single identifier or multiple identifier separated by comma (e.g. ac,mf).

In case of multiple successors elements that are part of a collection, all of the specified successor shall belong to the same CollectionId.

If the successor belong from different Collection, one row for each collection shall be specified in the supersede file.

SuccessorCollectionLocalId

The optional collection local identifier (id) is used to identify the successor item (to be provided if the item is part of a collection)

Recursive

This field tell the system to recursively invalidate any children items or linked items to the invalidated item. To set this flag put the word “true” in the field, otherwise leave it blank.

Comment

A textual comment for the change log stored by the system for each action.

Exmaple 6 - Supersession CSV example

Import.zip -> ApplicationSchema/invalidation.csv

LocalId|CollectionLocalId|SuccessorLocalId|SuccessorCollectionLocalId|Recursive|Comment

sd||ad|||this is a comment

so||ac,mf||true|this is a comment

 

This example contains an invalidation CVS file for the ApplicationSchema item class (linked to the ApplicationSchema register). The file contains two invalidations:

  • the “sd” item,  with the successor item "ad". This is not recursive and contains also a comment.
  • the “so” item,  with the successor items "ac" and "mf". This is recursive and contains a comment.

 

3.5.5 Retirement

This action allows retiring an item. The supersession CSV file format is the simpler one. It only requires the reference to the item to be retired plus a recursive flag (explained at table 5).

Table 5 - CSV fields for the retirement data file

Code

Description

LocalId (mandatory)

The local identifier (id) is used to identify the item to be retired

CollectionLocalId

The optional collection local identifier (id) is used to identify the item to be retired (to be provided if the item is part of a collection)

Recursive

This field tell the system to recursively retire any children items or linked items to the retired item. To set this flag put the word “true” in the field, otherwise leave it blank.

Comment

A textual comment for the change log stored by the system for each action.

Exmaple 7 - Retirement CSV example

Import.zip -> ApplicationSchema/retirement.csv

LocalId|CollectionLocalId|Recursive|Comment

sd|||this is a retirement

so||true|this is a retirement

 

This example contains a retirement CVS file for the ApplicationSchema item class (linked to the ApplicationSchema register). The file contains two item, to be retired:

  • the “sd” item.  This is not recursive and contains also a comment.
  • the “so” item.  This is recursive and contains a comment.

 

 

3.6 Data analyser

The data analyser component perform some checks in order to understand if the data file contains consistent data and also to understand if the database keeps its consistency after current data file is processed.

There are two type of message raised by the analyser:

  • Errors: these are locking problems; the data file shall be corrected before the system can continue its work.
  • Warnings: these are non-locking problems; the user is notified about the problem but the import can continue by setting the “ignore warning” flag available in the web interface.

Below there is a list of the checks performed by the analyser.

Table 6 – Analyzer check and report message type

Legend: errors - warnings

 

Actions

Situations

Addition

Clarification

Supersession

Invalidation

Retirement

Item already in the database

                 

-

-

-

-

Item not available in the database

-

                 

                 

                 

                 

Custom attribute (with foreign key flag true) pointing to an item not available in the database nor in the current addition file

                 

                 

-

-

-

Custom attribute (with foreign key flag true) pointing to an item not valid

                 

                 

-

-

-

Item pointing to a parent not available in the database nor in the current addition file

                 

                 

-

-

-

Item pointing to a parent not valid

                 

                 

-

-

-

Item pointing to a collection not available in the database nor in the current addition file

                 

-

-

-

-

Item pointing to a collection not valid

                 

-

-

-

-

Item pointing to a successor not available in the database nor in the current addition file

-

-

                 

                 

-

Item pointing to a successor not valid

-

-

                 

                 

-

The item’s operating language is not available

                 

-

-

-

-

The item’s language is not in the list of supported language

                 

-

-

-

-

The item has collection

-

-

                 

               

                 

The item has child

-

-

                 

                 

                 

The item has links (pointed by a custom attribute as foreign key)

-

-

                 

                

                 

If the system finds some errors or warnings, the procedure is stopped and an email notification is sent. The email contains an attachment with the detailed list of the errors.

In case the procedure produces both errors and warnings, the system will lock even if th the “ignore warning” flag is set. In this case the notification email will contain a mixed error and warning rows.

The user is notified by email also when the procedure ends correctly.

 

3.7 Data store

The data store component save the data to the database and performs all the action described in each csv contained in the ZIP data file. The data store component starts after the data check performed by the analyzer.

If something goes wrong in the store process, the user will recieve an email with the detailed list of errors, and the import is rollbacked.

 

3.8 Staticization

The “Staticization” component is responsible for saving  to the file system the data stored in the database by the data procedure.

 

3.8.1 Data staticization process

The data staticization procedure saves the items stored in the database to the file system. The system uses some XSLT trasformations files (described in the following paragraphs) to provide the different configured formats.

The staticizer creates firstly a master xml file, containing a structured export of the items stored in the database.

Then, using the custom XSLT files, it creates the different file formats (also called "custom" formats). The “custom” folder will contain all the files produced using the configured XSLT files.

 

3.8.2 XSLT

The transformation system allows the user to define different custom formats. The XSLT files are responsible for the creation of those custom file formats.

The XSLT files translate the information contained in the master XML file. There are 3 types of master XML files:

 

The XSLT files shall be contained in a folder specified in the system properties file (Re3gistryStaticizer.properties - properties files describet at chapter 5.2.3). The property is specified below:

Example 8 - XSL path properties
staticizer.formats.path= /your-path/xsl

 

Each format shall have a folder named with the exact name of the format itself.

E.g. the XSLTs for the .html format are contained in a folder called "html".

An example folder list related to the XSLT folder linked to each format used in the INSPIRE registry is provided below:

Figure 5 - XSL folder structure

 

Each of the folders used for the XSLT transformation shall contain one XSLT file for each registry, one for each register and one for each category of item:

Exmaple 9 - Xslt transformation for xml

<itemclass_uriname>.<format>.xsl - items

<register_uriname>.<format>.xsl - register

<registry_uriname>.<format>.xsl - registry

 

Figure 6 - XSLT trasformations files for the custom XML format (contained in the xml folder)

 

Each format shall also be configured in the format list property, in the Re3gistryStaticizer.properties file.

To configure each transformation, the relative entry has to be listed in the xsl.formats.list property. Example:

Exmaple 10 - Staticizer configurations

xsl.formats.list=xml,json,html,atom,rdf

 

This property tells the system which transformations are available.

 

The name of the folders containing the output files can be customized in the Re3gistryStaticizer.properties file:

Exmaple 11 - Staticizer folder configurations

staticizer.standard.folder=standard

staticizer.custom.folder=custom

staticizer.solr.folder=solr

staticizer.solrcustom.folder=solrcustom

 

3.8.3 Static element localization

To handle the localization of some static elements (such as some gui part of the the HTML format) there is a language mapping file. The “gui-languages” folder contains <language_code>.xml files (one for each language supported by the system) containing the keys used by the XSLT files for any static string not available in the database.

Example - the fr.xml gui translation file could be:

Exmaple 12 - Gui-languages folder, French file structure

<guilanguage languagecode="fr">

<item key="feedback">Retour d'utilisation</item>

<item key="powered-by">Construit avec</item>

<item key="about">A propos de</item>

<item key="contact">Nous contacter</item>

. . . .

</guilanguage>

 

Usage example

Place this piece of code to retrieve the translated word in the file.

<xsl:value-of select="$languagefile/guilanguage/item[@key='feedback']"/>

The $languagefile variable is the path of the document containing the translations and it is defined at the beginning of the xsl file. An example is provided below:

<xsl:variable name="language" select="item/language/isocode"/>
<xsl:variable name="languagefile" select="document(concat('../gui-languages/',$language,'.xml'))" />

 

3.8.4 Additional information

The staticizer is responsible aslo for the creation of some additional files to handle the content-negotiation (Var files) and the Apache Solr indexing.

The Var Generator generates the .var index configuration files needed to set-up the Apache content-negotiation feature. This allows the user to use the content-negotiation approach explained in section 5.2.

The Solr export will produce the files to be imported by the Apache SOLR application (one file for each item, containing all the languages). The Solr XSLT transformation produces files compliant with the Apache SOLR system.

 

3.8.5 Trasformation for additional pages

There is the possibility to create some custom pages for the HTML format in addition to the elements produced by the system.

An example use of these additional transformations is the creation of some static pages like the "about" or "help" pages. These pages are translated using the gui-languare key-value word mapping.

The list of additional XSLT to be processed shall be specified in the related property; an example is available below.

xsl.custom.list=about,help,system

The XSLTs for these additional pages shall be placed in the XSL folder html/common. Below you can find a folder structure following the additional pages specified in the example above.

{xsl-folder specified in 3.8.2 - Example 8}/html/common/about.xsl

{xsl-folder specified in 3.8.2 - Example 8}/html/common/help.xsl

{xsl-folder specified in 3.8.2 - Example 8}/html/common/system.xsl

In this way the system will create as an output also the static HTML files specified in the xsl files.

 

3.8.6 Run a staticization manually

When you start a new data procedure, uploading a data file, it will import and expor the data automatically. Once the first procedure has been completed, if you want to start the export of the static file manually you can use the "Run data export functionality" available for each proceure with the "completed" or "imported" state.

 

3.9 Deployer

The deployer module is responsible for deploying all the static files produced by the staticization system to the target production server. This is needed if the production server is in a different machine (or the files in the same machine need to be moved to another place in the same system).

The module automatically takes all the static file produced and move them to the configured target place.

The system uses a configurable script to launch the deployment process. This script can be found in the main application properties folder under scripts/Re3gistryDeployer.

The system automatically recognise the operative system (linux or windows) and launch the relative file (linux-deploy.sh or windows-deploy.bat). These shell script can be configured and customized to perform the needed operation in order to move the produced files to another dirctory as well as to another server.

To configure the Deployer module, the properties to be customized are contained in the RegistryDeployer.properties file. The properties are described below.

deploy.script.folder: this property represents the folder where the .sh of .bat file are stored.

deploy.rss.file: this property is the location of the base RSS file. It is the master file that keeps all the RSS news. If the "RSS update" option has been selected, the RSS update system takes this file and updates it with the new information. Then the file is copied to the target path

deploy.rss.targetfolder: the folder in wich the rss file is stored after the update (usually is the same folder that contains all the files produced by the staticizer)

deploy.rss.baseuri: this is the base uri of the RSS file, stored in the rss file as the channel's "link" element (example: for the inspire registry, it is http://inspire.ec.europa.eu/registry/rss/)

 

4 Serving the registry contents

The Re3gistry software, produces a set of files organized to set up a web service in order to provide access to the registers items to the INSPIRE community and other potential users. An important characteristic of the service is the possibility to provide the same information in multiple formats and multiple languages. The service is implemented using the content-negotiation approach. The server can provide the correct file language/format based on the http header sent in the http request.

If a user prefers to use the classic way to accessing a specific file, the system supports this approach too.

The file to be used in order to set up a web service can be found in the Staticizer’s output folder. In detail, the files to be used are under the “custom” folder; place it in the web server document root.

Detailed information about the web service and its configuration are provided in the following paragraph.

 

4.1 Web Service

The files produced by the Re3gistry software (staticizer module) can be used to provide a web service in two ways: a RESTful web service which adopts a content-negotiation approach for serving the resource and a web service, which uses the resource name in order to access the specific resources. These are both described below.

 

4.2 RESTful web service with content-negotiation

This web service is implemented using the Apache HTTPD web server and its content-negotiation capabilities.

The resource is accessed using the standard RESTful notation. Below there are some examples:

Table 7 - RESTful URL example

URL

Use

http://base_uri/register_uriname

http://base_uri/item_itemclass/item_uriname

http://base_uri/collection_itemclass/

collection_uriname/item_uriname

Used to access a register, an item or an item collection

http://inspire.ec.europa.eu/themes

Used to access the INSPIRE theme’s register

http://inspire.ec.europa.eu/Applicationschema/ad

Used to access a specific application schema

http://inspire.ec.europa.eu/codelist/ ActiveWellTypeValue/decontamination/

Used to access  a specific value in the code list

To ask for a specific file format or language, the HTTP request should have the proper http header set. If no header is set, the web service returns (by default) the xml format in English.

Below there are some examples:

Exmaple 13 - Example of Content Negotiation Parameters

Accept             application/xml

Accept-Language    en

 

Accept             application/atom

Accept-Language    fr

In order to enable the content-negotiation capabilities, the Apache HTTPD server has to be configured. Each content folder needs a configuration file that addresses the HTTPD server to serve the right file. This file is a .var file (cn .var). The staticizer module has an automatic procedure to produce this file after the execution of the staticization.

An example of the cn.var file is given below

Exmaple 14 - .var file example

URI: ICDValue.en.atom

Content-language: en, en-GB, en-US, en-EN

Content-type: application/atom+xml

 

URI: ICDValue.en.html

Content-language: en,en-GB,en-US,en-AU,en-NZ; q=1.0

Content-type: text/html

 

URI: ICDValue.en.json

Content-language: en, en-GB, en-US, en-EN

Content-type: application/json

 

URI: ICDValue.en.xml

Content-language: en, en-GB, en-US, en-EN

Content-type: application/xml

 

For detailed information on how to configure Apache HTTPD and content-negotiation, refer to this url: http://httpd.apache.org/docs/2.2/content-negotiation.html

 

4.3 Standard web service

To access the resources without using the content negotiation approach, it can be used the standard url-based request can be used.

Below there are some examples of requests in different formats and languages:

Exmaple 15 - Direct URL example

 

4.4 Customizing the web pages.

There are two way of customizing the look and feel of the web pages produced by the Staticizer.

  • Customize the CSS: use this approach if the pages’ HTML structure and common content (like the footer information) doesn’t need any modification. In fact, only a few graphical aspect of the pages can be changed editing the CSS. The css related to the service are placed in the project package, folder example/web-service-files/res/css.
  • Customize the pages’ structure: to change the HTML structure or some common contents (like the common text or common page part), edit the XSL transformation files placed in example/xsl

4.5 Example configurations for Apache

Below there is an example configuration to set-up an Apache web service using the file produced by the Re3gistry Software.

 

DocumentRoot "/[root-path]/inspire-regadmin/data/output/custom"


#### Alias ####

Alias /theme                    /[root-path]/data/output/custom/theme

Alias /Theme                    /[root-path]/data/output/custom/theme

Alias /applicationschema        /[root-path]/data/output/custom/applicationschema

Alias /applicationSchema        /[root-path]/data/output/custom/applicationschema

Alias /codelist                 /[root-path]/data/output/custom/codelist

Alias /codeList                 /[root-path]/data/output/custom/codelist

Alias /res                      /[root-path]/data/output/custom/res

Alias /registry                 /[root-path]/data/output/custom

Alias /featureconcept           /[root-path]/data/output/custom/featureconcept

Alias /featureConcept           /[root-path]/data/output/custom/featureconcept

Alias /document                 /[root-path]/data/output/custom/document

Alias /Document                 /[root-path]/data/output/custom/document

Alias /glossary                 /[root-path]/data/output/custom/glossary

Alias /Glossary                 /[root-path]/data/output/custom/glossary

Alias /metadata-codelist        /[root-path]/data/output/custom/metadata-codelist



<Directory "/[root-path]/data/output/custom">

####  General settings ####

Options +Indexes -Multiviews +FollowSymLinks +ExecCGI

AllowOverride All

Order Allow,Deny

Allow from all


#### Language settings ####

LanguagePriority en

ForceLanguagePriority Prefer Fallback


#### Var handler ####

AddHandler type-map .var

DirectoryIndex  cn.var


#### Rewrite rules ####

RewriteEngine On

RewriteBase /

RewriteRule (^.*):([^\/]+)              $1/$2/cn.var                    [L]

</Directory>

 

5 Technical description

The following information describes the technical specification of the system.

 

5.1 Technology

The system require the Java 1.7 or Java 1.8 technology. The libraries used in the system are described in the Project Object Model (POM) file. This project is built using Apache Maven technology.

 

5.1.1 Web Server

There are two different web servers: one used to serve the RESTful web service and another is the Servlet container for the Re3gistry Java webapp. The two servers are listed below:

  • Apache HTTPD Version 2
  • Apache Tomcat Version 7 or Apache Tomcat 8

 

5.1.2 Database

The database layer is handled by EclipseLink library. All the databases supported by EclipseLink can be used for the system.

Note: only the following database type/version, has been tested for the current version of the system: PostgreSQL 9.2

 

5.2 System structure

The Re3gistry software is organized in modules: this concept allow a simple customization and extension of the system.

The main components of the system are the Re3gistryCommon module and the Re3gistry webapp.

The paragraphs below explain the main structure of the system and the modules. By understanding the system’s structure the modules can easily edited and extended.

 

5.2.1 Module concept

The module is a Java library plus some optional web pages that adds functionality to the core system. Each module is composed by a core part (java library). There may also be a presentation part (web pages).

  • The module’s core part (java library) contains all the logic related to the functionality provided by the module itself. Each module needs to include the Re3gistryCore module in order to access all the common beans and functions.
  • The module’s presentation part is optional (you can access the module's core functionalities by customizing the standard interface provided by the Re3gistry) and is contained in the module’s folder of the Re3gistry webapp. This folder contains the user interface to interact with the function provided by the module, and the specific module’s configuration file.

The Re3gistry webapp contains also the main configuration properties files and all the localization properties files.

 

5.2.2 Re3gistryCommon module

The Re3gistryCommon module is the base of all the software. It contains the object definition that represents each entity in the system and the manager to work with them (read/write). This core module has no a related presentation part in the Re3gistry webapp because its function is to provide the object structure and the main functionalities to all the other modules.

The Re3gistryCommon library is composed by different package described below:

  • constants: contains a class with all the constant used in this library;
  • managers: contains all the method to access the data for each bean defined in the model package;
  • model: contains all the bean related to the information model;
  • utils: contains several general utility classes, such as StringUtils or Logger class.

Figure 7 - A view of the Re3gistryCommon module from an IDE

 

5.2.3 Re3gistry webapp

The R3gistry webapp contains the web interface to manage and use the functionalities provided by each modules. In this webapp are stored also all the configuration files, both related to the core module and to each of the additional module.

Below there is a description of the components contained in the Re3gistry webapp.

  • Web Pages folder: contains all the files related to the web pages (jsp, css, js, resources like images etc.). The “modules” folder, inside the “Web Pages” contains all the module specific web pages and configuration files.
  • Source folder: contains the java class related to the Re3gistry webapp (constants, module manager functionalities, servlets, utility, etc.).
  • Resources folder: contains all the configurations and localization file. Below there is a description for each file:
    • META-INF/persistence.xml: in this file are specified all the configurations related to the database.
    • configurations/Application.properties: this file contains all the main Re3gistry configurations (common to all the modules);
    • configurations/modules: this folder contains all the properties related to each module. The file name shall be exactly as the name of the module’s folder.
    • configurations/logcfg.xml: the configuration related to the logging system;
    • localization/Application/localizationBundle_xx.properties: the folder localization contains all the properties files related to the localization of the system. There is the subfolder “Application” that contains the core system localizations. Then, each module has its own localization file contained in a folder called exactly as the Library part of the module.

Figure 8 - A view of the Re3gistry webapp module from an IDE

More information about the creation of a new module is provided at chapter 7.

 

6 System installation and first run

There are two way to set up and run the system. The project package contains both the source code and a copy of the software precompiled. The example data is also provided in the package.

  • The precompiled webapp (compiled using Java 1.7) is faster to install and run. Using this installation, and editing the XSLT example files, there is the possibility to produce all the data needed for a custom registry in a fast way.
  • The source code needs to be imported in an Integrated Development Environment (IDE - such as NetBeans) and compiled. In this way, there is the possibility to execute the code and, if needed, to debug and edit the code itself (as well as extend and create new modules).
Note: you can also follow the quick installation guide and the quick customisation guide.

 

6.1 Common installation steps

Step 1 - External software installation

Install the required software:

  • Apache HTTPD;
  • Java 1.7 or Java 1.8;
  • Apache Tomcat 7 or Apache Tomcat 8;
  • PostgreSQL 9.2.

Remember to set two different ports, linked to the Apache HTTPD and the Apache Tomcat. Typically the standard 80 port is used for the Apache HTTPD web server and the 8080 port for the Apache Tomcat. For detailed information related to the external software installation, refer to their relative manuals.

The Re3gistry software requires the European Commission Central Autentication Services libraries. These libraries are available in the project package in the "required-files" folder. The following parafgraph describes how to install the ECAS libraries in Apache Tomcat.

 

6.1.1 ECAS installation

Follow the guide to install ECAS for Tomcat.

 

6.1.2 Step 2 - Database creation and initialization

The database initialization creates the structure of the DB and fills the registry and registers details and informations. An example of database initialization script is providet in the Re3gistry software pachage.

Execute the database initialisation using the example SQL scripts available in the project’s package “example\database” folder.

The database script could be executed either from the command line or using a Graphic User Interface (GUI) such as pgAdmin.

Open the create-tables.sql into an SQL editor and run the queries. Do the same for the database-initialization.sql and database-localization.sql. It is important to run the create-tables.sql before the other scripts. The database-initialization.sql and database-localization.sql files contain some sample data. If you need to cleanup all the database and recreate it, in the example package, there is also a drop-tables.sql script.

Note: for each new register to be added, the register table, the itemcalss table and the localisation table should all be filled in; for more information related to the creation of new register, please refer to paragraph 6.4.

 

6.2 Using the precompiled webapp

Step 3 - Copy the project folders

Before moving the project’s folder, make sure that the Tomcat server is not running. Take the “Re3gistry” folder from the “compiled” folder in the project package and move it in the tomcat’s “webapp” folder.

Step 4 - Setting up the configurations

Open the configuration files and edit it as explained below. The properties to be changer are marked with the code "_[name]_" (e.g _DBADDRESS_ or _USERNAME_).

  • Re3gistry\WEB-INF\classes\META-INF
    •  persistence.xml: this file provides the configuration for the database. The parameters to be configured are: “javax.persistence.jdbc.url", "javax.persistence.jdbc.password", "javax.persistence.jdbc.user;
  • Re3gistry\WEB-INF\classes\configurations
    • Application.properties: this file contains all the main configurations for the system. To run the system the properties to be updated are “application.language.available” which represents the available language on the interface and the related language label, “application.contact”, “mail.sender”, “mail.recipient” which are the e-mail address of the contact, sender and respectively recipient and “mail.smtp.host” which is the server host smpt email;
    • logcfg.xml: this file contains all the configurations related to the logger system.  In this file, the logger file path for each “log appender” has to be configured.
  • Re3gistry\WEB-INF\classes\configurations\modules\[modulename.properties]: each module has its own properties file. To run the system, the configuration to be updated are:
    • Re3gistry/WEB-INF/classes/configurations/modules/RegistryData.properties: each property is detailed with comment directly in the property file. The important properties to be updated are:
      • - “data.customtempfolder” (folder to save the temporary data files): if this property is left blank, a default folder is taken (inside the webapp folder);
      • - “data.operatinglanguage”: this property represents the main language of the system;
      • - data.supportedlanguage: this property represents the list of the language supported by the data management system.
    •  Re3gistry/WEB-INF/classes/configurations/modules/RegistryStaticizer.properties:
      • “xml.formats.list=xml,json,html,atom...”: the formats to be used by the xslt transformations (separated by comma)
      • “staticizer.SOLR.format=solr”: the SOLR folder name.
      • “staticizer.cutom.folder.path”: in this folder all the staticized files will be saved .
      • “staticizer.formats.path”: this property specifies the folder containing the xslt files for the transformations. Some example xslt files are available in the project package (example folder).
    • Re3gistry/WEB-INF/classes/configurations/modules/RegistryDeployer.properties:
      • "deploy.script.folder": the folder containing the deploy scripts.
      • "deploy.rss.file": the base RSS for the deployment system.
      • "deploy.rss.targetfolder": target folder for theproduced RSS.
      • "deploy.rss.baseuri": the base uri of the links inside the RSS.

Start the server: once all the previous steps are completed, start the Tomcat server and test the installation.

Try to access the webapp using its address. An example of url to be called is provided below (this url may be differ depending on the system’s configuration):

http://localhost:8080/Re3gistry

 

6.3 Using the source code

The following steps detail how to install and run the Re3gistry for the first time using the NetBeans IDE. We choose NetBeans for this documentation because it is the official open source IDE provided by Java.

Step 3 - Load projects

To deploy and test the application, load the projects contained in the “source” folder of the package using the IDE (make sure it supports Maven).

Figure 9 - Load projects with NetBeans IDE

Step 4 - Configuration files

After loading the projects, go to the configuration files under the “resources/META-INF” folder.

For convenience, all the configurations are placed under the “build profile” in the POM files of the Re3gistry web project. These Project Object Model (POM) files can be found under the project’s root folder.

Edit these configuration settings using the local settings:

  1. The path for the logs file;
  2. The information regarding the database;
  3. The path for the temporary import folder/ the custom export data folder (if it is left blank, a standard folder, inside the webapp folder will be taken).

To customize the set up for all the feature of each module, check the following  properties file:

  • Re3gistry/src/main/resources/configurations/modules/Application.properties:
    •  this file contains all the main configurations for the system. To run the system the properties to be updated are “application.language.available” which represents the available language on the interface and the related language label, “application.contact”, “mail.sender”, “mail.recipient” which are the e-mail address of the contact, sender and respectively recipient and “mail.smtp.host” which is the server host smpt email;
  • Re3gistry/src/main/resources/configurations/modules/RegistryData.properties:

- “data.operatinglanguage”: this property represents the main language of the system;

- data.supportedlanguage: this property represents the list of the language supported by the data management system.

  • Re3gistry/src/main/resources/configurations/modules/RegistryStaticizer/RegistryStaticizer.properties:
    • “xml.formats.list=xml,json,html,atom”: the formats to be used by the xslt transformations (separated by comma)
    • “staticizer.SOLR.format=solr”: the SOLR name.

An example of the POM file related to the profile is provided below:

Exmaple 16 - Example POM file

<pofiles>       

       <profile>

            <id>env-dev1</id>

            <activation>

                <property>

                    <name>env</name>

                    <value>dev1</value>

                </property>

            </activation>

 

            <properties>

              <!-- Loggers configurations -->

<Default.log.file.dir>C:/re3gistry-data/logs/Complete.log</Default.log.file.dir>

<Re3gistry.log.file.dir>C:/re3gistry-data/logs/Re3gistry.log</Re3gistry.log.file.dir>

<Re3gistryData.log.file.dir>C:/re3gistry-data/logs/Re3gistryData.log</Re3gistryData.log.file.dir>

<Re3gistryStaticizer.log.file.dir>C:/re3gistry-data/logs/Re3gistryStaticizer.log</Re3gistryStaticizer.log.file.dir>

               

             <!-- Database configurations -->

             <persistence.jdbc.url>jdbc:postgresql://db_ip:5432/inspire_registry </persistence.jdbc.url>

                <persistence.jdbc.driver>org.postgresql.Driver</persistence.jdbc.driver>

             <persistence.jdbc.username>db_username</persistence.jdbc.username>

             <persistence.jdbc.password>db_password</persistence.jdbc.password>

               

             <!-- Specific module configurations -->

                 

             <!-- RegistryData -->

<RegistryData.import.dir>C:/re3gistry-data/temp</RegistryData.import.dir>

             <!-- / -->

               

             <!-- RegistryStaticizer -->

<!-- The root folder where to save the files produced by the Staticizer. ->

<RegistryStaticizer.export.dir>C:/re3gistry-data/staticizer</RegistryStaticizer.export.dir>

<!-- The path from where to read the xslt transformation and the translations for the GUI interface -->

<RegistryStaticizer.xsl.dir>C:/re3gistry-data/xsl</RegistryStaticizer.xsl.dir>

                                                       

            </properties>

        </profile>

</profiles>

Prior to starting the build process, ensure that the build profile (specified in the POM) for each the Re3gistry web project is selected. To do check this, in NetBeans go to “Set Configuration” under the project and choose the right profile (right click on the project -> “Set Configuration”).

Step 5 - Build Projects

To get the system running, build each project and start the web application (Re3gistry).

 

6.4 Initialize the database with your data

After testing the system with the sample database initialization scripts, the system will be cleaned up and configured for the specific user needs. You will need to customize the database-initialization.sql and the database-localization.sql in order to create the required elements (Register,ItemClass).

Database customization step:

  • To clean up the example database run the dtop-tabels.sql script.
  • Then, run the create-table.sql script.
  • Customize and run the database-initialization.sql to fit your needs. In particular, edit the lines regarding the "References", “Register” and “ItemClass” insert statements. You can take the example database-initialization.sql file and customize it with your data.
  • Customize and run the database-localization.sql in accordance to the new customized database-initialization.sql. This initialization file contains the language translation for the static items (Register name,  Register description, ItemClass name, …). You can take the example database-localization.sql file and customize it with your data.

 

6.4.1 Adding a register

To add a new register after the first system configuration, follow the steps below.

  • Add the entry in the Register table. Use the code below; remember that the register_owner_id, register_manager_id, register_control_body_id, submitter_id, contact_point_id, license_id, registry_id fields has been previously created with the database-initialization.sql script.
    INSERT INTO register VALUES ('<uuid>', '<base_uri>', '<register_uri_name>', '<register_owner_id>', '<register_manager_id>', '<register_control_body_id>', '<submitter_id>', '<contact_point_id>', '<license_id>', '<registry_id>', 'yyyy-mm-dd hh:mm:ss.000000', NULL);
     
  • Add the ItemClass related to this Register. Use the code below.
    INSERT INTO itemclass VALUES ('<uuid>', '<register_id>', '<itemclass_uriname>', <order_number>, <data_procedure_processing_order>, <parent_item_class_id (can be null)>, 'yyy-mm-dd hh:mm:ss.000000', NULL);
     
  • Add the required entry in the database localization file. Use the code below.
    Localization for the Register: INSERT INTO localization VALUES('<uuid>',NULL,NULL,'<language_id>','<register_id>',NULL,'<label>','<definition>','<description>',NULL,'yyyy-mm-dd hh:mm:ss.000000',NULL,NULL,NULL,NULL);
    Localization for the ItemClass: INSERT INTO localization VALUES('<uuid>',NULL,<itemclass_id>,'<language_id>','NULL',NULL,'<label>',NULL,NULL,NULL,'yyyy-mm-dd hh:mm:ss.000000',NULL,NULL,NULL,NULL);
  • Restart the system

The example provided below is a script to add the "Test Register" to the system.

INSERT INTO register VALUES ('3', 'http://localhost', 'test', '6', '5', '3', '7', '1', '2', '1', '2015-02-01 14:16:22.677696', NULL);

INSERT INTO itemclass VALUES ('5', '3', 'Test', 0, 2, NULL, '2015-02-01 15:40:30.457694', NULL);

INSERT INTO localization VALUES('10',NULL,NULL,'1','2',NULL,'Test Register','This is a test register','Test register definition',NULL,'2015-02-01 16:21:00.000000',NULL,NULL,NULL,NULL);

INSERT INTO localization VALUES('11',NULL,'5','1',NULL,NULL,'Test',NULL,NULL,NULL,'2015-02-01 16:21:00.000000',NULL,NULL,NULL,NULL);

 

After adding the Register to the database, remebber to create the XSLT related to this new register, in order to be correctly processed by the staticization system (refer to: https://ies-svn.jrc.ec.europa.eu/projects/registry-development/wiki/Re3gistry_1_1_documentation#483-XSLT).

 

6.5 Running the Re3gistry software

Once the system is up and running, it can be tested using the example import data file (provided in the project package).

  • Firstly, the database has to be filled using the Data module. To start the data procedure, go to the “Data” button in main the web interface menu.
  • Select the example data file, from the project package, insert the procedure description and run the procedure.
  • Once the import procedure is finished, go to Staticizer module (“Staticizer” button in the main menu - http://localhost:8080/Re3gistryExport) and by click on the “Check” button will be shown all the items imported from the last statizer procedure, now by click on the “Start” button will start the procedure of producing the standard files, the custom files using the xslt transformations, the VAR generator will also produce the files, and the last si the producing of the SOLR files.

NOTE: some of the xsl for the custom export (for performance reasons) uses the “base_uri” property and reads the xml via a http request to this URL. Please check that the http server is running and that the “base_uri” points to the local http server (edit the Operative System’s (OS) host file. This file can be place in different folder depending on the OS).

 

6.6 Setting up the web service

Move the exported files (saved in the “custom” folder) under the Apache HTTPD document root (or under a specific, previously configured, folder).

To use the content-negotiation the Apache HTTPD has to be configured, following the details provided on the Apache HTTPD official web site, see: http://httpd.apache.org/docs/2.2/content-negotiation.html

 

7 Creating new modules

To start creating a new module, it is important to understand the module’s structure by reading the chapter 5 and taking a deep look at one of the modules included in the package (for example the Data module or the Staticizer Module).

After understanding the structure, the development of a new module can be started from an IDE like NetBeans. The following guide is provided using the NetBeans IDE as example.

Step 1

Open the main project contained in the project package, following the instruction provided at chapter 6 (paragraph 6.3 “Using the source code”).

Step 2

Create a new project: right click on the project browser and click on “new project”.

Figure 10 - Creating new project from the project browser

Choose “Maven” -> “Java Application” in the opened window, then click the “next” button.

Figure 11 - Project type selection

In the following window, give a name to the new module; in this example the “NewModule” name is used.

Figure 12 - Module name

Step 3

Go to the newly created project and edit the POM file, to insert the dependency to the RegistryCommon module as shown at example 16 .

Exmaple 17 - POM edit example

<dependency>

           <groupId>eu.europa.ec.re3gistrycommon</groupId>

        <artifactId>Re3gistryCommon</artifactId>

        <version>0.4</version>

</dependency>

Step 4

The library component of the module is now ready. To start the development, create the required package and classes in the project’s source folder.

It is important to create the Class “Constants.java” under package “eu.europa.ec.newmodule.constant”. This is a common used Class containing all the constants related to this module.

This class shall contain at least the class name. An example is provided below.

Exmaple 18 - Constants.java example

package eu.europa.ec.newmodule.constants;

 

public final class Constants {

 

    /* ###################################### */

    /* ########## Common constants ########## */

    /**

     * The name of this module

     */

    public static final String MODULE_NAME = "NewModule";

   

}

Step 5

If the new module requires a dedicated interface, the user shall create it using the following instructions. Otherwise you can jump to Step 6.

Open the Web Pages folder in the Registry webapp project. Search the modules folder, right click on it and select “New” - > “Folder”.

Figure 13 - Creation of the module's folder

Name the new folder exactly as the previously created Java Application (at step 2).

 

Step 6

Create the module’s property file, inside the “Other sources – configurations/modules” project folder (in the file system this folder is located under Re3gistry/ WEB-INF/classes/configurations/modules) with the same module name (“NewModule.properties”).

Figure 14 - Module's properties file

The required property keys for all the new modules are shown below:

  • module.active=true – if set to true, the module is active and is loaded in the system.
  • module.menuitem.visible=true – if this property is true, the menu entry related to this module is shown in the main system webpage. If no module has this property as visible, the menu bar is not visible.
  • module.menuitem.label=Data – This property rapresent the label of the menu item in the system main menu.
  • module.homewidget.enabled=true – This property enable/disable the system to show the module widget in the home page.
  • module.dateformat=dd-MM-yyyy HH:mm:SS – The module dat/time format.
  • module.menuitem.order=2 – The order in which this menu item is shown in the main menu. The higher values mean to be displayed later.

Step 7

Add the localization files in the “Other sources” -> “src/main/resources/localization” folder. To do this, right click on the “Other sources” -> “src/main/resources” folder and select “New” -> “Folder”.

Figure 15 - Localization folder for the new module

Name the new folder “localization/NewModule”.

Figure 16 - Localization properties folder

 

Then create a file for each language using this naming pattern: “LocalizationBundle_[language_code].properties”.

The minimum set of property to be available in the localization files are:

  • The module’s title: common.pagetitle=New Module
  • The module’s description: main.description=Descriptive text

Figure 17 - Module's localization properties file

Step 8

Add into the logger configurations the entry relative to this new module. Open the logcfg.xml file under “Other sources” –> configuration.

Add the entry related to the new module as defined below. Note that this example is using maven profile properties. In this case the property key related to this logger file has also to be added to the pom file.

Exmaple 19 – Logger configurations

<!-- NewModule -->

<appender name="appender.NewModule" class="org.apache.log4j.DailyRollingFileAppender">

  <param name="File" value="${NewModule.log.file.dir}"/>

  <param name="DatePattern" value="'.'yyyy-MM-dd"/>

  <layout class="org.apache.log4j.PatternLayout">

  <param name="ConversionPattern" value="%d - %-5p - %-10c [%C{1}.%M:%L] %m%n"/>

  </layout>

</appender>

 

<logger name="NewModule">

  <level value="info"/>

  <appender-ref ref="appender.NewModule"/>

</logger>

Step 9

The module structure is now complete.

Now the next step is to create the required web pages and to start the development of the functionalities for the module (both in the library and in the web part).

The web pages shall be placed in the module’s folder root, created at step 5.

There are 2 file required:

  • main.jsp: contain the main web page of the module, with all the functionalities.
  • widget.jsp: this page represents a sort of summary that can be placed in the home page of the system.

Notes

The main structure of the modules can be easily understood by taking a look at the files from the modules available in the project package.

There are some particular module related code snippets, used in the web pages that are explained below.

  • ${module.localization['property.name']} – It is used to retrieve a specified text in the language selected by the user. The keys passed as argument to this method has to be defined in the module specific localization file (created at step 7).
  • ${module.properties['property.name']} – It is used to retrieve a specific properties related to this module. The keys passed as argument are defined in the module’s related properties file (defined at step 6).