Frequently Asked Questions

ESPON M4D

Revision History
Revision 12014-12-19 17:31:03

Abstract

This document proposes the frequently asked questions regarding the ESPON Database Portal and ESPON M4D Project.


Table of Contents

A. About
1. What is ESPON M4D?
1.1. Who has developed the ESPON Database Portal?
1.2. What is the goal of the ESPON Database project?
2. The ESPON Database Portal
2.1. How is structured the ESPON Database Portal?
2.2. What kind of indicators is it possible to query from the Search Interface?
2.3. How is organised the Search Interface?
2.4. What kind of indicators is it possible to query from the Zoom-in interface?
2.5. How is organised the Zoom-in interface?
2.6. What is the Core Database Strategy?
2.7. Why the Core Database Strategy?
2.8. What are the potential benefits of the Core Database Strategy?
2.9. How is it possible to obtain ESPON M4D Core indicators?
2.10. What is frequency of update of ESPON M4D Core indicators?
2.11. I need data from the previous ESPON Program (e.g. ESPON Project 1.3.1 on natural hazards). I did not find these indicators on the Search Interface? How get them?
2.12. Are indicators from the ESPON 2006 program updated?
2.13. What is the geographical coverage of data contains in the ESPON Search interface?
2.14. How do I declare a mistake in ESPON Database content (strange values, bad description of metadata, breakdown in time series etc.)?
2.15. Is a catalog of indicators contained in the ESPON Database available?
2.16. How to quote ESPON Data?
3. Restricted part of the ESPON Database Portal
3.1. Who can have an access to the restricted part of the ESPON Database Portal?
3.2. What is inside the restricted part of the Database?
3.3. How do I register?
4. Data delivery
4.1. Where uploading my dataset?
4.2. What are the possibilities existing for delivering the data produced by my project?
4.3. What is a key indicator?
4.4. What happens to my key indicators after it is delivered?
4.5. What is a zoom-in data?
4.6. What happens to my zoom-in data after it is delivered?
4.7. What is a background data?
4.8. What happens to my background after it is delivered?
4.9. What is the syntactic check?
4.10. What is an outlier?
5. Metadata processing
5.1. What is metadata?
5.2. Why are you using a generic metadata model?
5.3. How is ESPON metadata built?
5.4. Does ESPON metadata follow INSPIRE directives?
5.5. How can I create a Metadata file? Where is the Metadata documentation available?
5.6. Are some templates existing for delivering my datasets in the relevant ESPON format?
5.7. Do you have a metadata template for raster files (grid data)?
6. Support to data creation
6.1. Who shall I contact to have some help for my data and metadata creation?
6.2. Where can I find lists of territorial nomenclatures contained in the ESPON Database (NUTS, cities etc.)?
6.3. What kind of solution do you propose for estimating missing values?
6.4. How referencing missing values in my dataset?
7. Mapkit
7.1. Where can I download the ESPON mapkits?
7.2. In which electronic formats are ESPON mapkits available?
7.3. What can I find in all the ESPON mapkits?
7.4. How can I use the ESPON mapkits?
7.5. What are the projections chosen for the mapkits?
7.6. Why have you chosen this kind of projection and not another one?
7.7. How can I get the geometries for local units?
7.8. What shall I do if my maps do not fit with the ESPON mapkits?
7.9. The zoom-in mapkit displays an example for Chinese regions. Is it possible to have the same for other countries (Japan, USA, etc)?
7.10. I don't have any knowledge about mapping, do you have some tutorials available for cartographic purpose?
8. Local/Urban data
8.1. What are the data available at a LAU2 level for the ESPON Area?
8.2. What are the existing solutions for collecting local data?
8.3. How to access to the SIRE database?
8.4. Do EuroGeographics LAU2 geometries fit the SIRE data?
8.5. What are the links between the urban areas in the SIRE DB and Urban Audit areas?
8.6. There is no answer to my question(s) in that FAQ, what should I do?

1. What is ESPON M4D?

1.1. Who has developed the ESPON Database Portal?
1.2. What is the goal of the ESPON Database project?

1.1.

Who has developed the ESPON Database Portal?

The ESPON Database Portal has been developed by the ESPON M4D Project. M4D means “Multi-Dimensional Data Design and Development”. It is the ESPON project in charge of the ESPON database.

The ESPON Multi Dimensional Database Design and Development (M4D) project corresponds to the consortium working on the improvement of the ESPON Database for the period 2011-2014.

The consortium involves teams specialized in different fields:

  • UMS RIATE (Paris, France) is in charge of the coordination of the project. This team is specialised in cartography, spatial analysis, multiscalar data collection and estimation in Europe, European neighborhood and at a World scale. [contact: ]

  • LIG: Laboratoire d’Informatique de Grenoble (Grenoble, France) is in charge of the development of the application. This team is specialised in complex data modeling. [contact: ]

  • NCG: The National Centre for Geocomputation (Maynooth, Ireland) is specialised in outlier detection, data estimation and time-series modeling. [contact: ]

  • UMR Géographie-cités (Paris, France) is a team specialized in urban objects expertise and database integration. [contact: ]

  • UAB: Universitat Autònoma de Barcelona (Barcelona, Spain) is a team specialised in grid data. Among other, they have developed an OLAP cube which allows converting data coming from heterogeneous geographical delineations. [contact: ]

  • TIGRIS (Iasi, Romania) is a team specialised in local data. They have proposed innovative methods for displaying phenomena at local scales. [contact: ]

1.2.

What is the goal of the ESPON Database project?

The aim of the M4D Project (2011-2014) is to develop an operational and sustainable database through the following guidelines:

  • Extension of the schema of the database in order to store an enlarged collection of data, integrating new geographical objects which can be queried dynamically. Namely, users will be able to download and upload data in the SNUTS (Similar NUTS) delineation (for Eastern and Southern European neighborhoods), urban data (LUZ, MUA, FUA, UMZ), local data (LAU1 and LAU2 units) and World nomenclatures (UN, WUTS).

  • Improvement of the compliance with European and international standards for spatial referencing and data storage, in particular with the INSPIRE recommendations. See also the FAQ section entitled Metadata processing.

  • Improvement of the usability of the query and download interface. Two Search interfaces have been developed for different user profiles: policy makers, practitioners, researchers). The first one for querying regional data, world data and data on cities: the Search Interface using different path of investigations (search menus, filters). The second one for querying case study data (Zoom-in interface).

  • Indicator upload. A process for uploading new indicators and ensuring their quality has been defined and a platform – the tracking tool – has been developed to manage this data flow.

  • More interoperability between geographical objects. The ESPON M4D project aims to provide pivots methods (OLAP Cube) to load the database with basic indicators in different geographical or spatial delineations (cities, grids, administrative units).

  • New data in the different geographical delineations.The ESPON M4D aims to collect basic indicators for feeding the ESPON Database (cf the M4D Core indicators section).

  • Methods: As far as data collection, data estimation, and expertise are concerned, the M4D Project has created technical reports to describe how to solve specific problems of data integration that cannot be fully explained in the metadata files.

2. The ESPON Database Portal

2.1. How is structured the ESPON Database Portal?
2.2. What kind of indicators is it possible to query from the Search Interface?
2.3. How is organised the Search Interface?
2.4. What kind of indicators is it possible to query from the Zoom-in interface?
2.5. How is organised the Zoom-in interface?
2.6. What is the Core Database Strategy?
2.7. Why the Core Database Strategy?
2.8. What are the potential benefits of the Core Database Strategy?
2.9. How is it possible to obtain ESPON M4D Core indicators?
2.10. What is frequency of update of ESPON M4D Core indicators?
2.11. I need data from the previous ESPON Program (e.g. ESPON Project 1.3.1 on natural hazards). I did not find these indicators on the Search Interface? How get them?
2.12. Are indicators from the ESPON 2006 program updated?
2.13. What is the geographical coverage of data contains in the ESPON Search interface?
2.14. How do I declare a mistake in ESPON Database content (strange values, bad description of metadata, breakdown in time series etc.)?
2.15. Is a catalog of indicators contained in the ESPON Database available?
2.16. How to quote ESPON Data?

2.1.

How is structured the ESPON Database Portal?

The ESPON Database Portal is composed with different key elements:

  • Welcome page with a news section and newsletters to inform the community about the last data integrated, last developments, etc.

  • Search menu, the principle access to retrieve data produced by ESPON Projects (search Menu) via the Search Interface or the Zoom-in Interface.

  • Resources page, fed with technical reports, tools, data, mapkits, documentation.

  • Upload page, to automatically upload data by ESPON Projects.

  • Help page, from where it is possible to retrieve all information concerning the ESPON Database Portal, ESPON data and metadata etc.

2.2.

What kind of indicators is it possible to query from the Search Interface?

The Search Interface contains key indicators provided by the ESPON Projects that are most relevant for the EU Cohesion policy, mainly at NUTS2 or NUTS3 levels on various thematic fields (demography, environment, labour force etc.). These datasets cover all the ESPON Space at least (EU28+Iceland, Liechtenstein, Norway and Switzerland). It is also possible to query indicators related to other geographical objects: cities (MUA, FUA, UMZ), World countries or data on the European Neighborhood at regional level.

Please consult the ESPON Database Portal user manual at the following link:http://database.espon.eu/db2/usermanual?idDoc=64 for further information.

2.3.

How is organised the Search Interface?

Basically, the user can choose between four strategies to make his request:

  • By Theme: The research strategy is based on a thematic classification of indicators existing in the ESPON Database. The hierarchy is based on a combination of the standard EUROSTAT classification and specific targets of the ESPON program.

  • By Policy: The research strategy is governed here by the content of a selection of policy reports (e.g. EU 2020, Territorial Agenda 2020, ESPON Synthesis Report etc.) This criterion is supposed to be more useful for policymakers who are not looking for a specific thematic field but rather by a political issue (e.g. “smart growth”, “polycentrism”).

  • By Project: The research strategy concerns the name of the ESPON Project that has produced the information. This criterion appears relevant for users interested in extracting data related to a specific ESPON publication or a specific map where the name of the project is mentioned.

  • By Keyword: The research strategy is based on the keywords associated with the indicators of the ESPON Database (using mainly the GEMET thesaurus, e.g. policy instrument, cooperation, population dynamics etc.) and/or the indicator names. This criterion offers a general interoperability of the ESPON database in the framework of the INSPIRE directive.

Whatever their initial research strategy it is then possible for the user to refine his selection by the optional addition of specific filters.

  • General: Filter by statistical properties (count data, ratios, typologies, tables etc.)

  • Where? Filter by geographical coverage (Europe, Neighborhood, World) or territorial level (regional data, local data, cities, grids).

  • What? Filter by ESPON Projects, thematic or policy classifications of the indicator contained in the database.

  • When? Filter by time period.

Additional functionalities are also available (a metadata page, a basket page, an inventory of indicators contained in the database.

Please consult the ESPON Database Portal user manual at the following: http://database.espon.eu/db2/usermanual?idDoc=64 for further information.

2.4.

What kind of indicators is it possible to query from the Zoom-in interface?

The Zoom-in interface contains case-study data provided by ESPON Projects. This interface is mainly fed by ESPON Projects under priority 2 (targeted analysis), but not only. Generally, the datasets downloadable from this Search interface contains data on a specific thematic fields for several territories at a local scale (LAU2). The Zoom-in interface is available at the following link http://database.espon.eu/db2/faces/jsf/case-study-map.xhtml.

2.5.

How is organised the Zoom-in interface?

The Zoom-in interface is built as a map (using OpenStreetMap technology) from where it is possible to see the location of the case-study by a pins symbol. By clicking on the pin, the user can display the metadata and download the data and possibly the geometries of the case-study.

2.6.

What is the Core Database Strategy?

The Core Database Strategy (CDS) is a ESPON M4D proposal to support the second objective through the creation of a specific part inside the ESPON database. The idea is to deliver a label “Core data” to a very limited number of indicators that are most relevant for EU Cohesion Policy and should be regularly updated in the future for the purpose of territorial monitoring but also cartographic applications like Web-mapping tool, HyperAtlas, etc. Basically, the CDS is based on five bullet points:

  • Keep only count variables: In concrete terms, it means that a variable like GDP per capita is initially excluded from the core database as it is not a count variable but the ratio of two count variables (total population and total amount of GDP (in $ or pps). As a consequence, the CDS focuses only on a limited number of count variables.

  • Store formula of indicators of interest derived from count variables (e.g. a dependency ratio is a ratio between “young” and “old” divided by “adult”. They are therefore virtual data that are computed on the fly when a user requests it.

  • Enlarge time series of count variables in past and future with estimation of missing values. The core data should be absolutely complete in order to make possible the computation of total aggregates at different levels.

  • Develop automatic procedure to exchange count variables between geometries of various types: For example, thanks to the ESPON OLAP Cube, the GDP data at NUTS 3 level will be firstly estimated at grid level (using ancillary variable like JRC 2001 or GEOSTAT 2006 population grids) and then re-aggregated to other geometries like water basin, FUA, etc.

  • Propose innovative procedures of multi-representation and multi-level analysis of indicators for territorial monitoring and political decision: Whatever the geometry of interest (cities, regions…) is, an isolated indicator can be transformed into a family of variants targeted to answer to specific political questions (comparison to the EU average, territorial discontinuities…).

2.7.

Why the Core Database Strategy?

The ESPON Scientific Platform (Priority 3) has to fill two complementary but also challenging objectives. On the one hand, (1) to store the memory of the production of the ESPON 2013 program because each project is supposed to deliver data and to contribute to the elaboration of a giant repository of archives. On the other hand, (2) to propose monitoring tools based on selected indicators available in long term time series in the past and likely to be reproduced in the future.

The two objectives are certainly complementary, because it is on the basis of research realized by ESPON projects that new innovative indicators can be discovered. But they are also challenging because not all the indicators produced by ESPON projects are eligible to the status of indicator of prior interest that should be updated in the future. And it can also happen that very interesting indicators produced by an ESPON project are available only at one time and are too difficult to be regularly updated because they are based on data or computer application with private copyrights.

2.8.

What are the potential benefits of the Core Database Strategy?

The CDS may become an efficient strategy for the monitoring of regions and cities within the ESPON area. It focuses precisely on the indicators that can be regularly updated in past and future. It may be useful to understand the current dynamics and the forecast evolutions.

The fact that CDS only stores a limited number of count variables allows yearly updates. And then, all the other derived or complex indicators will be automatically recomputed as they are based on a formula. Yet, any innovation adopted by policy makers (such as maps of deviation, analysis of discontinuities, potential based on time distance…), can be easily transposed to the most recent data.

Last but not least, when modification occurs in the geometry of spatial units (change of NUTS delineations for instance), the revision of indicators is feasible by the use of a limited number of count variables and formula.

[Note]

More information? The Core Database Strategy is presented in details in a dedicated technical report http://database.espon.eu/db2/resource?idCat=32

2.9.

How is it possible to obtain ESPON M4D Core indicators?

The easiest way to query ESPON M4D Core indicators is to go into the Search interface, query by project and select “ESPON M4D project”. Then, open the general filter and select Core data only (by default, the all option is activated).

2.10.

What is frequency of update of ESPON M4D Core indicators?

Updating such indicators is a huge work since it implies to estimate missing values for a long time-period. Consequently, ESPON Core indicators have been updated more or less yearly (depending of Eurostat updates) during the lifetime of the ESPON M4D Project (2011-2014). Afterwards, it is the ESPON Coordination Unit which will manage the update of such indicators.

2.11.

I need data from the previous ESPON Program (e.g. ESPON Project 1.3.1 on natural hazards). I did not find these indicators on the Search Interface? How get them?

The most representative indicators produced during the ESPON 2006 Program (the previous one) have been integrated into the current version of the ESPON Database. But these indicators are available only for obsolete NUTS versions (1999 and 2003). For querying such data, open the where? filter and select the NUTS1999 and NUTS2003 nomenclature.

2.12.

Are indicators from the ESPON 2006 program updated?

Some typologies, developed during the ESPON 2006 Program, have been updated in the NUTS 2006 or the NUTS 2010 delineation with recent data (map updates).

For more information concerning these map updates, please contact the ESPON Coordination unit () or have a look to subsection of the ESPON Website dedicated to such updates (http://www.espon.eu/main/Menu_Projects/Menu_ScientificPlatform/Menu_MapUpdates).

2.13.

What is the geographical coverage of data contains in the ESPON Search interface?

Most of the data delivered by ESPON projects covers at least the ESPON Area (e.g. European Union plus Switzerland, Liechtenstein, Norway and Iceland). Some datasets include also the EU Candidate Countries and regions of the European Neighborhood. Please have a look to the Where? filter to select the geographical coverage you are looking for.

2.14.

How do I declare a mistake in ESPON Database content (strange values, bad description of metadata, breakdown in time series etc.)?

Although the data has been checked in detail, ESPON data may contain some strange values. Please do not hesitate to contact us by sending an email to the M4D Manager (), who will forward to the concerned people.

2.15.

Is a catalog of indicators contained in the ESPON Database available?

The overview of the indicators contained in the Search interface of the ESPON Database Portal is available at the following link: http://database.espon.eu/db2/overview

2.16.

How to quote ESPON Data?

Please consult the Terms and conditions of use at the following link:http://database.espon.eu/db2/terms

3. Restricted part of the ESPON Database Portal

3.1. Who can have an access to the restricted part of the ESPON Database Portal?
3.2. What is inside the restricted part of the Database?
3.3. How do I register?

3.1.

Who can have an access to the restricted part of the ESPON Database Portal?

To access the restricted part of the ESPON Database, it is necessary to be involved in the ESPON Program as a lead partner, project partner or Monitoring Committee. Please note that most of data available in the ESPON database is downloadable without registration requirement.

3.2.

What is inside the restricted part of the Database?

The restricted part of the Database proposes access to:

  1. The tracking tool for uploading and following your Data and Metadata.

  2. The restricted data and datasets. Please note that only 1 % of data is restricted either for validation reasons or protected by copyright (99 % is free of use without registration).

  3. The ESPON Mapkits.

  4. Some resources provided by ESPON Projects and external data providers.

3.3.

How do I register?

You must fill a slight registration form available from a link on the Login menu item page of the ESPON Database Portal.

This registration form contains a small number of fields required for establishing the identity of the user and her/his relationship with ESPON. Once the administrator has checked the information and validated the request, you receive an automatic notification confirming the possibility to access to the restricted part of the ESPON Database Portal.

4. Data delivery

4.1. Where uploading my dataset?
4.2. What are the possibilities existing for delivering the data produced by my project?
4.3. What is a key indicator?
4.4. What happens to my key indicators after it is delivered?
4.5. What is a zoom-in data?
4.6. What happens to my zoom-in data after it is delivered?
4.7. What is a background data?
4.8. What happens to my background after it is delivered?
4.9. What is the syntactic check?
4.10. What is an outlier?

4.1.

Where uploading my dataset?

ESPON Projects are now requested to upload their data through the “Upload” page of the ESPON Database Portal (http://database.espon.eu/db2/upload). You have first to be logged for accessing to this part of the ESPON Database Portal.

4.2.

What are the possibilities existing for delivering the data produced by my project?

The ESPON P1, P2 and P3 Projects are obliged to deliver all data collected and produced within their project. Depending of the nature of the data itself, three possibilities have been implemented for the data delivery:

  • Key indicators, covering the entire ESPON Space;

  • Zoom-in data, covering the case-studies;

  • Background data, covering all data produced by the project.

For a complete description of these possibilities, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.3.

What is a key indicator?

The key indicators are innovative indicators highly relevant for policy making and should cover the entire ESPON Space (EU28+4). These indicators will be the only ones searchable from the query interface.

The ESPON projects deliver in principle the indicators related to the maps included in Part B of the (Draft) Final Report - around 10 indicators. In case a typology or composite indicator is included, the data and methodology used to build it should also be delivered.

The requirement in terms of data and metadata is high for this delivery and the ESPON Projects are requested to upload the data via the Upload page.

The key indicators delivery has to follow the ESPON Data and Metadata specifications. To build a strong and efficient query interface, these indicators will be checked in depth before integration.

For a complete description of the key indicator, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.4.

What happens to my key indicators after it is delivered?

Before being integrated into the ESPON Database, your data delivery must succeed a workflow. This is managed by the tracking tool. It is composed of the following different steps which concern your "Key indicators" only:

  1. Data and metadata files upload. When you deliver your data and metadata, it activates a dedicated module in the Upload part of the Portal. It means that the data integration process is started and the tracking toll is activated for your project. A notification is sent both to the ESPON Coordination Unit and to the M4D team.

  2. Syntactic checking. This step aims to check the compliance of your data and metadata with the ESPON Database profile. In concrete terms, it checks if all the mandatory fields of the ESPON data and metadata are correctly filled. This control is done automatically when you upload your dataset. This is the only compulsory step of the data integration process.

  3. Semantic checking. Your dataset in transferred to the semantic checker (a M4D team until 2014) in order to analyse the content of the data and metadata (and namely the free-text fields). The aim of this step consists in analysing if all the indicators of the dataset are correctly described and understandable by a large public. The result of the semantic check is a short report which is uploaded on the tracking tool. You have to validate or not the report.

  4. Outlier detection. At this step, an outlier detection tool will be run on your dataset. Your dataset will be checked by multiple statistical methods in order to define if it contains any outliers. The result of the outlier detection is a report which is uploaded pn the tracking tool. At this stage, you can make the choice to validate outlier detection or not (in other terms say that the outliers highlighted can be considered as “normal values”).

  5. Integration into the DB. Those three checks allow us giving a stong expertise on the quality of the datasets delivered by the project. For integrating your dataset in the database and make it available for download through the web interface, we lastly need the agreement both of your project and the ESPON Coordination Unit.

[Note]

For a complete description of the data integration process, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.5.

What is a zoom-in data?

Besides the key indicators delivery, some ESPON Projects (in particular for Targeted Analysis, but not only) analyze specific territories of the ESPON Area at local scale. To make this kind of complementary and very interesting data easy accessible, a case-study interface will be developed.

To set up this interface, the projects are requested to deliver their most representative data, their geometries (in a shape file format) and a documentation highlighting the content of the data and geometries (following a dedicated template).

Regarding to this delivery, the M4D project will only check if it is possible to map the data and if all mandatory fields of the documentation file are correctly filled.

For a complete description of the zoom-in data, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.6.

What happens to my zoom-in data after it is delivered?

Zoom-in delivery must be sent both to your TPG Project officer and the ESPON M4D manager (). When zoom-in data is delivered, a compliance check is organised by the M4D Project (TIGRIS team in particular) in order to check that:

  • The codes of the territorial units contained in the geometries and the dataset are the same (is it possible to make a map?)

  • The geometries are georeferenced (is it possible to display the case-study on the ESPON Mapkit?)

  • All the mandatory fields of the documentation file are correctly filled.

At the end of the compliance check, a notification is sent both to the coordinator of the project and to the project officer. It means that the zoom-in delivery will be available from the zoom-in interface.

For a complete description of the data integration process, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.7.

What is a background data?

In order to fill their contractual obligations and to make all data as a coherent set available, each ESPON Project has to deliver a zip file that contains all data, metadata and geometries (if different than the usual ones delivered via ESPON) used in the project.

This zip file is considered as an annex to the final report of the project and is stored on the ESPON Website project page.

For a complete description of the background data, please have a look to the how to deliver my data document and the presentation made by the ESPON M4D Project in December 2012 during the ESPON Seminar in Cyprus which are available in the upload menu of the ESPON Database Portal.

4.8.

What happens to my background after it is delivered?

You are free to deliver your background data in the format you want. However, it must be contained in a .zip format which does not exceed 100 Mo.

That being said, a template is available under the upload part of the ESPON Database Portal. If your background delivery contains files in various formats (grids, datasets in a .xls format, documentation etc.), we strongly suggest you to organize your files in folders and create a document “read me first” to help external users to manage your background data.

4.9.

What is the syntactic check?

The data check is the first step of the dataset integration process. It aims at insuring a coherent set of data together with their associated metadata. The data check will perform automatic consistency checking (and when possible, corrections) in the uploaded data file, concerning for instance:

  • the general data file structure (expected column headers, row order...);

  • the expected types of values for the indicators (for instance, a failure occurres if some decimal is encountered while an integer is expected);

  • the consistency of labels in the columns headers (indicators, data sources labels, time periods, etc).

4.10.

What is an outlier?

An outlier is a value which can be considered as unusual. An exceptionally high or low value when compared with the other values for an indicator could be an aspatial outlier. An exceptionally high or low value when compared with the values of the neighbouring areas could be a spatial outlier. We can test not just indicators on their own, but also combinations of indicators. If an indicator is considered to be an outlier across the range of aspatial and spatial tests, this provides a measure of the weight of evidence in favour of it being a candidate for further attention. Outliers might arise from errors in the data entry to integration processes, but they might also be legitimate values.

[Note]

More information? Have a look to the ESPON M4D Newsletter (Decemeber 2012) and to the Technical report "Spatial analysis for quality control, available in the resource part of the ESPON Database Portal.

5. Metadata processing

5.1. What is metadata?
5.2. Why are you using a generic metadata model?
5.3. How is ESPON metadata built?
5.4. Does ESPON metadata follow INSPIRE directives?
5.5. How can I create a Metadata file? Where is the Metadata documentation available?
5.6. Are some templates existing for delivering my datasets in the relevant ESPON format?
5.7. Do you have a metadata template for raster files (grid data)?

5.1.

What is metadata?

The information which describes the data is known as metadata. Metadata describes how, when, and by whom, a particular set of data was collected, and how the data are formatted. Metadata is essential for understanding information stored in the ESPON Database.

5.2.

Why are you using a generic metadata model?

Information about the data in the database is vital. Building a rich database would be useless without a strong effort to describe precisely the data that have been gathered and integrated within the database. Creating and organizing metadata is therefore an additional, important, and universal challenge for the ESPON community.

To be useful for ESPON projects and other end-users, data should always be accompanied by metadata, including information about its quality and sources. It is also particularly important that the metadata are created in a manner that is consistent with international (ISO) and European (INSPIRE) standards in order to ensure the use of the database in the longer-run and to make it compatible with other national and international database initiatives.

To ensure correct data processing and integration into the ESPON 2013 Database, the ESPON metadata specifications must be carefully respected by all the data providers participating in the project and by the organizations/persons who intend to create new software implementations interacting with the ESPON Database.

5.3.

How is ESPON metadata built?

In the context of the statistical results produced by different ESPON projects, data are represented by sets of socio-economic indicator values that have spatial, temporal and source bindings. In that order, the ESPON Abstract Metadata Model is based on four elements which have to be systematically filled:

  • Dataset information: an ESPON dataset can be identified by its name, publication date and the reference to the ESPON project that produced it.

  • Spatial binding (described in the dataset sheet): the Spatial Binding element groups two composite properties describing the spatial relation between the data part of the dataset and the statistical/spatial units used. The first property, Geographic Location declares the geographic area where all the territorial units of the dataset are located. This property is required by the INSPIRE directive for any spatial dataset. The second property, Nomenclature, enumerates the nomenclatures of geographical or statistical units that are used by the dataset.

  • Indicator identification: the Indicator Identification element describes the socio-economic indicator used in the dataset (methodology, unit of measure, thematic structuring). It can be considered as the most important element of the metadata. A dataset cannot be considered as such if there are no indicators specified in it and they are not described by the metadata.

  • Source reference: the Source Reference element references the origin of the data existing in the dataset, citing the publication, the organization that produced the data and other significant pieces of information, always related to its origin.

Figure 1. Relationship between the Data and the Metadata

Relationship between the Data and the Metadata

This ESPON Metadata Model is adapted for key indicators only and must be filed and delivered in a Excel format. As displayed in the figure above, the data sheet and the metadata sheets are linked by some elements (indicator code, territorial units classification, temporal extent, data source) which have to be filed both in the data and the metadata part of a ESPON Dataset. For a full overview of the ESPON Metadata Model, please consult the ESPON Metadata specifications, under the help part of the ESPON Database Portal.

5.4.

Does ESPON metadata follow INSPIRE directives?

The implementation of the abstract metadata model follows the normalized ISO-19115 INSPIRE-compatible XML document.

5.5.

How can I create a Metadata file? Where is the Metadata documentation available?

The following reference documents provide useful information for editing in a correct way the ESPON Metadata:

  • ESPON Metadata specifications (http://database.espon.eu/db2/metaspecifs): Describes in detail each field of the ESPON metadata. We also strongly suggest you to have a look to the annexes of this specification, which provide the possibilities existing for coding some metadata fields (thematic structure existing for coding indicator, definition of the statistical type of the indicator etc.)

  • How to deliver my data? (http://database.espon.eu/db2/htdmd) Among other, this documentation explains how coding ESPON Metadata on the basis of concrete examples.

  • Templates are also available (http://database.espon.eu/db2/upload). A empty Excel template describes all the mandatory fields of the ESPON Metadata and their related reference in the ESPON Metadata specifications. It contains also pre-filed files showing the input metadata of datasets already included in the Search interface of the ESPON Database.

5.6.

Are some templates existing for delivering my datasets in the relevant ESPON format?

The ESPON M4D project proposes some templates (http://database.espon.eu/db2/upload) to help the metadata edition: A empty Excel template describes all the mandatory fields of the ESPON Metadata and their related reference in the ESPON Metadata specifications. It contains also pre-filed files showing the input metadata of a sample of datasets already included in the Search interface of the ESPON Database.

5.7.

Do you have a metadata template for raster files (grid data)?

Unfortunately, the ESPON M4D Project has not developed such template. We strongly suggest you to use the Geonetwork metadata editor, adapted for environmental data, for editing this kind of metadata. It is an online metadata editor which is fully INSPIRE compliant and generate .xml files you can join to the layers you may deliver.

[Note]

More information? The Geonetwork metadata editor is available on-line: http://www.geonetwork-opensource.org

6. Support to data creation

6.1. Who shall I contact to have some help for my data and metadata creation?
6.2. Where can I find lists of territorial nomenclatures contained in the ESPON Database (NUTS, cities etc.)?
6.3. What kind of solution do you propose for estimating missing values?
6.4. How referencing missing values in my dataset?

6.1.

Who shall I contact to have some help for my data and metadata creation?

Until December 2014, you can contact the ESPON M4D manager if you have any question regarding to the data creation process. But please not that it is not the work of the M4D project to gather your data and create your data and metadata files.

After 2014, please contact the ESPON Coordination Unit.

6.2.

Where can I find lists of territorial nomenclatures contained in the ESPON Database (NUTS, cities etc.)?

The list of territorial units used in the Search interface of the ESPON Database Portal are available under the help part of the ESPON Database Portal: http://database.espon.eu/db2/spatialspecifs But please note that it does not necessarily fit fully with the so-called “official” nomenclatures proposed by Eurostat. For instance, the NUTS nomenclature used in the ESPON Program combines the "official" NUTS nomenclature (for the EU28) and the codes for the EFTA and Candidate Countries nomenclature, which has not exactly the same status than the previous one.

To consult the official list of NUTS codes, please have a look on the Eurostat Web site: http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction

6.3.

What kind of solution do you propose for estimating missing values?

We cannot provide a general solution for estimating missing values; different possibilities exist taking into account the data availability and the purpose of your statistical analysis. The annex 2 of the technical report on the Core Database Strategy (http://database.espon.eu/db2/resource?idCat=32) proposes some alternative solutions which are based on the results of the ESPON Data Navigator II project (ESTI model).

Indeed, it is very important to mention as precisely as possible the methodology used in estimating your data in the metadata (source sheet): if strange or unusual values appear in the dataset, it is helpful for us to be able to understand the steps of data creation.

6.4.

How referencing missing values in my dataset?

Two possibilities exist: n/a for “not available” or n/r for “not relevant” (e.g. number of sea-port for non-coastal regions). Do not put other valuers like -9999 or 0 or nothing.

For further information, please follow the recommendations made in 3.3.4 of the ESPON Metadata Specifications.

7. Mapkit

7.1. Where can I download the ESPON mapkits?
7.2. In which electronic formats are ESPON mapkits available?
7.3. What can I find in all the ESPON mapkits?
7.4. How can I use the ESPON mapkits?
7.5. What are the projections chosen for the mapkits?
7.6. Why have you chosen this kind of projection and not another one?
7.7. How can I get the geometries for local units?
7.8. What shall I do if my maps do not fit with the ESPON mapkits?
7.9. The zoom-in mapkit displays an example for Chinese regions. Is it possible to have the same for other countries (Japan, USA, etc)?
7.10. I don't have any knowledge about mapping, do you have some tutorials available for cartographic purpose?

7.1.

Where can I download the ESPON mapkits?

ESPON Mapkits will be accessible directly on the resource part of the ESPON Database Portal http://database.espon.eu/db2/mapkits. The access to this resource is limited to registered users.

7.2.

In which electronic formats are ESPON mapkits available?

Each mapkit will be available in the following three formats:

  • ARCGIS mxd document + shapefiles (ESRI)

  • Quantum GIS (QGIS) which is a user friendly Open Source Geographic Information System (GIS) licensed under the GNU General Public License. Please consult http://www.qgis.org.

  • Philcarto which is a free tool dedicated to thematic mapping and spatial analysis. The application and the documentation are available at the following URL: http://philcarto.free.fr/.

7.3.

What can I find in all the ESPON mapkits?

The ESPON Map-Kit Tool is composed of a set of six specific mapkits, adapted for different geographical levels, from local to global. In any case, geometries are available in shapefile (.shp) format. Layouts are provided in mxd (ArcGis), qgt (Quantum GIS Template) and ai (template for Philcarto). A more detailed description is available in the mapping guide available at http://www.espon.eu/main/Menu_ToolsandMaps/MappingGuide/.

7.4.

How can I use the ESPON mapkits?

To make maps with the mapkits, you need to use one of the 3 following applications: Quantum Gis, ArcGis, Philcarto.

7.5.

What are the projections chosen for the mapkits?

For each mapkit, the projection depends on the covered area:

  • European mapkits: ETRS-LAEA 52N 10E: EPSG 3035.

  • Global Mapkit: Polar (north).

  • Euromed mapkits : LAEA 50N 18E.

  • Zoom in mapkit : adapted local projection.

  • Zoom out mapkit : adapted regional projection.

7.6.

Why have you chosen this kind of projection and not another one?

The projection chosen for the European mapkit is the ETRS-LAEA 52N 10E. It is the usual projection used for mapping the European Union. Moreover, it ensures the interoperability with other geographical objects such as grids from EEA (European Environment Agency). As a rule, the projection chosen for the mapkits aims to fit the covered area (e.g. national spatial reference system).

7.7.

How can I get the geometries for local units?

Please contact your project officer to ask an access to the ESPON Intranet where this information provided by Eurogeographics and GISCO has been uploaded.

[Note]

Please note that the ESPON GEOSPECS project has made an attempt to create a LAU2 shapefiles without use constraint (for more information, have a look to the M4D Newsletter of December 2012).

7.8.

What shall I do if my maps do not fit with the ESPON mapkits?

Theoretically, if you use the ESPON mapkit, your maps will respect the EPSON design. Nevertheless, if it is not the case, you can read the mapping guide to know the elements that absolutely have to appear on the map layout, please consult http://www.espon.eu/main/Menu_ToolsandMaps/MappingGuide/.

7.9.

The zoom-in mapkit displays an example for Chinese regions. Is it possible to have the same for other countries (Japan, USA, etc)?

This zoom-in mapkit is actually an example of a layout. If you want to have the same layout on another area, you just have to replace the supplied geometries with yours and adapt the projection to the covered area.

7.10.

I don't have any knowledge about mapping, do you have some tutorials available for cartographic purpose?

To help you, some guidelines are available in a Technical Report on cartography at the following URL: http://www.espon.eu/main/Menu_ToolsandMaps/MappingGuide.

8. Local/Urban data

8.1. What are the data available at a LAU2 level for the ESPON Area?
8.2. What are the existing solutions for collecting local data?
8.3. How to access to the SIRE database?
8.4. Do EuroGeographics LAU2 geometries fit the SIRE data?
8.5. What are the links between the urban areas in the SIRE DB and Urban Audit areas?
8.6. There is no answer to my question(s) in that FAQ, what should I do?

8.1.

What are the data available at a LAU2 level for the ESPON Area?

Different types of resources can be mobilised by the ESPON Projects to get data at local level.

First of all, information provided by external data providers (GISCO, Eurogeographics, Eurostat, DG REGIO) which consists mainly by layers and basic data (SIRE Databases). To have a complete listing of available information, please have a look to the M4D Newsletter of June 2012. These datasets are available under the ESPON intranet.

Secondly, we suggest you to go to the resource part of the ESPON Database Portal, subsection local data, where some datasets related to local data have been uploaded. In particular, datasets provided by the ESPON GEOSPECS Project and the ESPON Database 1 project: the ESPON GEOSPECS and the ESPON Database 1 project have gathered data for all the ESPON Area.

  • The ESPON GEOSPECS project has gather data from National Statistical Institutes and GIS calculations at LAU2 level in the field of demography, employment, access to airports, physical environment and time distance to nearest urban centres (More information in the M4D Newsletter of December 2014).

  • The ESPON Database 1 project provided some indicators at LAU2 based on GIS calculations, available for ESPON Projects (you need to be logged in the database): Corine Land Cover, disaggregation of GDP at LAU2 level etc.

8.2.

What are the existing solutions for collecting local data?

Different resources, with different spatial coverage, LAU2 versions, and degree of quality can be considered of domain of interest. The SIRE database provides the most common and harmonized resources: It includes census data (around 1991 and around 2001 with employment household and dwelling data) from New Member States at LAU 2 level. There are also some census data for 1981 for the Member States at that date.

[Warning]

The quality of some indicators contained in the SIRE Database is sometimes uncertain. We suggest to potential users of this database to check carefully this data (in particular for specific thematic fields such as commuters) with "official values" coming from from National Statistical Institutes.

8.3.

How to access to the SIRE database?

The European Infra-Regional Information System (SIRE) database, as well as information delivered on the Communication & Information Resource Centre Administrator (CIRCA) Portal is available in the ESPON Intranet, managed by the ESPON Coordination Unit. Please contact your project officer for more information about the conditions to download the SIRE database since its condition of use is extremely limited

8.4.

Do EuroGeographics LAU2 geometries fit the SIRE data?

The answer depends on the reference year:

  • For Census 1981 and 1991: in the archives of the GISCO database, there are population data for 1981 (EU-12) and 1991 (EU-15) as well as geometries for these years. The matching between the population data and geometry is not one-to-one since the reference dates are not really comparable. Population data in GISCO comes from SIRE.

  • For Census 2001: the GISCO feature dataset "COMM_CENS_2001" and census data in SIRE provide a perfect one-to-one link between communal census data and geometry (EU-28,EFTA).

  • For Population data 2006: the GISCO has performed a preliminary exercise to match 2006 communal geometry to population. This data has not been yet included in the GISCO database.

[Note]

Please have a look to the second M4D Newsletter for further information. It is available under the homepage of the ESPON Database Portal.

8.5.

What are the links between the urban areas in the SIRE DB and Urban Audit areas?

Urban areas in SIRE are morphological objects (continuous built-up areas delineated from density or maximal distance criteria), while UA Larger Urban Zones (LUZ) are based on administrative and/or functional urban units (See Technical Report “LUZ specifications, UA 2004”, from ESPON Data Base Project). These sources are not always comparable. It can be possible that there is more than one morphological urban area within one LAU2 unit. Data for morphological urban areas that are contained in SIRE are not available in many Member States (Some examples of countries for which data are available: France, Italy, Portugal, United Kingdom, etc).

8.6.

There is no answer to my question(s) in that FAQ, what should I do?

Please contact the M4D project: .

A. About

This document is part of the ESPON 2013 Database Phase 2 project, also known as M4D (Multi Dimension Database Design and Development). It was generated on the 2014-12-19 17:31:04, from the sources of the m4d forge imag project at the svn rev 2420.

This document has been written by UMS RIATE and LIG STeamer M4D Partners.

For any comment question or suggestion, please contact .

Colophon

Based on DocBook technology [1], this document is written in XML format, sources are validated with DocBook DTD 4.5CR3, then sources are transformed to HTML and PDF formats by using DocBook xslt 1.73.2 stylesheets. The generation of the documents is automatized thanks to the docbench LIG STeamer project that is based on Ant [2], java [3], processors Xalan[4] and FOP [5]. Note that Xslt standard stylesheets are customized in order to get a better image resolution in PDF generated output for admonitions icons: the generated sizes of these icons were turned from 30 to 12 pt.



[1] [on line] DocBook.org (last visit: July 2011)

[2] [on line] Apache Ant - Welcome. Version 1.7.1 (last visit: July 2011)

[3] [on line] Developer Resources For Java Technology (last visit: July 2011). Version 1.6.0_03-b05.

[4] [on line] Xalan-Java Version 2.7.1 (last visit: 18 november 2009). Version 2.7.1.

[5] [on line] Apache FOP (last visit: July 2011). Version 0.94.