The update of our EOxMaps (opens new window) infrastructure has been long overdue. Finally, in 2021 we have completely reworked our former EOxMaps infrastructure, which allows us to generate, alter and update EOxMaps more frequently than before.

Firstly, we focused on updating the somewhat outdated map label layers.

Data Foundation - OpenStreetMap & Natural Earth

Like with most OpenData maps today we have opted to use a mix of OpenStreetMap (opens new window) and Natural Earth (opens new window) as sources for our map labels update.

PostgreSQL + PostGIS Database

The OSM dataset is imported into a PostGIS database via an import tool: imposm3 (opens new window). Here we specify the nessesary parameters to extract the attributes needed by the map layers from the OpenStreetMap .pbf file.

Given the size of the imported data and the complexity of some spatial features, we have generalized some layers to make the vector tiles generation more performant.

Centerlines

For large geographical features like oceans, lakes, mountain ranges, etc. we use the custom centerlines of these objects for labeling. The EOX has a custom solution to generating these centerlines as described in the "Curved labels" (opens new window) article. The output of these centerlines is then imported into the PostGIS database.

Custom Tiles-API as requests handler

Since numerous tools are used in this pipeline, we were forced to consolidate these tools under a single endpoint that handles input and output (I/O) from the database over the vector tiles to rendering of raster tiles in the final map layers.

For this data throughput management we have utilized FastAPI (opens new window) to serve vector-tiles and rendered raster tiles as well as some metadata around the whole EOxMaps infrastructure.

FastAPI docs for the EOxMaps Tiles-API

FastAPI Introduction

For those unfamiliar with FastAPI (opens new window) here is a python3 'Hello World' (opens new window) on how to set up a simple API instantly.

1. Save a main.py python file with:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello World"}

2. Run the app from bash terminal via:

$ uvicorn main:app --reload

3. Open a browser at localhost: http://127.0.0.1:8000 (opens new window). Response should be following json output: {"message": "Hello World"}.

Vector Tiles (Mapbox-Vector-Tiles)

At EOX we are already familiar with vector tiles (MVT (opens new window)) as for instance this article from last year shows: Vector tiles with dynamically updated style in Leaflet LPvis (opens new window). Having already had some idea on how we should proceed in handling the data from the PostGIS database, we decided that apart from our agriculturally driven projects we will use a similar data pipeline here as well.

Vector tiles are extremely handy for anything with high I/O vector data throughput. As the files are independent from each other, this enables convenient concurrently driven utilization. There are some PostGIS native functions that create MVTs on the fly: ST_AsMVT (opens new window) and ST_AsMVTGeom (opens new window). If you are comfortable with the SQL language, you can customize and add some handy SQL functions to clip and reproject the geometries and edit attributes of the vector layers. These operations can increase or decrease the performance of the MVT generation, so be careful.

Finally, we serve the optimized vector tiles to the renderer via our Tiles-API.

TileServer GL as renderer

To make a smooth transition between our former and current maps infrastructure we still do render raster (.png,.jpg) tiles to serve them via our WMS/WMTS services. For this we have used TileServer GL (opens new window) written by Maptiler (opens new window). This tool is based on the open version of Mapbox-GL-Native (opens new window).

TileServer GL allows for an easy rendering of vector tiles into raster tiles. It also comes with some handy configuration options (opens new window) that give somewhat more control over the whole rendering procedure. It also provides vector tiles (source data), styles and rendered tiles (WMS/WMTS) endpoints (opens new window). This is particularly handy if you don't want to handle the requests yourself like we do via the Tiles-API.

Mapbox-GL Styles

The final layer look is defined in a Mapbox Style File (opens new window), which is a json file specifying the outcome of your map.

This is the most time-consuming part where fiddling with the map details comes into play. There are numerous ways on how to configure and review your styling; our primary style creation tool was maputnik (opens new window). Maputnik allows editing Mapbox Style Files (opens new window) in a nice map editor and exporting the final style so that you can use it in the TileServer GL endpoints.

Example of a mapbox-gl-js json style config:

{
  "version": 8,
  "name": "osm_borders", // name of the style
  "metadata": { // some metadata information, this can be whatever you need
    "name": "osm_database",
    "version": 1.0,
    "timestamp": "2022-03-08",
    "info": "OSM Borders Tutorial",
    "sources": ["openstreetmap"],
    "maputnik:renderer": "mbgljs"
  },
  "sources": {       // sources where to look for vector tiles,
    "osm_borders": { // in this case we use the regular EPSG:3857 MVT grid
      "minZoom": 0,
      "maxZoom": 8,
      "tiles": ["http://{VECTOR_TILES_HOST_ADRESS}/vt/xyz/overlay_mercator/{z}/{x}/{y}.pbf"],
      "type": "vector"
    }
  },
  "glyphs": "http://{FONTS_HOST_ADRESS}/fonts/{fontstack}/{range}.pbf",
  // one layer for this example should be enough
  "layers": [
    {
	  "id": "osm_border_linestrings_countries",
	  "type": "line",
	  // where to look in the sources list above
	  "source": "osm_borders",
	  // name of the layer in the MVT files
	  "source-layer": "osm_border_linestrings_countries",
	  // filter what you need based on attributes in the MVT
	  "filter": ["all"],
	  "layout": {
	      "line-cap": "round",
	      "line-join": "round",
	      "visibility": "visible"
	    },
	  "paint": {
	    "line-color": "rgba(255, 130, 40, 1)",
	    "line-width": {"stops": [[0, 0.5], [8, 1], [11, 2]]}
	  }
    }
  ],
  "id": "12345"
}

Preview and Map Layer Cache

Having everything set up and running, it is nice to see your map layers in action. For the preview we have taken our own Openlayers (opens new window) extension - EOxElements (opens new window) package and provided it with access to the vector tiles, WMS and WMTS endpoints—all of these served by the custom Tiles-API.

Dual Projection (left - EPSG:3857, right - EPSG:4326) EOxElements Overlay 2022 Demo

MapServer MapCache

Serving already cached map layers increases the performance significantly. Therefore we are using MapCache (opens new window), which provides a tile based caching solution and also offers various serving endpoints (opens new window).

The caching itself happens in a command called "mapcache_seed" (opens new window), where a lot of options are offered to tailor the (Map)cache into the final endpoints and services, which will utilize the cache once it has been generated.

Overview

As this is the first article about our maps technical details, we hope that it was an appropriate introduction into the EOxMaps topic on how we utilize available tools to provide up-to-date solutions for one of our many services.

It might seem that a lot of tools in this scheme are somewhat doubled in the infrastructure, which is partly true. For the purpose of a flawless transition from our old EOxMaps solution we have adapted available geospatial tools and combined them into one big pipeline which is capable of serving map tiles on the fly.

Here is a graph visualising the content of this article.

EOxMaps in 2022

What comes next?

Like in my last article (opens new window), there is more stuff happening at EOX right now. After the EOxMaps labels, we have started working on even more advanced automatization of our EOxCloudless (opens new window) data processing.

As for the maps, in a bit more distant future, apart from labels we aim to update our other EOxMaps (opens new window) layers (i.e. the "Terrain" and "Terrain-Light" layers) so that they get a fresh coat of paint as well.

---

Petr Ševčík, EOX IT Services GmbH Email: petr.sevcik@eox.at

Data ©OpenStreetMap (opens new window) contributors and others.

Contains Modified Copernicus Sentinel-2 Data 2020.

Our thanks to maptiler (opens new window) for TileServer GL (opens new window)