New Country Subdivision Vectors in the Elastic Maps Service

There is a science joke that goes, "Geology rocks, but Geography is where it's at." At Elastic, we understand that location matters. That's why we created the Elastic Maps Service. The Elastic Maps Service (EMS) provides the basemap tiles, region map boundaries, and key attribute data for geospatial visualizations in Kibana. Let's take a look at some of the latest updates with EMS.

New Country Subdivision Layers

If you have ever created a Region Map in Kibana, you are likely familiar with our World Countries vector map layer. If not, here is a previous blog post that shows how it works.

We have been adding additional vector map layers showing subdivisions of countries such as Australia states, Ireland counties, and Switzerland cantons. Each of these new layers has a field called iso_3166_2 which contains the subdivision code defined by part 2 of the ISO 3166 Country Codes Collection. For example, ISO 3166 identifies eight subdivisions in Australia each with its own unique code.

A common use case for region maps is showing website visits by country. This is accomplished by ingesting web server logs, and using either the Logstash GeoIP filter plugin or the Ingest GeoIP Processor plugin to add location data from the MaxMind GeoIP databases to your documents in Elasticsearch. For best results I suggest downloading the latest version of the MaxMind GeoIP 2 City database rather than the older one included with Logstash. You can specify the file location of the downloaded database in your Logstash config.

If you have been using the Logstash GeoIP filter plugin to ingest your web server logs, you may already have the necessary data to visualize by country subdivisions. Use the Discover tab in Kibana to look for the geoip.country_code2 and geoip.region_code fields. We can use these fields to create a new Painless scripted field called geoip.region_iso_code that we can join to the "ISO-3166-2 Identifier" field in a region map.

if (doc['geoip.region_code.keyword'].value != null && doc['geoip.country_code2.keyword'].value != null) {
return  doc['geoip.country_code2.keyword'].value + '-' + doc['geoip.region_code.keyword'].value;
}
return "";

The Using Painless in Kibana scripted fields blog post has detailed instructions and more examples for creating scripted fields. Version 6.4 of Kibana now adds the ability to preview your scripted field. For more information on this feature, check out the release blog post.

Using the geoip.region_iso_code field let's create a new region map showing web visitors from regions in Australia.

1. Add a filter on a new region map to limit only logs from Australia (AU).
2. Leave the Metrics as default and set the Buckets shape field to a Terms aggregation on the geoip.region_iso_code field.
3. Switch to the Options tab and choose "Australia States" from the "Vector map" dropdown list and choose "ISO-3166-2 Identifier" as the "Join field".

You can create additional region maps for other countries the same way and add them to a dashboard alongside other metrics.

Elastic Maps Service Data

Perhaps you want to create region map visualizations from other data in Elasticsearch. You will want to know what vector map layers and attributes are available from Elastic Maps Service. The vector map data and attributes can be previewed and downloaded from https://maps.elastic.co. There you may find that some regions have additional attributes fields that you can use to visualize your data in Elasticsearch. For example, the France Departments layer has an INSEE field with codes used by the Institut national de la statistique et des études économiques (INSEE). We previously showed an example of this in our Visualizing France Salary Data blog post.

While region names are provided in the layers, we recommend joining your Elasticsearch data using the ISO codes or other coded attribute fields (ex. INSEE, Dantai, Logainm) included in the layers. Among other issues, regions may have multiple official names. The Falsehoods programmers believe about geography article has more details about the problems with geographic names.

Looking Ahead

We are continuing to add new vector map layers to Elastic Maps Service. Most of the layers we provide are not tied to software versions. This means you will not need to upgrade to the latest version of Elasticsearch and Kibana to use most of the new layers in your visualizations. Region map visualizations are available on Kibana version 5.5 and up.

Please create an issue on the Kibana GitHub repository if there's a new vector map layer or a coded attribute value on an existing layer that you think should be included in Kibana.