How to

Goodbye, types. Hello, typeless.

By
Adrien Grand

Since 5.0, we've been paving the way for the removal of types from Elasticsearch, and with 7.0, deprecation has arrived. The goal of types was to provide multi-tenancy within a single index, which is incompatible with Lucene. Unfortunately, types proved to cause more problems than they solved. After long discussions regarding whether we should work around the issue — e.g. using unique fields per type or having one index per type under the hood — we decided instead to remove support for types. To make the transition easier to our users, we spread changes over four major versions:

  • 5.0 started enforcing that fields that share the same name across multiple types have compatible mappings.
  • 6.0 started preventing new indices from having more than one type and deprecated the _default_ mapping.
  • 7.0 deprecated APIs that accept types, introduced new typeless APIs, and removed support for the _default_ mapping.
  • 8.0 will remove APIs that accept types.

Typeless APIs

All APIs that accept types in their URL path, request body or response body are getting a new "typeless" counterpart. This is the case of some very common APIs such as the index creation, index and GET APIs. An example is worth a thousand words, so here is what an interaction with a 7.0 cluster looks like with typeless APIs: 

PUT index
{
  "mappings": {
    "properties": { // Note how there is no top-level key with the type name
      "@timestamp": {
        "type": "date"
      }
    }
  }
}
PUT index/_doc/1 // _doc becomes the new endpoint for document index, get and delete requests
{
  "@timestamp": "2019-02-20"
}
POST index/_bulk
{"index": {"_id": "2"}} // no _type in the URL or in document metadata
{"@timestamp": "2019-03-01"}
{"update": {"_id": "1"}}
{"@timestamp": "2019-02-21"}
GET index/_doc/2
GET index/_search // unchanged, the `GET index/{type}/_search` endpoint is deprecated however
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "2019-01-01"
      }
    }
  }
}
GET index/_explain/1 // note how the type does not appear in the URL anymore, this would have been "GET index/{type}/1/_explain" in 6.x
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "2019-01-01"
      }
    }
  }
}

What do I need to do?

Some changes are breaking and will require you to perform changes to your cluster before upgrading to 7.0. Some other changes will only be required for an upgrade to 8.0 and should be done after the upgrade to 7.0.

Before upgrading

Make sure you are on 6.8

If you plan to upgrade without downtime in a rolling fashion, you should upgrade to 6.8 first, which is the only 6.x release to have some features like support for the include_type_name parameter, which is required to upgrade to 7.0 smoothly.

Stop using _default_ mappings

The _default_ mapping is going away with 7.0. If some existing 6.x indices have a _default_ mapping, they will be fine as the _default_ mapping will only be rejected on new indices.

You should update your index templates and your application code to add these mappings to the actual type, rather than include them in a _default_ mapping. If the name of the type is not important for you, the recommendation is to use _doc, which will make the transition to typeless APIs easier in the future.

Pass include_type_name=true to index creation, template and mappings APIs

The move to typeless APIs means that APIs whose request bodies required a type are now going to expect a different format:

  • create index
  • put mapping
  • put template

while other APIs that returned types in the response are going to have a different response format:

  • get index
  • get mapping
  • get field mapping
  • get template

In order to not be broken by this change, you should make your application pass include_type_name=true to these API calls, which is a no-op on 6.x, and will tell 7.x to make these APIs behave as they used to do in 6.x. For instance here is an index creation call with this parameter. It works the same way on 6.8 and 7.x. 

PUT index?include_type_name=true
{
  "mappings": {
    "my_type": {
      "properties": { // Note how there is no top-level key with the type name
        "@timestamp": {
          "type": "date"
        }
      }
    }
  }
}

After upgrading

If you followed the recommended pre-upgrade steps, everything should keep working. However, at this stage you are still using APIs that accept types, and this triggers deprecation warnings. In order to get rid of them, you should move to typeless APIs.

For index creation, template and mappings APIs, you will need to remove include_type_name=true from the URL parameters, and change request bodies or expectations about the response format, which no longer include types. Typically, if the request body or response included mappings, it had a section that looked like this: 

{
  "my_type": { // this key is no longer accepted in mappings as of 7.0
    "properties": ...
  }
}

It needs to be updated to no longer include a type name, and would now look like that: 

{
  "properties": ...
}

For other APIs like the index, GET, or update API, you should replace the type name in the URL with _doc. This will work regardless of the name of the type of your index. For instance if you used to perform index calls for an index whose type name is my_type, the index call looked like this: 

PUT index/my_type/1
{
  "@timestamp": "2019-02-20"
}

and should be replaced with that: 

PUT index/_doc/1
{
  "@timestamp": "2019-02-20"
}

Learn more

If you would like to know more about the removal of types and how to migrate to typeless APIs, I'd recommend that you have a look at our removal of types documentation, where we dive into greater details regarding the implications of these changes and how to prepare for them.

Download 7.0.0, give these new typeless APIs a go, and let us know what you think on Discuss.