New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
« ICU collation keyword field Japanese (kuromoji) analysis plugin »
Elastic Docs ›Elasticsearch Plugins and Integrations [8.17] ›Analysis plugins ›ICU analysis plugin

ICU transform token filter

edit

ICU transform token filter

edit

Transforms are used to process Unicode text in many different ways, such as case mapping, normalization, transliteration and bidirectional text handling.

You can define which transformation you want to apply with the id parameter (defaults to Null), and specify text direction with the dir parameter which accepts forward (default) for LTR and reverse for RTL. Custom rulesets are not yet supported.

For example:

PUT icu_sample
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "latin": {
            "tokenizer": "keyword",
            "filter": [
              "myLatinTransform"
            ]
          }
        },
        "filter": {
          "myLatinTransform": {
            "type": "icu_transform",
            "id": "Any-Latin; NFD; [:Nonspacing Mark:] Remove; NFC" 
          }
        }
      }
    }
  }
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "你好" 
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "здравствуйте" 
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "こんにちは" 
}
Copy as curlTry in Elastic 

This transforms transliterates characters to Latin, and separates accents from their base characters, removes the accents, and then puts the remaining text into an unaccented form.

Returns ni hao.

Returns zdravstvujte.

Returns kon'nichiha.

For more documentation, Please see the user guide of ICU Transform.

« ICU collation keyword field Japanese (kuromoji) analysis plugin »
Was this helpful?
Feedback