Migrate from a self-managed cluster with a self-signed certificate using remote reindex
editMigrate from a self-managed cluster with a self-signed certificate using remote reindex
editThe following instructions show you how to configure remote reindex on Elasticsearch Service from a cluster that uses a self-signed CA.
Let’s assume that the self-managed cluster that uses a self-signed certificate is called Source
, and you want to migrate data from Source
to Destination
on Elasticsearch Service.
Step 1: Create the Source
certificate in a bundle
edit-
Get the self-signed CA on the
Source
cluster, or extract the certificate from the cluster by running the following command:echo quit | openssl s_client -showcerts -servername "$SOURCE_SERVER_NAME" -connect "$SOURCE_SERVER:$PORT" > cacert.pem
-
Test
cecert.pem
you have just created withcurl
, this should return a successful response:curl -XGET https://$SOURCE_SERVER:$PORT -u <username>:<password> --cacert cacert.pem
-
Create the folder
my_source_ca
to store the filecacert.pem
, and compress the folder tomy_source_ca.zip
.
Both the folder and file names must correspond to the settings configured in Step 4.
Step 2: Upload the zip bundle to your Elasticsearch Service account
editTo upload your file, follow the steps in the section Add your extension.
Enter wildcard *
for Version in order to be compatible for all future upgrades, and select A bundle containing dictionary or script
as Type.
Step 3: Create a new deployment on Elasticsearch Service
editFrom the Elasticsearch Service Console create a new deployment. This will be the Destination
cluster.
The Destination
cluster should be the same or newer version as the Source
cluster. If you already have a cluster available, you can skip this step.
Step 4: Enable bundle and add reindex
settings on the Desination
cluster.
edit-
From your deployment page, go to the Edit page, click Manage user settings and extensions, select tab Extensions then enable
my_source_ca
. -
Switch tab to User settings, append the following settings to the
elasticsearch.yml
. This step addssource_server
to thereindex.remote.whitelist
, points source CA bundle to be trusted by theDestination
cluster using the settingreindex.ssl.certificate_authorities
.reindex.remote.whitelist: ["$SOURCE_SERVER:$PORT"] reindex.ssl.certificate_authorities: "/app/config/my_source_ca/cacert.pem" reindex.ssl.verification_mode: "full"
Make sure
reindex.remote.whitelist
is in an array format. All uploaded bundles will be uncompressed into/app/config/
folder. Ensure the file path corresponds to your uploaded bundle in Step 1. You can optionally setreindex.ssl.verification_mode
tofull
,certificate
ornone
depending on the validity of hostname and the certificate path. More details can be found in reindex setting. - Click Back to the Edit page and scroll to the button of the page to Save changes. This step will restart all Elasticsearch instances.
Step 5: Reindex from remote Source
cluster.
editYou can now run reindex
on the Elasticsearch Service Destination
cluster from Source
cluster:
POST _reindex { "source": { "remote": { "host": "https://$SOURCE_SERVER:$PORT", "username": "username", "password": "xxx" }, "index": "my_source_index" }, "dest": { "index": "my_dest_index" } }
If you have many sources to reindex, it’s is generally better to reindex them one at a time and run them in parallel rather than using a glob pattern to pick up multiple sources. Check reindex from multiple sources for more details.