Machine learning decider
editMachine learning decider
editThe machine learning decider (ml) calculates the memory and CPU requirements to run machine learning
jobs and trained models.
The machine learning decider is enabled for policies governing ml nodes.
For machine learning jobs to open when the cluster is not appropriately scaled, set
xpack.ml.max_lazy_ml_nodes to the largest number of possible machine learning nodes (refer
to Advanced machine learning settings for more information). In Elasticsearch Service, this is
automatically set.
Configuration settings
editBoth num_anomaly_jobs_in_queue and num_analytics_jobs_in_queue are designed
to delay a scale-up event. If the cluster is too small, these settings indicate
how many jobs of each type can be unassigned from a node. Both settings are only
considered for jobs that can be opened given the current scale. If a job is too
large for any node size or if a job can’t be assigned without user intervention
(for example, a user calling _stop against a real-time anomaly detection job), the
numbers are ignored for that particular job.
-
num_anomaly_jobs_in_queue -
(Optional, integer)
Specifies the number of queued anomaly detection jobs to allow. Defaults to
0. -
num_analytics_jobs_in_queue -
(Optional, integer)
Specifies the number of queued data frame analytics jobs to allow. Defaults to
0. -
down_scale_delay - (Optional, time value) Specifies the time to delay before scaling down. Defaults to 1 hour. If a scale down is possible for the entire time window, then a scale down is requested. If the cluster requires a scale up during the window, the window is reset.
Examples
editThis example creates an autoscaling policy named my_autoscaling_policy that
overrides the default configuration of the machine learning decider.
PUT /_autoscaling/policy/my_autoscaling_policy
{
"roles" : [ "ml" ],
"deciders": {
"ml": {
"num_anomaly_jobs_in_queue": 5,
"num_analytics_jobs_in_queue": 3,
"down_scale_delay": "30m"
}
}
}
The API returns the following result:
{
"acknowledged": true
}