IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Evaluate data frame analytics API Put Filter API »
Elastic Docs ›Java REST Client [7.4] ›Java High Level REST Client ›Machine Learning APIs

Estimate memory usage API

edit
IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Estimate memory usage API

edit

Estimates memory usage of data frame analytics. Estimation results can be used when deciding the appropriate value for model_memory_limit setting later on.

The API accepts an PutDataFrameAnalyticsRequest object and returns an EstimateMemoryUsageResponse.

Estimate memory usage request

edit
DataFrameAnalyticsConfig config = DataFrameAnalyticsConfig.builder()
    .setSource(DataFrameAnalyticsSource.builder().setIndex("estimate-test-source-index").build())
    .setAnalysis(OutlierDetection.createDefault())
    .build();
PutDataFrameAnalyticsRequest request = new PutDataFrameAnalyticsRequest(config);

Constructing a new request containing a data frame analytics config for which memory usage estimation should be performed

Synchronous execution

edit

When executing a PutDataFrameAnalyticsRequest in the following manner, the client waits for the EstimateMemoryUsageResponse to be returned before continuing with code execution:

EstimateMemoryUsageResponse response = client.machineLearning().estimateMemoryUsage(request, RequestOptions.DEFAULT);

Synchronous calls may throw an IOException in case of either failing to parse the REST response in the high-level REST client, the request times out or similar cases where there is no response coming back from the server.

In cases where the server returns a 4xx or 5xx error code, the high-level client tries to parse the response body error details instead and then throws a generic ElasticsearchException and adds the original ResponseException as a suppressed exception to it.

Asynchronous execution

edit

Executing a PutDataFrameAnalyticsRequest can also be done in an asynchronous fashion so that the client can return directly. Users need to specify how the response or potential failures will be handled by passing the request and a listener to the asynchronous estimate-memory-usage method:

client.machineLearning().estimateMemoryUsageAsync(request, RequestOptions.DEFAULT, listener);

The PutDataFrameAnalyticsRequest to execute and the ActionListener to use when the execution completes

The asynchronous method does not block and returns immediately. Once it is completed the ActionListener is called back using the onResponse method if the execution successfully completed or using the onFailure method if it failed. Failure scenarios and expected exceptions are the same as in the synchronous execution case.

A typical listener for estimate-memory-usage looks like:

ActionListener<EstimateMemoryUsageResponse> listener = new ActionListener<EstimateMemoryUsageResponse>() {
    @Override
    public void onResponse(EstimateMemoryUsageResponse response) {
        
    }

    @Override
    public void onFailure(Exception e) {
        
    }
};

Called when the execution is successfully completed.

Called when the whole PutDataFrameAnalyticsRequest fails.

Response

edit

The returned EstimateMemoryUsageResponse contains the memory usage estimates.

ByteSizeValue expectedMemoryWithoutDisk = response.getExpectedMemoryWithoutDisk(); 
ByteSizeValue expectedMemoryWithDisk = response.getExpectedMemoryWithDisk();

Estimated memory usage under the assumption that the whole data frame analytics should happen in memory (i.e. without overflowing to disk).

Estimated memory usage under the assumption that overflowing to disk is allowed during data frame analytics.
« Evaluate data frame analytics API Put Filter API »