Description of changes: Added a new API endpoint to AD in order to validate anomaly detector configurations and return back if any failures or suggestions occurred. The validation API takes an input of detector configurations and then validates it against both general checks and against the data source to know if the configurations will likely lead to successful anomaly detector creation. Endpoint: Post _opendistro/_anomaly_detection/detectors/_validate Input : Anomaly Detector configurations image

Overall the validation checks if any fields are missing, if any data can be found with the given filter query and feature query. I also check whether the feature aggregation type is valid for the field type. The last two steps of the validation, checks if the detector interval used will produce data that is dense enough, recommending a different interval if this isn't the case in milliseconds. The API also checks for the last seen data point and returns it as the window delay recommendation.

Example input for detector interval that is too short :

{
    "name":"test-detector-02",
    "description":"Test detector",
    "time_field":"timestamp",
    "indices":[
        "test-index-sparse-3-demo"
    ],
    "feature_attributes":[
        {
            "feature_name":"total_order",
            "feature_enabled":true,
            "aggregation_query":{
                "total_order":{
                    "max":{
                        "field":"field-1"
                    }
                }
            }
        }
    ],
    "detection_interval":{
        "period":{
            "interval":10,
            "unit":"Minutes"
        }
    },
    "window_delay":{
        "period":{
            "interval":150,
            "unit":"Minutes"
        }
    }
}

Output:

{
    "failures":{

    },
    "suggestedChanges":{
        "detection_interval":[
            "4320000" *90 minutes recommendation
        ]
    }
}

Testing Done: Integration testing added manually tested that API works with both dense and sparse sample data and some of the tms-issue data.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sep 02 '20 01:09 amitgalitz

Codecov Report

Merging #221 into master will decrease coverage by 4.07%. The diff coverage is 22.59%.

@@             Coverage Diff              @@
##             master     #221      +/-   ##
============================================
- Coverage     72.41%   68.34%   -4.08%     
- Complexity     1290     1324      +34     
============================================
  Files           139      145       +6     
  Lines          6073     6593     +520     
  Branches        469      518      +49     
============================================
+ Hits           4398     4506     +108     
- Misses         1464     1874     +410     
- Partials        211      213       +2

Flag	Coverage Δ	Complexity Δ
#cli	`80.30% <ø> (ø)`	`0.00 <ø> (ø)`
#plugin	`66.88% <22.59%> (-4.48%)`	`1324.00 <42.00> (+34.00)`	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ	Complexity Δ
...distroforelasticsearch/ad/model/DateTimeRange.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
.../handler/ValidateAnomalyDetectorActionHandler.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (?)`
...rch/ad/transport/AnomalyResultTransportAction.java	`78.46% <ø> (ø)`	`59.00 <0.00> (ø)`
...stroforelasticsearch/ad/util/RestHandlerUtils.java	`100.00% <ø> (ø)`	`13.00 <0.00> (ø)`
...rch/ad/rest/RestValidateAnomalyDetectorAction.java	`63.63% <63.63%> (ø)`	`3.00 <3.00> (?)`
...stroforelasticsearch/ad/model/AnomalyDetector.java	`88.17% <88.73%> (-0.72%)`	`63.00 <25.00> (+17.00)`	:arrow_down:
...troforelasticsearch/ad/model/ValidateResponse.java	`95.65% <95.65%> (ø)`	`8.00 <8.00> (?)`
...stroforelasticsearch/ad/AnomalyDetectorPlugin.java	`93.61% <100.00%> (+0.06%)`	`10.00 <0.00> (ø)`
...oforelasticsearch/ad/model/ValidationFailures.java	`100.00% <100.00%> (ø)`	`3.00 <3.00> (?)`
...ticsearch/ad/model/ValidationSuggestedChanges.java	`100.00% <100.00%> (ø)`	`3.00 <3.00> (?)`
... and 7 more

Sep 02 '20 01:09 codecov[bot]

Unit test coverage drops and cause workflow to fail. Is it possible to write unit tests to cover it? Or can you let the workflow count your IT tests to the new coverage?

Sep 03 '20 19:09 kaituo

Unit test coverage drops and cause workflow to fail. Is it possible to write unit tests to cover it? Or can you let the workflow count your IT tests to the new coverage?

src/main/java/com/amazon/opendistroforelasticsearch/ad/rest/handler/ValidateAnomalyDetectorActionHandler.java may require more unit tests.

Sep 04 '20 17:09 kaituo

Validation API Backend for AD everywhere

Codecov Report