chaostoolkit-lib icon indicating copy to clipboard operation
chaostoolkit-lib copied to clipboard

Support multi-values (expect_one_of) in jsonpath tolerance expect

Open saravanan30erd opened this issue 3 years ago • 7 comments

Added support for multi-values (expect_one_of) in jsonpath tolerance expect field.

Example:

"tolerance": {
                    "type": "jsonpath",
                    "path": "$.status",
                    "expect": ["yellow"],
                    "target": "body"
                },

In this case, path $.status provides value either ["green"] or ["yellow"] or ["red"]. Both yellow and green considered as healthy status. Example: In Elastic-search cluster, If you remove or shutdown one or two nodes in cluster, the status will be green before action and yellow after action. But both should be considered as healthy state in ES.

Currently it doesn't support or / expect_one_of feature in expect value.

so added support for expect_one_of feature in expect value.

  "tolerance": {
                    "type": "jsonpath",
                    "path": "$.status",
                    "expect": [["yellow", "green"]],
                    "target": "body"
                },

If we want to validate the output against multiple expected values, then we can use list of values instead of string.

Output value: ["green", "completed"] OR sometimes  ["yellow", "completed"]

**Default**:
Expect:  ["green", "completed"]
**expect_one_of**
Expect:  [["green", "yellow"], "completed"]

saravanan30erd avatar Oct 17 '20 12:10 saravanan30erd

@Lawouach I did a mistake, thought to use list inside list to mention both expect_one_of values bcoz I used one sample json payload which is straight forward. But later realized this won't work for other payloads so tried some other method but it didn't suit for some complex payloads.

At last, I used another expect field called expect_alt to mention the alternative expected values.

Output value: ["green", "completed"] OR sometimes  ["yellow", "completed"]

Default:
Expect:  ["green", "completed"]

expect_one_of:
Expect:  ["green", "completed"],
Expect_alt:  ["yellow", "completed"],

saravanan30erd avatar Oct 17 '20 17:10 saravanan30erd

Hello, for a better review, could you please squahs your commits and force push?

Lawouach avatar Oct 19 '20 09:10 Lawouach

@Lawouach

expect_alt- its alternative expect field for expect. since we are adding one more expect field for using alternative values, I named it as expect_alt Please provide your suggestion for naming convention. I will use that name.

Sure, I will raise the PR for documentation once the naming convention is finalized.

saravanan30erd avatar Oct 20 '20 18:10 saravanan30erd

I thought expect_one_of was more direct and descriptive. Unless expecte_alt is behaving differentlty, I'd go for expect_one_of.

Lawouach avatar Oct 21 '20 09:10 Lawouach

@Lawouach Renamed the new field as expect_one_of and also added change log.

saravanan30erd avatar Oct 21 '20 20:10 saravanan30erd

Codecov Report

Merging #191 into master will decrease coverage by 0.04%. The diff coverage is 77.77%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #191      +/-   ##
==========================================
- Coverage   88.02%   87.97%   -0.05%     
==========================================
  Files          25       25              
  Lines        2113     2121       +8     
==========================================
+ Hits         1860     1866       +6     
- Misses        253      255       +2     
Impacted Files Coverage Δ
chaoslib/hypothesis.py 89.37% <77.77%> (-0.58%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f781dfe...e196962. Read the comment docs.

codecov-io avatar Oct 21 '20 21:10 codecov-io

@Lawouach Did changes based on your comments. Also added the information in documentation and raised PR. https://github.com/chaostoolkit/chaostoolkit-documentation/pull/103

saravanan30erd avatar Mar 09 '21 15:03 saravanan30erd