lux icon indicating copy to clipboard operation
lux copied to clipboard

FIX-#431: implement sampling threshold and edit tests and docs

Open westernguy2 opened this issue 2 years ago • 1 comments

Signed-off-by: Kunal Agarwal [email protected]

Overview

I removed the sampling_cap and sampling_start from the config and execute_sampling. Instead, I added a sampling_thresh, which is a threshold for which we'd begin sampling. This threshold forces any data larger than the threshold to sample the data such that the sample is equal to the size specified by the threshold. More information can be found in the documentation that was edited as part of this PR.

Changes

I edited config and execute_sampling to implement this fix. I also edited the tests to account for the removal of the old configs and the addition of the new one. I finally edited the documentation to reflect these changes.

Example Output

The issue described in #431 should be fixed now.

westernguy2 avatar Nov 10 '21 08:11 westernguy2

The code changes look great. It seems like test_maintainence.py is failing because the 10k row sample is probably not sufficient for these use cases. The sampling can be problematic for Similarity or Filter based actions. In this case, given that the intent involves filter, the Similarity action iterates over all the line charts on each filter.

dorisjlee avatar Nov 13 '21 21:11 dorisjlee