pyhydroqc
pyhydroqc copied to clipboard
AmberSJones
This software was designed with the purpose of anomaly detection and correction for time series water sensor data. This software was developed using the Logan River Observatory data set.
The sklearn package has been depreciated. It might be helpful to replace 'sklearn' with 'sci-kit-learn' in your pip requirements file setup.py setup.cfg, files https://github.com/scikit-learn/sklearn-pypi-package Thank you
The performance and speed of ARIMA correction might be improved by limiting the window of piecewise model development to 2-6 times the length of the gap.
write a function to use an edge detection like filter to determine likely candidates for calibration events
add to the calibration detection function to ensure that output event dates are at least 3 in length.
Check that dataframes are being copied so that changes aren't being made on original copies
currently, the LSTM correction proceeds in only one direction. need to assess if this results in significant nonlinearities, and if so, consider how to implement a cross-fade.
Consider improving the ARIMA detection by 1. apply detection as is 2. linear interpolate 3. build model again 4. detect again
there are cases where warnings in model development, etc. need to be suppressed.
consider training LSTM to classify and detect linear drift/calibration
Need to consider implementation of feature based anomaly detection.
Metadata
Owner
Metadata
This software was designed with the purpose of anomaly detection and correction for time series water sensor data. This software was developed using the Logan River Observatory data set.