superset
superset copied to clipboard
feat(python): add purge Event logger database
SUMMARY
An example to add purge for Event Logger in Database, because today Logs tables is not purged and grow to infinity.
Is not perfect, ! :warning: warning this PR can slow down Superset Interface if you have a lot of element in logs table.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in SIP-59)
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
Hi everyone ! Do you think this PR could be merged as is ?
After test we need an index on dttm column for table logs ;)
Ping @eschutho @geido please (i don't know if you are the good persons to ping, but we found nobody in the CODEOWNERS :'-(). Do you think this kind of purge could be a good idea ?
If not, we will manage it in our own superset config code.
:pray:
Adding some people who might have a better understanding of this part of the application
Ping @jplanckeel, do you think these changes could be done on our way ?
i will test to copy prune report.log ;) https://github.com/apache/superset/blob/master/superset/tasks/scheduler.py#L99
i create a new PR with the new solution to prune logs table with celery task https://github.com/apache/superset/pull/21873/files
thank for your reponse ;)
@jplanckeel closing this in favour of the new PR