sanecleanup
sanecleanup copied to clipboard
Sensible defaults for data retention and cleanup for SAP Commerce
sanecleanup
Sensible defaults for data retention and cleanup for SAP Commerce, based on my CX Works article Data Maintenance and Cleanup
How-To
-
Download the latest release
-
Unpack to
hybris/bin/custom
-
If possible, disable saved values / change history (ref. help.sap.com, further recommendations in my article)
-
Add extension to your
localextensions.xml
<extension name="sanecleanup" />
-
:red_circle: Adapt the retention rules to your project requirements :red_circle:
(check the available properties inproject.properties
) -
Build and deploy.
(The rules will be automatically imported during system update)If you get a build error regarding missing types like the example below:
- Open
sanecleanup-items.xml
- Search for the type
- Comment-out the whole
<itemtype>
tag
invalid index sanecleanup for type _TYPE_ on [...] declared at ((sanecleanup))::YIndex[sanecleanup-items.xml:...] due to missing enclosing type '_TYPE_'
- Open
WARNING
The very first execution of the retention cron jobs will take a while, depending on how long your poject
is already live and if you have cleaned up anything in the past.
Consider performing a one-time cleanup before adding the extension / enabling the retention rules.
Especially the first run of cronJobLogCleanupCronJob
will take a very long time, if you have never removed any cronjob log files (type LogFile
).
Please consider importing and executing the script job defined in bulkdelete-cronjoblogs.impex before you set up the automated cleanup!
The job will remove all log files except the five most recent logs per CronJob.
(Disclaimer: the script was tested on MS SQL / Azure SQL and SAP HANA. It is not guaranteed to work for other databases)
Do I have to clean up?
If have never even thought about that topic - yes!
You can run the following scripts in the administration console to get a quick overview:
-
excessive-platform-types.groovy
- Generates a report about "known troublemakers" -
check-audit.groovy
- Check if you have too many audit logs
Here are some additional queries and "rules of thumb" that help you investigate further:
Type(s) | Query | Notes |
---|---|---|
AbstractRule |
|
Are there any outdated rules? i.e rules that aren't valid anymore because their enddate is in the past. Warning: change |
BusinessProcess |
|
Are there too many (let's say > 1000) or very old BusinessProcess in your system? Also, if a lot of processes are stuck in "RUNNING" / "WAITING", you have to investigate what's wrong. (What is causing your processes to be stuck?) |
Cart |
|
|
CronJob (auto-generated) |
|
Are there too many (>10) outdated, auto-geneated jobs in your system? |
CronJobHistory |
|
Is there any job with > 50 histories and/or histories older than an hour? This cleanup is enabled by default in recent SAP Commerce patch releases, so this query shouldn't find anything. |
EmailMessage |
|
|
ImpExImportCronJob (distributed impex) |
|
|
ImpexMedia |
|
Are there more than a handful (>100) of generated impex medias? |
ImportBatchContent |
|
Are there any left-over distributed import batches? |
LogFile |
|
Are there are cronjob with more than ~10 logs and/or logs older than 14 days? (those are default values for log file retention) |
ProcessTaskLog |
|
We recommend customer to BusinessProcess cleanup, which will eventually take care of TaskLogs cleanup. There might be the few scenarios for ProcessTaskLog cleanup:
|
SavedValues,SavedValueEntry |
|
A lot of those items accumulated over the project lifetime.
If possible, disable storing saved values. ( |
SolrIndexOperation |
|
Too many solr operations (more than ~100 per index)? |
StoredHttpSession |
|
Excessive amount of session? This is hard to generalize as it highly depends on your site's traffic, but if you are near or over 5 digits, it's probably too much. Simarly, stale sessions (e.g older than a day) don't need to be retained. |
TaskCondition |
|
Is there an excessive amount of "premature events"? Or very old (older than a a few weeks) events? |
Support
Please open an issue describing your problem or your feature request.
Contributing
Any and all pull requests are welcome.
Please describe your change and the motiviation behind it.