V1next history log

[isc-shuliu]

Implement #366 TODO List

• [x] Figure out what "committed" is
• [x] Work out the "history" command
• [x] Add public API for querying history
• [x] Record recursively loaded/installed modules
• [x] Properly implement "commit"
• [x] Allow more matching patterns when filtering histories
• [x] Display log entries in a prettier manner
• [x] Command to see the details of a history log by ID
• [x] Command to show supported columns for filtering
• [x] Allow specifying direction for sorting and limit max number of rows

[isc-tleavitt]

Re: "committed" - this would track, in non-dev mode, if installation was attempted but failed in some way - that is, it would be set to 0 in this case and 1 if installation succeeded. The reason for the failure (probably a %Status) would be worth recording as well for a failed installation.

[isc-kiyer]

Re: "committed" - this would track, in non-dev mode, if installation was attempted but failed in some way - that is, it would be set to 0 in this case and 1 if installation succeeded. The reason for the failure (probably a %Status) would be worth recording as well for a failed installation.

This comes into the overall lifecycle rework where we flag the phase in which a module reached during install and track the error that occurred as well. For modules in dev mode, can permit resuming install from the failed phase onwards and for non-dev mode, always start from scratch for the module.

[isc-shuliu]

Not sure why 2c240de doesn't work.

I have a local registry with modA and modB, where modB depends on modA. When installing modB, IPM will call %IPM.Utils.Module:LoadDependencies(), which uses %SYSTEM.WorkMgr to spawn a new process that runs ..LoadModuleReference().

The change in commit 2c240de is simple. We create a %Persistent history log object, save it in database, and pass the ID to the work unit. In the work unit, we save the success/failure and TimeEnd.

However, when calling %Save() from the spawned process, it raises the error Failed to acquire exclusive lock on instance of '%IPM.General.History. Why is there any lock involved?

[isc-shuliu]

It appears the lock is created because we're in a transaction. (Tried stepping through the code while monitoring management portal, lock disappears after manually running TCOMMIT)

[isc-shuliu]

Here's how it currently looks like

[isc-shuliu]

@isc-tleavitt Based on your feedback, I changed the time diff display to always show seconds and exposed API queries instead of API methods.

Currently, the table data is shared among all namespaces where %IPM is mapped. Do we want to separate them?

[isc-tleavitt]

I think this is good now, just need @isc-kiyer's review

[isc-tleavitt]

This should account for #704 (the pitfalls of being stuck here for a while, while we've been focused on more pressing things...)

[isc-dchui]

The big changes since Shuheng last worked on this:

1. History is now stored per namespace

2. The update command is also logged

3. Added logging of individual lifecycle phases as part of every given load/install/uninstall/update action that is visible in the details view. This is what it looks like for a failed load: (This requires temporarily storing in IRISTEMP to avoid losing data to transaction rollback. See %IPM.General.AbstractHistory %IPM.General.HistoryTemp)

4. Lots of tests

[isc-jili]

Sorry if this is already addressed, but I am wondering whether there is a housekeeping default of keeping only history within x number of days? In case over time the logs grow to be too much storage burden

[isc-kiyer]

Sorry if this is already addressed, but I am wondering whether there is a housekeeping default of keeping only history within x number of days? In case over time the logs grow to be too much storage burden

Yes this is set to a default of 20 records I believe

[isc-dchui]

@isc-jili, @isc-kiyer no actually the default records of 20 is just how many records will be shown at a given time when running history. There's no current mechanism to purge old records automatically. I can see how that might be helpful but I also wonder if that would defeat the point of logging all actions for an audit trail.

[isc-kiyer]

@isc-jili, @isc-kiyer no actually the default records of 20 is just how many records will be shown at a given time when running history. There's no current mechanism to purge old records automatically. I can see how that might be helpful but I also wonder if that would defeat the point of logging all actions for an audit trail.

So generally with auditing, there is often a purge "by time" mechanism since only audit history for a certain time period matters. Lets spawn off purging into its own issue to not add scope creep to this one.