docs
docs copied to clipboard
mattermost
Help Wanted: Clarify Audit Logging Configuration Placement
In the Audit Log v2 docs under Configuring Audit Log in Mattermost it says this:
Open config.json and navigate to the audit settings under ExperimentalAuditSettings. Within the setting AdvancedLoggingConfig you are able to specifiy a filespec to another config file, a database DSN, or JSON.
However, it does not provide an example of what this looks like in the config.json, nor a reference to describe what a filespec means.
I'm with @icelander here. I've tried and all I can get is errors:
Mar 16 17:53:33 mattermost mattermost[117406]: Error: failed to load configuration: failed to create store: unable to load on store creation: parsing error at line 167, character 35: json: cannot unmarshal object into Go struct field ExperimentalAuditSettings.ExperimentalAuditSettings.AdvancedLoggingConfig of type string
And that comes from this:
"AdvancedLoggingConfig": {
"file_1": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 100, "Name": "audit-api"},
{"ID": 101, "Name": "audit-content"},
{"ID": 102, "Name": "audit-permissions"},
{"ID": 103, "Name": "audit-cli"}
],
"Options": {
"Compress": true,
"Filename": "audit.log",
"MaxAgeDays": 1,
"MaxBackups": 7,
"MaxSizeMB": 500
},
"MaxQueueSize": 1000
}
}
Hello,
I try to use this parameter also for the LoggingSettings and NotificationLogSettings, it doesn't work for me also.
I put a JSON file with relative path, absolute path or else a Nested/Embedded JSON in the configuration file and i have this error or 2 others errors. I've try to use brackets [ ] and double quotes " " or not with the Embedded JSON with no sucess, i minified the nested JSON on a single line and it doesn't work also.
I can see on the documentation 2 formats of JSON, i try also this 2 formats and it doesn't work too. -> https://docs.mattermost.com/_downloads/b7017a864f9dd114c67f6dc381621c4f/sample-logger-config.json -> https://docs.mattermost.com/_downloads/c4100cd437d430795506d20eee0a068f/advanced-logging-options-sample-json.zip
I don't used the syntax of the text file config here -> https://github.com/mattermost/docs/files/5066579/Log.Settings.Options.txt
My configs files :
config-log.json :
{
"console-log": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
},
"file-log": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Compress": true,
"Filename": "mattermost_log.log.json",
"MaxAgeDays": 15,
"MaxBackups": 3,
"MaxSizeMB": 100
},
"MaxQueueSize": 1000
}
}
config-log-notification.json :
{
"console-notification": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
},
"file-notification": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Compress": true,
"Filename": "mattermost_notification_log.log.json",
"MaxAgeDays": 15,
"MaxBackups": 3,
"MaxSizeMB": 100
},
"MaxQueueSize": 1000
}
}
config-audit.json :
{
"console-audit": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
},
"file-audit": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Compress": true,
"Filename": "mattermost_audit_log.log.json",
"MaxAgeDays": 15,
"MaxBackups": 3,
"MaxSizeMB": 100
},
"MaxQueueSize": 1000
}
}
When i use the path of the json config :
{"level":"error","msg":"Could not initiate logging","fields":{"error":"\"invalid config source for logging, failed to scan data from row for /mattermost/config/config-log.json: sql: no rows in result set\""}}
[...]
{"level":"error","msg":"Error configuring audit","fields":{"error":"\"invalid config source for audit, failed to scan data from row for /mattermost/config/config-audit.json: sql: no rows in result set\""}}
When i use the content of the file with quotes or brackets :
"AdvancedLoggingConfig": [
{
"console-log": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
},
"file-log": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Compress": true,
"Filename": "mattermost_log.log.json",
"MaxAgeDays": 15,
"MaxBackups": 3,
"MaxSizeMB": 100
},
"MaxQueueSize": 1000
}
}
]
{"level":"warn","msg":"Error loading custom configuration defaults: unable to decode custom defaults configuration: invalid character '\\n' in string literal"}
"AdvancedLoggingConfig": "
{
"console-log": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
},
"file-log": {
"Type": "file",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true},
{"ID": 1, "Name": "fatal", "Stacktrace": true},
{"ID": 0, "Name": "panic", "Stacktrace": true}
],
"Options": {
"Compress": true,
"Filename": "mattermost_log.log.json",
"MaxAgeDays": 15,
"MaxBackups": 3,
"MaxSizeMB": 100
},
"MaxQueueSize": 1000
}
}"
{"level":"warn","msg":"Error loading custom configuration defaults: unable to decode custom defaults configuration: invalid character '\\n' in string literal"}
With the content of the file on a single line :
"AdvancedLoggingConfig": [{ "console-log": { "Type": "console", "Format": "json", "Levels": [ {"ID": 5, "Name": "debug", "Stacktrace": false}, {"ID": 4, "Name": "info", "Stacktrace": false, "color": 36}, {"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33}, {"ID": 2, "Name": "error", "Stacktrace": true, "color": 31}, {"ID": 1, "Name": "fatal", "Stacktrace": true}, {"ID": 0, "Name": "panic", "Stacktrace": true} ], "Options": { "Out": "stdout" }, "MaxQueueSize": 1000 }, "file-log": { "Type": "file", "Format": "json", "Levels": [ {"ID": 5, "Name": "debug", "Stacktrace": false}, {"ID": 4, "Name": "info", "Stacktrace": false}, {"ID": 3, "Name": "warn", "Stacktrace": false}, {"ID": 2, "Name": "error", "Stacktrace": true}, {"ID": 1, "Name": "fatal", "Stacktrace": true}, {"ID": 0, "Name": "panic", "Stacktrace": true} ], "Options": { "Compress": true, "Filename": "mattermost_log.log.json", "MaxAgeDays": 15, "MaxBackups": 3, "MaxSizeMB": 100 }, "MaxQueueSize": 1000 } }]
{"level":"warn","msg":"Error loading custom configuration defaults: unable to decode custom defaults configuration: json: cannot unmarshal array into Go struct field LogSettings.LogSettings.AdvancedLoggingConfig of type string"}
"AdvancedLoggingConfig": "{ "console-log": { "Type": "console", "Format": "json", "Levels": [ {"ID": 5, "Name": "debug", "Stacktrace": false}, {"ID": 4, "Name": "info", "Stacktrace": false, "color": 36}, {"ID": 3, "Name": "warn", "Stacktrace": false, "color": 33}, {"ID": 2, "Name": "error", "Stacktrace": true, "color": 31}, {"ID": 1, "Name": "fatal", "Stacktrace": true}, {"ID": 0, "Name": "panic", "Stacktrace": true} ], "Options": { "Out": "stdout" }, "MaxQueueSize": 1000 }, "file-log": { "Type": "file", "Format": "json", "Levels": [ {"ID": 5, "Name": "debug", "Stacktrace": false}, {"ID": 4, "Name": "info", "Stacktrace": false}, {"ID": 3, "Name": "warn", "Stacktrace": false}, {"ID": 2, "Name": "error", "Stacktrace": true}, {"ID": 1, "Name": "fatal", "Stacktrace": true}, {"ID": 0, "Name": "panic", "Stacktrace": true} ], "Options": { "Compress": true, "Filename": "mattermost_log.log.json", "MaxAgeDays": 15, "MaxBackups": 3, "MaxSizeMB": 100 }, "MaxQueueSize": 1000 } }"
{"level":"warn","msg":"Error loading custom configuration defaults: unable to decode custom defaults configuration: invalid character 'c' after object key:value pair"}
In my case, i use a centralized configuration in database because we using High Availability and SAML features (my certificates and key are not loaded on database) so i have a valid licence E20. I could see it a feature who is enable with a licence E20.
@wiggin77 - Would you be able to assist with this issue, please? Are there specific documentation updates or corrects needed to address the issues raised in this issue?
n my case, i use a centralized configuration in database
@Gregesp I know the docs don't (yet) make this clear, but when the main config is stored in the database then the auxiliary files also need to be in the database. So your AdvancedLoggingConfig files would have to be added to the database via mmctl. Someone did this recently and I'm trying to find the example command line for adding these files via mmctl.
@untergeek, when embedding the AdvancedLoggingConfig in config.json the config should be embedded as a string containing escaped JSON. I know this is not ideal, and I'll create a ticket to allow non-escaped JSON embedding, but for now that is what the code will accept. A tool like this https://www.freeformatter.com/json-escape.html can be used to escape the JSON to something that be included as a string.
n my case, i use a centralized configuration in database
@Gregesp I know the docs don't (yet) make this clear, but when the main config is stored in the database then the auxiliary files also need to be in the database. So your AdvancedLoggingConfig files would have to be added to the database via
mmctl. Someone did this recently and I'm trying to find the example command line for adding these files via mmctl.
Hello,
I tried to import JSON file on database with the CLI mattermost import, but it doesn't work. I tried also a formatted/escaped JSON file and it doesn't work.
I dont use mmctl import upload or others commands because i try to automate the deployment and as mattermost doesn't have an admin account builtin i can't use this CLI, to use it i think i do firstly create the account who will be affected with the System Admin Role.
@Gregesp I was able to get the audit logging to work with configuration in database like this:
First, put the audit logging settings into a JSON file, e.g. audit_logging.json
Next, run this command to use set a shell variable to the escaped JSON using jq
config=`jq -c '.' < ./audit_logging.json`
Then, run this command to set the configuration value:
mmctl config set ExperimentalAuditSettings.AdvancedLoggingConfig $config
If you are trying to automate deployment, you can use mmctl --local and enable Local Mode
Other improvements to add:
- List the available log levels and what events are included
- Indicate which log levels should go in
ExperimentalSettings.AdvancedLoggingConfigand which should be inLogSettings.AdvancedLoggingConfig - Add a note about having to restart when changes are made
- Also note that if you use a JSON object in the config file it must be escaped
- Provide more example configurations and an explanation of what they all log
Thanks, @TomerPacific! This is an older ticket that doesn't need further updates and can be closed. I appreciate you bringing these to my attention!