Ability to override log levels for contributions

[vijaynalawade]

What kind of change does this PR introduce? (check one with "x")

[] Bugfix
[x] Feature
[] Code style update (formatting, local variables)
[] Refactoring (no functional changes, no api changes)
[] Other... Please describe:

Fixes: #

What is the current behavior? Currently, app developers can not set different log level for contributions. Today, engine log level controls log levels for all contributions. For debug log level, app developers end up seeing unnecessary logs from the engine and other contributions. What is the new behavior? With this change, developers can set different log level for contributions(activity/trigger) through below environment variable. FLOGO_CONTRIB_LOG_LEVEL_OVERRIDE="<CONTRIB_LOGGER_MATCHER>=<LOG_LEVEL>,<CONTRIB_LOGGER_MATCHER>=<LOG_LEVEL>" e.g. To enable debug logs for Rest activity, set FLOGO_CONTRIB_LOG_LEVEL_OVERRIDE="activity-rest=DEBUG" To enable debug logs for Rest activity and trigger, set FLOGO_CONTRIB_LOG_LEVEL_OVERRIDE="rest=DEBUG"

[lixingwang]

Nice to have feature. sometimes it is useful to debug a big flogo.json

[fm-tibco]

I think the problem is that we have "application" level debug logging and then developer elvel "debug" logging. Should we distinguish those.. or have a best practices.. maybe add a TRACE level or something?

[retgits]

Hmm, so as an app developer I need to remember that not only there is an app level but also a separate extension level debug setting 🤔 that’s gonna make my life as a developer more difficult, to be honest. I agree with Frank that a TRACE level might be better here.

[mellistibco]

I agree with @retgits & @fm-tibco I don't think this is the right approach to the problem we're trying to solve.

[fm-tibco]

I think having DEBUG & TRACE and a best practices on when to use which would probably alleviate this issue a bit. I know that the log currently gets overwhelmed with debug statements that are just helpful to developers

[mellistibco]

@vijaynalawade does the problem have an issue associated with it? Want to make sure we clearly capture the problem we're trying to solve...

[vijaynalawade]

@mellistibco @fm-tibco @retgits : Today debug logs are really overwhelming and with more number of contributions used in the given app, they can get more and more difficult to read. The idea here is to just let developers (app or contrib) quickly set different log level for certain contributions than the engine. With many contributions in single app, I dont think TRACE level logging would help either. And I dont think educating users on using new variable would be a bad idea or difficult. In fact few developers might like to have the control over log levels during testing.

[retgits]

While I'd disagree with that, I also believe that we should have an open discussion so we can explore the arguments on both ends.

Why is it difficult to read? Is it the sheer number of messages on the screen, or does it have to do with the ability to find certain data points? Assuming that in a production environment the logs would go to a system like Splunk, this would only be for development purposes. So we could make the logs more readable by adding color to statements (the word error could be red, a warning could be yellow and so on). That would give you a visual indication of what to look for.

I personally think the TRACE option is a better idea because a lot of the statements I see in the DEBUG log today are most likely more TRACE statements. I'm guessing the functionality is inspired by Log4J and looking at their definitions of those two levels:

• The DEBUG Level designates fine-grained informational events that are most useful to debug an application.
• the TRACE Level designates finer-grained informational events than the DEBUG

Something like printing mapping statements that we execute might in that case actually be more of a TRACE than a DEBUG statement (and yes, I fully realize I've created PRs in the past that output debug logs where, in hindsight, a trace would have been a better option).

Educating users of a library or product is one of the most difficult things to do and I fundamentally believe that we should most certainly think about the user impact and benefits and how we help the user understand this new setting. Parts of our documentation is off-the-charts good, but other parts (like some of our settings) deserve a lot more attention.