Make error logs more informative

[Nidhognit]

Problem statement

1. The logger that logs internal messages is deep in the package:

• It's very hard to extend the logger if you want to log it in more places than just to stream (like New Relic etc.).
• It's not so useful when you see an exception in one place, and details for this exception in another, for HiLoad projects it takes a huge amount of time to compare them.

2. Logs are not informative:

• For HiLoad projects with millions of rows in the table it's very hard to understand what's wrong with data processing when you see only failed SQL without any details about parameters.
• All errors are the same - because we log only SQL all logs are the same, even if the reason and place for them are different.

With more informative logs development and bugfixing will be much more easy.

[codecov-commenter]

:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 89.28%. Comparing base (f4889ff) to head (69ddd71). :warning: Report is 49 commits behind head on master. :exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #1998   +/-   ##
=========================================
  Coverage     89.28%   89.28%           
  Complexity     8058     8058           
=========================================
  Files           232      232           
  Lines         24534    24535    +1     
=========================================
+ Hits          21905    21906    +1     
  Misses         2629     2629           

Flag	Coverage Δ
5-max	89.28% <100.00%> (+<0.01%)	:arrow_up:
7.4	89.28% <100.00%> (+<0.01%)	:arrow_up:
agnostic	67.44% <0.00%> (-0.01%)	:arrow_down:
mysql	69.83% <100.00%> (+<0.01%)	:arrow_up:
pgsql	69.87% <100.00%> (+<0.01%)	:arrow_up:
sqlite	67.81% <100.00%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mringler]

Hey @Nidhognit ,

have you checked output after enabling debug mode? This will give you fully formed queries in error messages, which includes the values set in the prepared statement.

You can enable debug mode through the service container:

Propel::getServiceContainer()->useDebugMode()

or per connection:

Propel::getServiceContainer()->getWriteConnection('default')->useDebug();

The idea behind using debug mode is that Propel should not set parameters in error messages from a production database, as it might expose sensitive data to log files (see discussion in #1727).

Please check if debug output solves your issue.

[Nidhognit]

@mringler yes, maybe all parameters are a bit risky, and definitely there should be some configurations on what to log and what is not. But why I have to enable debug mode on production just to see a real error? why real error is not a part of the log, what benefit of this hiding?