Improve logging feature

[ixti]

First of all logging works badly on redirects:

HTTP.use(:logging => { :logger => logger }).follow.get("http://www.sensortower.com")

Prints out:

I, [2018-10-24T10:50:00.444268 #20827]  INFO -- : > GET http://www.sensortower.com/
I, [2018-10-24T10:50:00.788062 #20827]  INFO -- : < 301 Moved Permanently
I, [2018-10-24T10:50:01.476361 #20827]  INFO -- : < 200 OK

Secondly, I believe it should be more "configurable" and allow specify options Hash to set which parts of request/response needed to be logged and what level e.g.

HTTP.use(:logging => {
  :logger => logger,
  :response => false, # Disable response logging altogether,
  :request => {
    :headers => true, # Default level of request headers reporting,
    :body => :info # Non-default level of request body reporting
  }
})

[wasifhossain]

Filtering support would be awesome too. e.g.

• filter by key (strip Authorization from headers)
• filter by regexp (strip credentials from query string)

[ixti]

@wasifhossain Oh. Great idea. This can be in fact managed by allowing to pass :formatter.

[ixti]

One thing to keep in mind is that some responses (as well as requests) can't have body easily (and efficiently) log-able (like streaming ones). We should think about that.

[wasifhossain]

thanks @ixti for reminding me about the :formatter :smile: missed that completely!

we have tailored the logger to handle different cases in their own places while initializing HTTP. 1 such example:

def logger
  @logger ||= Logger.new(STDOUT).tap do |logger|
    logger.formatter = proc do |severity, time, progname, msg|
      Logger::Formatter.new.call(
        severity, time, progname,
        msg.gsub(/(Authorization: Bearer )\w+/, '\1[FILTERED]')
      )
    end
  end
end

and the log looks like

D, [2018-11-19T13:10:01.466128 #26908] DEBUG -- : Authorization: Bearer [FILTERED]
Connection: close
Host: www.example.com
User-Agent: http.rb/4.0.0