'(not showing binary data)' when the content-type is text/html

[sobrinho]

Hi there!

I created a module to prepend on HttpLog and check the content of data that is passed to json_payload and data seems to be already formatted:

[17] pry(HttpLog)> data.keys
=> [:method, :url, :request_body, :request_headers, :response_code, :response_body, :response_headers, :benchmark, :encoding, :content_type, :mask_body]
[18] pry(HttpLog)> whereami

From: (pry):76 Gabriel::Gabriel#json_payload:

    75: def json_payload(data = {})
 => 76:   binding.pry
    77:   super
    78: end

The problem is that it is trying to check for data[:content_type] to decide wether or not to parse the response body.

=> ["/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:208:in `dump_json'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:198:in `log_json'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:35:in `call'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/adapters/ethon.rb:28:in `perform'",
 "/app/vendor/cache/ruby/3.1.0/gems/prometheus_http_logger-1.1.1/lib/prometheus_http_logger/adapters/ethon_adapter.rb:18:in `perform'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/operations.rb:16:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/cacheable.rb:18:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/block_connection.rb:31:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/stubbable.rb:25:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/before.rb:26:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/instruments/typhoeus.rb:61:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/stor_edge_client-2.0.0/lib/stor_edge_client/requests/base_request.rb:58:in `execute'",
 "/app/vendor/cache/ruby/3.1.0/gems/stor_edge_client-2.0.0/lib/stor_edge_client/requests/base_request.rb:41:in `execute'",
 "/app/vendor/cache/ruby/3.1.0/gems/stor_edge_client-2.0.0/lib/stor_edge_client/client.rb:43:in `get_unit_groups'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:116:in `parse_body'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:231:in `json_payload'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:208:in `dump_json'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:198:in `log_json'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/http_log.rb:35:in `call'",
 "/app/vendor/cache/ruby/3.1.0/gems/httplog-1.6.2/lib/httplog/adapters/ethon.rb:28:in `perform'",
 "/app/vendor/cache/ruby/3.1.0/gems/prometheus_http_logger-1.1.1/lib/prometheus_http_logger/adapters/ethon_adapter.rb:18:in `perform'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/operations.rb:16:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/cacheable.rb:18:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/block_connection.rb:31:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/stubbable.rb:25:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/typhoeus-1.4.0/lib/typhoeus/request/before.rb:26:in `run'",
 "/app/vendor/cache/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/instruments/typhoeus.rb:61:in `run'",

The request itself doesn't have anything special, it's a GET with text/html response and status is 200.

[sobrinho]

Found it:

[2] pry(#<Ethon::Easy>)> puts response_headers
HTTP/2 404
date: Mon, 22 Jan 2024 17:29:28 GMT
content-type: text/html

[6] pry(#<Ethon::Easy>)> response_headers.scan(/Content-Type: (\S+(; charset=\S+)?)/).flatten.first
=> nil
[7] pry(#<Ethon::Easy>)> response_headers.scan(/Content-Type: (\S+(; charset=\S+)?)/i).flatten.first
=> "text/html"

The response is on lowercase, I will monkey patch here and open a PR.