Easiest way to log "hackney.large_pool.free_count" in an Elixir app?

[thbar]

Please bear with me as I'm just diving into hackney (I'm using it via httpoison)!

I think my app (a "business proxy") could have reached a bottleneck in terms of connection pool (I'm using a supervised connection pool as suggested in the httpoison documentation).

To properly tune this, I'd need to be able to trace down hackney.large_pool.free_count after each of my proxying request.

I've tried using exometer & folsom (which I'm just discovering), but could not get a value out of them so far.

Is there an example of use from an Elixir app that one could share?

In advance, many thanks - I'm happy to try things out and issue a documentation PR for newcomers afterward!

[benoitc]

you can provide your own backend for metrics: https://github.com/benoitc/erlang-metrics

Plan for 2.0 is to replace it by opencensus.

[benoitc]

Or we could also replace metrics by a backend using hooks

[thbar]

you can provide your own backend for metrics

Thanks for the feedback - I saw erlang-metrics before, but I was unsure if this meant I could implement my own backend or not.

Now that it's confirmed, I googled a bit more and found this blog post.

It looks like this is the way to go. I'll try this out, and will report back with a PR to make it easier for newcomer to implement this.

Also - thanks for the notes on v2 ; it's good to keep this in mind.

[mgwidmann]

This doesnt work.

Can't get hackney to start (same as #359) when bundled in a distillery release.

=SUPERVISOR REPORT==== 22-Jun-2019::01:35:48.476623 ===
    supervisor: {local,hackney_sup}
    errorContext: start_error
    reason: {badarg,
                [{ets,member,[folsom,[hackney,nb_requests]],[]},
                 {folsom_ets,handler_exists,1,
                     [{file,
                          "/Users/mgwidmann/code/elixir/prototype/deps/folsom/src/folsom_ets.erl"},
                      {line,96}]},
                 {folsom_ets,add_handler,2,

Same happens if I try to use exometer. Setting :applications like the following still doesn't fix it...

applications: [
  :logger,
  :runtime_tools,
  :folsom,
  :hackney,
  # ... rest
],

Can't you just add :exometer and :folsom to the dependencies list to fix this issue?

[benoitc]

@mgwidmann i will try tomorrow

[mgwidmann]

Any update on trying this? I'm running Elixir version 1.8.1 Erlang 21.2 if that helps any.

[benoitc]

@mgwidmann I still have to test an elixir application, but using an erlang application I don't reproduce the issue:

○ → rebar3 shell
===> Verifying dependencies...
===> Compiling testapp
Erlang/OTP 22 [erts-10.4.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Eshell V10.4.3  (abort with ^G)
1> ===> The rebar3 shell is a development tool; to deploy applications in production, consider using releases (http://www.rebar3.org/docs/releases)
===> Booted bear
===> Booted folsom
===> Booted unicode_util_compat
===> Booted idna
===> Booted mimerl
===> Booted certifi
===> Booted ssl_verify_fun
===> Booted metrics
===> Booted hackney
===> Booted testapp
 
1> hackney:get("https://friendpaste.com"). 
{ok,200,
    [{<<"Server">>,<<"nginx">>},
     {<<"Date">>,<<"Fri, 28 Jun 2019 19:22:25 GMT">>},
     {<<"Content-Type">>,<<"text/html; charset=utf-8">>},
     {<<"Transfer-Encoding">>,<<"chunked">>},
     {<<"Connection">>,<<"keep-alive">>},
     {<<"Set-Cookie">>,
      <<"FRIENDPASTE_SID=0e7b98df9d31e97dc197d09279097aa49342b731; expires=Fri, 1"...>>},
     {<<"Access-Control-Allow-Origin">>,<<"None">>},
     {<<"Access-Control-Allow-Credentials">>,<<"true">>},
     {<<"Access-Control-Allow-Methods">>,
      <<"POST, GET, PUT, DELETE, OPTIONS">>},
     {<<"Access-Control-Allow-Headers">>,
      <<"X-Requested-With, X-HTTP-Method-Override, Content-Type, "...>>},
     {<<"Access-Control-Max-Age">>,<<"86400">>},
     {<<"X-Frame-Options">>,<<"SAMEORIGIN">>},
     {<<"X-Content-Type-Options">>,<<"nosniff">>},
     {<<"X-XSS-Protection">>,<<"1; mode=block">>}],
    #Ref<0.1175794421.757596161.120162>}
2> file:read_file("rebar.config").
{ok,<<"{erl_opts, [debug_info]}.\n{deps, [hackney, folsom]}.\n\n{shell, [\n  % {config, \"config/sys.config\"},\n    {apps"...>>}
3> 

app.src:

{application, testapp,
 [{description, "An OTP application"},
  {vsn, "0.1.0"},
  {registered, []},
  {mod, {testapp_app, []}},
  {applications,
   [kernel,
    stdlib,
    folsom,
    hackney
   ]},
  {env,[{mod_metrics, folsom}]},
  {modules, []},

  {licenses, ["Apache 2.0"]},
  {links, []}
 ]}.

folsom is loaded before hackney. I will try ASAP with elixir.

[benoitc]

@mgwidmann

Can't you just add :exometer and :folsom to the dependencies list to fix this issue?

no hackney doesn't really depend on them.

Do you have any other error loading folsom?

[mgwidmann]

As I mentioned earlier, all works fine using the shell as you did. The problem is with an OTP release. When bundled in a distillery release folsom does not start up first.

[gfviegas]

Any update on that? Trying to get response times in elixir (with HTTPoison or hackney itself) and faced the same issue...