Compil. time: http_client.h -> abstract_http_client.h

[mj-xmr]

Result

Compilation time reduced by -10% .

Description

Below is the complexity of the http_client class:

It's a class derived from epee::net_utils::http::abstract_http_client and could be pointed at using this base class' pointer.

This is how the http_client's dependees tree looks like on master:

and this is how I was able to reduce it:

According to my compilation time report, on master, the header net/http_client.h takes over a minute to compile:

78479 ms: /tmp/monero/monero-clean/external/easylogging++/easylogging++.h (included 226 times, avg 347 ms), included via:
(...)
68317 ms: /tmp/monero/monero-clean/contrib/epee/include/net/http_client.h (included 38 times, avg 1797 ms), included via:
(...)
64580 ms: /tmp/monero/monero-clean/src/cryptonote_basic/difficulty.h (included 89 times, avg 725 ms), included via:
(...)

A side effect was, that I had to add the following 3 pure virtual methods to epee::net_utils::http::abstract_http_client:

    virtual const std::string &get_host() const = 0;
    virtual const std::string &get_port() const = 0;
    virtual void wipe_response() = 0;

Measurements

using time make on master and on the branch:

Previous	Current	Change
real 49m0,890s	real 44m16,309s	-10.06%

Of course, individual verification is welcome.

[mj-xmr]

Thanks for testing, @reemuru . Yes, there's a lot of potential here, which you can see in the CBTA report . The Doxygen report to the right helps in devising an appropriate strategy.

[mj-xmr]

As a side note, single threaded tests are preferred, due to multithreaded tests suffering from thread starvation.

[mj-xmr]

@reemuru The correct link to the report is here.

[reemuru]

@reemuru The correct link to the report is here.

@mj-xmr thanks for the info and report. I will keep a mental note for single-threaded tests.

[selsta]

master (time make)

Executed in   17.34 mins    fish           external
   usr time   16.40 mins    0.14 millis   16.40 mins
   sys time    0.86 mins    1.41 millis    0.86 mins

patch (time make)

Executed in   17.00 mins    fish           external
   usr time   16.10 mins    0.27 millis   16.10 mins
   sys time    0.83 mins    3.83 millis    0.83 mins

Seeing as this is a small diff and other have higher gains this looks good to merge once reviewed.

[mj-xmr]

@selsta I'd hate to argue all the time about this, but it seems, that you might be hitting some I/O bottlenecks, since the differences on your machine are always smaller.

[jeffro256]

Here are my results:

master
real	63m45.448s
user	60m5.132s
sys	3m28.104s

compil-epee-http-client
real	61m43.451s
user	58m18.784s
sys	3m22.170s

Specs:

Ubuntu 20.04.4 LTS
Dell XPS 15 9570
Intel Core i7-8750H
31 GiB RAM
NVME SSD

For each run:

rm -rf build/
ccache --clear
time make

So I had a (3825s - 3703s) / (3825s) * 100% = 3.190% speedup

[mj-xmr]

@jeffro256 Thanks for the test. I should have said it beforehand, but you didn't need to sacrifice your ccache for me though :)

Here's a softer and more objective way to test the compilation time alone:

mkdir -p build/time && cd build/time
git checkout master
cmake ../.. -DUSE_CCACHE=OFF -DBUILD_TESTS=ON -DBUILD_SHARED_LIBS=ON && make clean && time make > /dev/null
git checkout TESTED_BRANCH_NAME
cmake ../.. -DUSE_CCACHE=OFF -DBUILD_TESTS=ON -DBUILD_SHARED_LIBS=ON && make clean && time make > /dev/null

The shared libs reduce the effects of linkage time on the measurements.

On top of that, to reduce the effects of other I/O bottlenecks, my build directory, as well as source are both placed in a RAMDisk.

[mj-xmr]

I'll be doing a couple of small tests here, publish the results, and then you decide what to do with this PR.