benchmarks

benchmarks copied to clipboard

Hi,

Thank you for creating this benchmark.

I tried building it on Ubuntu 16.04 64bit with the following config:

• gcc 5.4
• clang 5.0.1, from http://apt.llvm.org
• cmake 3.5.1
• conan 0.29.1, installed via sudo pip3 install conan

However, I got the following error:

$ git clone [email protected]:rpclib/benchmarks.git rpclib-benchmark
$ cd rpclib-benchmark
$ mkdir build && cd build
$ conan install .. --build missing
libevent/2.0.22@theirix/stable requirement OpenSSL/[>1.0.2a,<1.0.3]@lasote/stable overriden by thrift/0.9.3@sztomi/testing to OpenSSL/1.0.2g@lasote/stable
Version range '>1.0.2a,<1.0.3' required by 'libevent/2.0.22@theirix/stable' valid for downstream requirement 'OpenSSL/1.0.2g@lasote/stable'
WARN: Conflict in grpc/1.1.0@sztomi/testing
    Requirement OpenSSL/1.0.2i@lasote/stable conflicts with already defined OpenSSL/1.0.2g@lasote/stable
    Keeping OpenSSL/1.0.2g@lasote/stable
    To change it, override it in your base requirements
PROJECT: Installing /media/data/code/github/rpclib-benchmark
Requirements
    Boost/1.60.0@lasote/stable from conan-transit
    OpenSSL/1.0.2g@lasote/stable from conan-transit
    autoconf/2.69@sztomi/testing from conan-transit
    automake/1.15@sztomi/testing from conan-transit
    bison/3.0.4@sztomi/testing from conan-transit
    bzip2/1.0.6@lasote/stable from conan-transit
    capnproto/0.5.3@lasote/stable from conan-transit
    flex/2.6.3@sztomi/testing from conan-transit
    google-benchmark/1.1.0@cpace6/stable from conan-transit
    grpc/1.1.0@sztomi/testing from conan-transit
    libevent/2.0.22@theirix/stable from conan-transit
    libtool/2.4.6@sztomi/testing from conan-transit
    m4/latest@sztomi/testing from conan-transit
    pkg-config/0.29.1@sztomi/testing from conan-transit
    rpclib/1.0.0@sztomi/testing from conan-transit
    thrift/0.9.3@sztomi/testing from conan-transit
    zlib/1.2.8@lasote/stable from conan-transit
Packages
    Boost/1.60.0@lasote/stable:ca4920553084963bd998d7690b348e9806f8f446
    OpenSSL/1.0.2g@lasote/stable:c5d4f5a0443cee7ecb3c66e5b6736ad9745ead7c
    autoconf/2.69@sztomi/testing:fe389e50c4bb925b7cb51de74be9ac223b79e6df
    automake/1.15@sztomi/testing:873714f9ff21f6af4d44aafc6d007613a7e4cc3a
    bison/3.0.4@sztomi/testing:fe389e50c4bb925b7cb51de74be9ac223b79e6df
    bzip2/1.0.6@lasote/stable:0bb6f9a537a76b4aa6af7fb4dbef0aa6dddcabf4
    capnproto/0.5.3@lasote/stable:0af8d75efe38bba3cbf8f16eed4218fe0d5f9605
    flex/2.6.3@sztomi/testing:9aeda9e19058c99b9a22043c3a3f60bee7cfbeab
    google-benchmark/1.1.0@cpace6/stable:a2069270db732b89d402b71d74949d09f29d3535
    grpc/1.1.0@sztomi/testing:544e7cf9ec5368cfff8c992429499f206c66d06e
    libevent/2.0.22@theirix/stable:c63329b52ce2d11e4190404aef6b363b6c78b0a9
    libtool/2.4.6@sztomi/testing:fe389e50c4bb925b7cb51de74be9ac223b79e6df
    m4/latest@sztomi/testing:a2069270db732b89d402b71d74949d09f29d3535
    pkg-config/0.29.1@sztomi/testing:a2069270db732b89d402b71d74949d09f29d3535
    rpclib/1.0.0@sztomi/testing:a2069270db732b89d402b71d74949d09f29d3535
    thrift/0.9.3@sztomi/testing:d62b80f979e42c0f117c743b133c559bd39defa4
    zlib/1.2.8@lasote/stable:8094f5147bc9378351237bfd795472578065861f

bzip2/1.0.6@lasote/stable: Building your package in /path/to/my/home/dir/.conan/data/bzip2/1.0.6/lasote/stable/build/0bb6f9a537a76b4aa6af7fb4dbef0aa6dddcabf4
bzip2/1.0.6@lasote/stable: Copying sources to build folder
bzip2/1.0.6@lasote/stable: Generator cmake created conanbuildinfo.cmake
bzip2/1.0.6@lasote/stable: Calling build()
bzip2/1.0.6@lasote/stable:
bzip2/1.0.6@lasote/stable: ERROR: Package '0bb6f9a537a76b4aa6af7fb4dbef0aa6dddcabf4' build failed
bzip2/1.0.6@lasote/stable: WARN: Build folder /path/to/my/home/dir/.conan/data/bzip2/1.0.6/lasote/stable/build/0bb6f9a537a76b4aa6af7fb4dbef0aa6dddcabf4
ERROR: bzip2/1.0.6@lasote/stable: Error in build() method, line 36
	cmake = CMake(self.settings)
	ConanException: First argument of CMake() has to be ConanFile. Use CMake(self)

I'm not sure what I did wrong.

Could you please have a look at this problem?

Conan changed its API, the error is right there:

ERROR: bzip2/1.0.6@lasote/stable: Error in build() method, line 36
	cmake = CMake(self.settings)
	ConanException: First argument of CMake() has to be ConanFile. Use CMake(self)

So you have to change CMake(self.settings) to CMake(self). There might be other changes needed and unfortunately downgrading might not be an option because old conan versions won't work with bintray.

Thanks.

I used cmake = CMake(self) in:

• ~/.conan/data/bzip2/1.0.6/lasote/stable/export/conanfile.py:36
• ~/.conan/data/google-benchmark/1.1.0/cpace6/stable/export/conanfile.py:24
• ~/.conan/data/rpclib/1.0.0/sztomi/testing/export/conanfile.py:30

Also, the packages (libtool and libevent in my case) listed in https://github.com/tmux/tmux/issues/257#issuecomment-245315362 seem to be required.

However, I got a new error when building thrift

thrift/0.9.3@sztomi/testing: Building your package in /path/to/my/home/dir/.conan/data/thrift/0.9.3/sztomi/testing/build/677e80ecf6a3a3269db723ac2306e8981c24087b
thrift/0.9.3@sztomi/testing: Copying sources to build folder
thrift/0.9.3@sztomi/testing: Generator cmake created conanbuildinfo.cmake
thrift/0.9.3@sztomi/testing: Generator virtualenv created deactivate.sh
thrift/0.9.3@sztomi/testing: Generator virtualenv created activate.sh
thrift/0.9.3@sztomi/testing: Calling build()
thrift/0.9.3@sztomi/testing: 
thrift/0.9.3@sztomi/testing: ERROR: Package '677e80ecf6a3a3269db723ac2306e8981c24087b' build failed
thrift/0.9.3@sztomi/testing: WARN: Build folder /path/to/my/home/dir/.conan/data/thrift/0.9.3/sztomi/testing/build/677e80ecf6a3a3269db723ac2306e8981c24087b
ERROR: thrift/0.9.3@sztomi/testing: Error in build() method, line 110
        ACLOCAL_PATH = '$ACLOCAL_PATH:{}'.format(self.deps_env_info['pkg-config'].path[1])
        IndexError: list index out of range

I tried setting the environment variable using export ACLOCAL_PATH=$(aclocal --print-ac-dir) but it didn't help.

Yeah, this is horribly broken, which is a shame, because I did a lot of investment in packaging those libs precisely in the interest of build reproducibility. I will try to package them again, but I don't know when. If you really want to build this yourself, maybe the best way is to locally install the libs and get rid of the outdated conan deps. Otherwise, I think it's unlikely that you will get significantly different performance ratios from what I collected in the my blog post, so if you only want to get a general picture,, I suggest reading that: http://szelei.me/rpc-benchmark-part1/

I see, thanks anyway :)

BTW, I've recently started playing with hunter which I found to be quicker to get up and running with and less "intrusive" (less changes to my CMakeLists.txt and my cmake workflow in general) than conan. (plus, the developer is particularity active) You might want to check it out when you have time.

I spent some time today wrestling with those Conan dependencies, but had to conclude that it's pointless to follow that route, even when updating dependencies and fixing code issues. Got lots of commands which don't work (Linux Mint 18.3), outdated packages/channels and more fun.

Will try to do the benchmarking in some other fashion. Probably omitting Google Benchmark as well, since building it is already troublesome (CMake requiring the exact right GTest library).

Thanks for taking the time @MayaPosch !

No problem. One of the people behind Conan took a look at your project after I linked it on Twitter and concluded that yes, the dependencies are broken. He said that it's just due to Conan moving to a new API and such for Conan 1.0.

Basically one should prod and poke all of the owners of the relevant dependencies to make them update their packages. Could take a while, I'd say. I did notice that some dependencies in this project should simply be changed to conan/stable channel (like bzip). But good luck finding all of that out.

Honestly not sure whether it's worth the trouble to keep using Conan, but that's up to you :)

I try to run conan install .. --build missing in my machine ,but it shows

Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

google-benchmark/1.1.0@cpace6/stable: Not found in local cache, looking in remotes...
google-benchmark/1.1.0@cpace6/stable: Trying with 'conan-center'...
ERROR: Failed requirement 'google-benchmark/1.1.0@cpace6/stable' from 'PROJECT'
ERROR: Unable to find 'google-benchmark/1.1.0@cpace6/stable' in remotes

Could anyone tell me how to slove it ?

conan remote add conan-transit https://api.bintray.com/conan/conan/conan-transit

run this command before building. but I have another problem as well, checking it as well.

• conan 0.29.1 run conan remote add conan-transit https://api.bintray.com/conan/conan/conan-transit. then, I tried conan 0.28.1, which can bypass issues relating with conan version miss-matching.