stan icon indicating copy to clipboard operation
stan copied to clipboard

Published 20 hours ago • verified publisher icon stan-dev

Revert "Revert "Opencl indexing""

Open SteveBronder opened this issue 3 years ago • 11 comments

Revert revert of #3048

SteveBronder avatar Jul 27 '21 17:07 SteveBronder

Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.05 2.98 1.02 2.39% faster
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.96 -3.66% slower
eight_schools/eight_schools.stan 0.11 0.11 1.07 6.54% faster
gp_regr/gp_regr.stan 0.16 0.17 0.98 -2.26% slower
irt_2pl/irt_2pl.stan 5.97 5.86 1.02 1.76% faster
performance.compilation 89.39 87.23 1.02 2.42% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.68 8.54 1.02 1.62% faster
pkpd/one_comp_mm_elim_abs.stan 31.35 32.01 0.98 -2.09% slower
sir/sir.stan 128.48 125.8 1.02 2.09% faster
gp_regr/gen_gp_data.stan 0.03 0.04 0.98 -1.88% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 3.01 3.04 0.99 -0.91% slower
pkpd/sim_one_comp_mm_elim_abs.stan 0.39 0.41 0.96 -4.67% slower
arK/arK.stan 1.88 1.87 1.01 0.68% faster
arma/arma.stan 0.94 0.82 1.15 12.76% faster
garch/garch.stan 0.64 0.53 1.19 16.31% faster
Mean result: 1.02486413129

Jenkins Console Log Blue Ocean Commit hash: 3b99831455aab0daf4f55c1af1945734bc860681

Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU: Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

Clang: Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

stan-buildbot avatar Jul 27 '21 21:07 stan-buildbot

Now that's odd, the same tests pass here but failed on develop

PR passing test https://jenkins.mc-stan.org/blue/organizations/jenkins/Stan/detail/PR-3053/1/pipeline/112#step-161-log-2747

develop failing: https://jenkins.mc-stan.org/blue/organizations/jenkins/Stan/detail/develop/917/pipeline#step-169-log-2814

@t4c1 was there some change in the underlying Stan math impl that would do this?

SteveBronder avatar Jul 27 '21 21:07 SteveBronder

I don't think so. Odd indeed.

t4c1 avatar Jul 28 '21 07:07 t4c1

Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.08 3.08 1.0 -0.11% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.96 -3.71% slower
eight_schools/eight_schools.stan 0.11 0.1 1.09 8.26% faster
gp_regr/gp_regr.stan 0.16 0.16 1.03 2.58% faster
irt_2pl/irt_2pl.stan 5.89 5.86 1.0 0.44% faster
performance.compilation 89.21 87.42 1.02 2.01% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.62 8.61 1.0 0.21% faster
pkpd/one_comp_mm_elim_abs.stan 29.1 29.43 0.99 -1.12% slower
sir/sir.stan 126.83 130.26 0.97 -2.7% slower
gp_regr/gen_gp_data.stan 0.04 0.03 1.01 1.44% faster
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.98 2.97 1.0 0.18% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.39 0.4 0.97 -3.42% slower
arK/arK.stan 1.91 1.87 1.03 2.49% faster
arma/arma.stan 0.93 0.82 1.13 11.76% faster
garch/garch.stan 0.63 0.53 1.2 16.4% faster
Mean result: 1.02716310707

Jenkins Console Log Blue Ocean Commit hash: 3b99831455aab0daf4f55c1af1945734bc860681

Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU: Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

Clang: Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

stan-buildbot avatar Jul 28 '21 12:07 stan-buildbot

Now that's odd, the same tests pass here but failed on develop

Could you check the order and included headers in src/stan/model/indexing.hpp?

- #include <stan/model/indexing/access_helpers.hpp>

Also, I see a Segmentation fault (core dumped) in the logs. It might be due to a system/environment dependency.

hsbadr avatar Jul 30 '21 14:07 hsbadr

This one has conflicts that need resolving, then lets see if the tests pass.

rok-cesnovar avatar Aug 03 '21 06:08 rok-cesnovar

Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.57 3.62 0.99 -1.38% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.94 -6.82% slower
eight_schools/eight_schools.stan 0.09 0.09 1.03 3.33% faster
gp_regr/gp_regr.stan 0.14 0.14 0.99 -0.93% slower
irt_2pl/irt_2pl.stan 5.68 5.69 1.0 -0.15% slower
performance.compilation 93.3 90.58 1.03 2.92% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.64 8.1 1.07 6.34% faster
pkpd/one_comp_mm_elim_abs.stan 30.38 32.39 0.94 -6.62% slower
sir/sir.stan 120.97 123.54 0.98 -2.12% slower
gp_regr/gen_gp_data.stan 0.03 0.04 0.97 -3.31% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 3.07 3.0 1.02 2.31% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.37 0.38 0.99 -1.18% slower
arK/arK.stan 2.06 2.06 1.0 -0.15% slower
arma/arma.stan 0.28 0.28 1.02 2.28% faster
garch/garch.stan 0.61 0.61 1.01 0.74% faster
Mean result: 0.998027605201

Jenkins Console Log Blue Ocean Commit hash: 0fa74e172028d59770129fc6f9449720ae459ce0

Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU: Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

Clang: Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

stan-buildbot avatar Jan 18 '22 20:01 stan-buildbot

Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.58 3.58 1.0 -0.02% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.98 -1.76% slower
eight_schools/eight_schools.stan 0.08 0.09 0.98 -1.71% slower
gp_regr/gp_regr.stan 0.14 0.14 1.0 0.4% faster
irt_2pl/irt_2pl.stan 5.77 5.81 0.99 -0.77% slower
performance.compilation 93.11 90.88 1.02 2.4% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.13 8.22 0.99 -1.09% slower
pkpd/one_comp_mm_elim_abs.stan 30.99 30.64 1.01 1.14% faster
sir/sir.stan 120.76 119.63 1.01 0.93% faster
gp_regr/gen_gp_data.stan 0.03 0.04 0.98 -2.46% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 3.0 3.0 1.0 -0.11% slower
pkpd/sim_one_comp_mm_elim_abs.stan 0.37 0.38 0.97 -3.19% slower
arK/arK.stan 2.08 2.1 0.99 -0.54% slower
arma/arma.stan 0.28 0.28 1.0 -0.27% slower
garch/garch.stan 0.61 0.61 1.0 0.17% faster
Mean result: 0.995629117978

Jenkins Console Log Blue Ocean Commit hash: 0fa74e172028d59770129fc6f9449720ae459ce0

Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU: Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++: Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

Clang: Apple LLVM version 7.0.2 (clang-700.1.81) Target: x86_64-apple-darwin15.6.0 Thread model: posix

stan-buildbot avatar Jan 19 '22 10:01 stan-buildbot

Thanks @SteveBronder !

rok-cesnovar avatar Jan 19 '22 10:01 rok-cesnovar

Np! But if you wouldn't mind waiting on this I am working on another PR to fix #3076 that I'd like to merge before this one so I don't have to do a big remerge and have a bunch of merge conflicts

SteveBronder avatar Jan 20 '22 00:01 SteveBronder

Do your thing Steve, will let you touch the big green button. Thanks.

rok-cesnovar avatar Jan 20 '22 08:01 rok-cesnovar

@serban-nicusor-toptal could you checkout the Jenkins file in this PR and check that the GPU setup is correct?

SteveBronder avatar May 02 '23 20:05 SteveBronder

Hey @SteveBronder I've checked it out and it looks good! ( I've cross-checked with math to be sure ) Just one small note: Recently we've pushed a new image for math with cpp17 stanorg/ci:gpu-cpp17 just in case you might need a newer version.

serban-nicusor-toptal avatar May 03 '23 10:05 serban-nicusor-toptal