First run slow with CUDA

[kai-lan]

I am using CUDA backend. In the code, I have pretty much the same setting as the Poisson tutorial. And I tested on the same examples: https://sparse.tamu.edu/FEMLAB/poisson3Db. I set up profiler that times the setup time and solve time. However, setup is very slow for the first run and is fast from then on. Solve time is consistent all the time.

But in the Poisson tutorial, the profiled time is always consistent.

Also, what is self is profiler?

First time:

[AMGCL solver:     1.539 s] (100.00%)
[ self:            0.280 s] ( 18.16%)
[  read:           1.141 s] ( 74.13%)
[  setup:          0.111 s] (  7.22%)
[  solve:          0.007 s] (  0.48%)

From then on:

[AMGCL solver:     1.200 s] (100.00%)
[ self:            0.055 s] (  4.54%)
[  read:           1.039 s] ( 86.60%)
[  setup:          0.099 s] (  8.25%)
[  solve:          0.007 s] (  0.60%)

[ddemidov]

That is a known issue/normal behavior. The first run is "warm up", when the driver does things like kernel compilation for your specific device, caching etc. The numbers in the tutorial are all from the second or later runs.

The "self" portion of the profile is anything that belongs to the outer item, but is not enclosed by any of the inner items. Here is a possible example:

prof.tic("outer");
foo(); // will be recorded as "outer.self"
prof.tic("inner");
bar();
prof.toc("inner");
prof.toc("outer");

[kai-lan]

That is a known issue/normal behavior. The first run is "warm up", when the driver does things like kernel compilation for your specific device, caching etc. The numbers in the tutorial are all from the second or later runs.

The "self" portion of the profile is anything that belongs to the outer item, but is not enclosed by any of the inner items. Here is a possible example:

prof.tic("outer");
foo(); // will be recorded as "outer.self"
prof.tic("inner");
bar();
prof.toc("inner");
prof.toc("outer");

Thanks for your reply. I have a follow-up question. How to access the total runtime from the profiler? I want to use it as a return value for my method.

[ddemidov]

profile.toc() returns time since the initiating tic, so you could use that. There is no method to get the total time across all iterations, so you could either accumulate it yourself, or possible make a PR with the functionality.