moose icon indicating copy to clipboard operation
moose copied to clipboard

Add debug loop execution ordering output

Open GiudGiud opened this issue 2 years ago • 1 comments

early PR to get feedback still a few things to do at least:

  • [x] add tests
  • [x] make sure FEProblem always knows its execute_on (currently doesnt seem like it)
  • [x] make sure we only print the objects that are executed on each execute_on and not all the objects

Currently this looks like this

[DBG] Beginning Elemental loop to compute residual on NONE
[DBG] Execution order on each element:
[DBG] - kernels on element quadrature points
[DBG] Ordering of kernels:
[DBG] diff convection diff_v 
[DBG] Only objects on local block/sideset are executed in each list

GiudGiud avatar Jul 14 '22 06:07 GiudGiud

Changing a few things, not ready for feedback for now

GiudGiud avatar Jul 15 '22 15:07 GiudGiud

Job Precheck on 797d49f wanted to post the following:

Your code requires style changes.

A patch was auto generated and copied here
You can directly apply the patch by running, in the top level of your repository:

curl -s https://mooseframework.inl.gov/docs/PRs/21586/clang_format/style.patch | git apply -v

Alternatively, with your repository up to date and in the top level of your repository:

git clang-format 9faad45639efa6b11ea6e6ac170d1f39f4d9891e

moosebuild avatar Aug 30 '22 16:08 moosebuild

This should roughly pass.

I d be happy to get feedback on lowering the footprint of this on the loops code. Maybe add some stringify definitions? I m not sure what to do without making crazy templates that won't really help with code clarity (but maybe that s the way, stringify<any class>( .. ) { return ..->name() } ??

GiudGiud avatar Mar 22 '23 22:03 GiudGiud

@loganharbour @lindsayad @roystgnr need a volunteer for a coarse first pass

GiudGiud avatar Mar 22 '23 22:03 GiudGiud

Job Documentation on 89024e5 wanted to post the following:

View the site here

This comment will be updated on new commits.

moosebuild avatar Mar 23 '23 01:03 moosebuild

Job Coverage on 89024e5 wanted to post the following:

Framework coverage

40cceb #21586 89024e
Total Total +/- New
Rate 84.91% 84.96% +0.05% 94.56%
Hits 84351 84732 +381 608
Misses 14987 14997 +10 35

Diff coverage report

Full coverage report

Modules coverage

Coverage did not change

Full coverage reports

Reports

This comment will be updated on new commits.

moosebuild avatar Mar 23 '23 01:03 moosebuild

I think I can get a tad more coverage with simple tests. I ll do That concurrently to addressing reviews

GiudGiud avatar Mar 23 '23 01:03 GiudGiud

I think I should be able to start reviewing sometime today

lindsayad avatar Mar 23 '23 15:03 lindsayad

Might be worth waiting a bit at this point. I think there s two big tasks left

GiudGiud avatar Mar 24 '23 20:03 GiudGiud

works for me. Just mention me to re-subscribe me

lindsayad avatar Mar 28 '23 00:03 lindsayad

I messed up a few things in the process (32 tests), but this is closer to where we want to be I think Nesting under nonlinearthread allows me to consolidate the print routines a lot

@lindsayad

GiudGiud avatar Mar 28 '23 15:03 GiudGiud

Should be ready for another round now. The 32 errors were me trying to move the general UOs into their own separate loop instead of joinAndFinalize. I should have known better.

GiudGiud avatar Mar 28 '23 17:03 GiudGiud

Do you expect this to affect non-debug performance at all?

Hopefully not. But I did change up the Jacobian loops so maybe?

GiudGiud avatar Mar 28 '23 20:03 GiudGiud

@lindsayad going to resolve the suggestions manually because it didnt auto-resolve on merging them. I checked that i didnt miss any though

GiudGiud avatar Mar 28 '23 21:03 GiudGiud

I went through and resolved all but 2 it looks like

lindsayad avatar Mar 28 '23 21:03 lindsayad

well that explains the clicking not working thanks!

GiudGiud avatar Mar 28 '23 21:03 GiudGiud

Job Test timings on 89024e5 wanted to post the following:

View timings here

This comment will be updated on new commits.

moosebuild avatar Mar 28 '23 22:03 moosebuild

I have tracked down the app failure to this commit 23c14883d8b6cc27e323fb6ae3c539a40f6276e8 it doesnt make much sense. I ll see if my new debugging tool can help me figure it ^^

GiudGiud avatar Mar 28 '23 23:03 GiudGiud

@dschwen could you please help here This bison test runs kernel twice on every LINEAR iteration before this commit (https://github.com/idaholab/moose/commit/23c14883d8b6cc27e323fb6ae3c539a40f6276e8) test:tensor_mechanics/mn_thermal_expansion.g/nonad and once after (when the flag is set to LINEAR on residual evaluations instead of NONE)

do you know why that could be?

also why is the final error from an AB2PredictorCorrector? I dont see it in the log or the input file?

GiudGiud avatar Mar 28 '23 23:03 GiudGiud

I'm not sure what's going on with the bison failure. The test is for simple 3d thermal expansion verification test. The strange thing is that the AD version works just fine, but the nonad version hits a DIVERGED_NANORINF. It uses decomposition_method = EigenSolution and small strain, and nothing else too crazy. It can be simplified by turning the temperature into an aux variable, but I fear the fact nonad isn't working and AD is indicates something deeper...

tophmatthews avatar Apr 03 '23 19:04 tophmatthews

Found the problem. I'm ashamed how long it took

GiudGiud avatar Apr 03 '23 21:04 GiudGiud

We should be unstuck now. This was a good thing overall, i think we're in a better place with that last commit.

Down for another round of review!

GiudGiud avatar Apr 03 '23 22:04 GiudGiud

Job Python 3.8 on 89024e5 : invalidated by @lindsayad

moosebuild avatar Apr 06 '23 19:04 moosebuild

Job Python 3.9 on 89024e5 : invalidated by @lindsayad

moosebuild avatar Apr 06 '23 19:04 moosebuild