View of View usage

[pgrete]

Conceptual question: Our View of Views are mostly device/device and are initialized through a host mirror because we're not using a unified memory space (see https://kokkos.org/kokkos-core-wiki/ProgrammingGuide/View.html#i-really-want-a-view-of-views-what-do-i-do) , correct? (ping people who have been working on packs @Yurlungur @lroberts36 @brryan )

This view of view business in #1191 is driving me ...

To summarize my current understanding:

For a host/device view of views with the outer view in unified memory as in the Kokkos docs, we should pass the SequentialHostInit to the outer view so that the inner device views destructors are automatically called on the host (and not within a parallel region, which is illegal).

For our device/device view of views we have the following situation with the basic pattern of

1. Create outer view
2. Create host mirror
3. Create inner views on host
4. Deep copy host mirror to device
5. Host mirror might fall out of scope (but inner views continue to live -- reference counted -- as part of the outer view)

Therefore, when compiled on device:

1. Create outer device view without SequentialHostInit(because the outer view is on device and thus adding SequentialHostInit is illegal)
2. Create host mirror with SequentialHostInit
3. Create inner device views on host
4. Deep copy host mirror to device (@dalg24 does this transfer the SequentialHostInit property)?
5. Host mirror might fall out of scope (but inner views continue to live -- reference counted -- as part of the outer device view)

Therefore, when compiled on host:

1. Create outer view with SequentialHostInit(because the next step is no-op and add SequentialHostInit there would be ignored)
2. No-op: Create host mirror with SequentialHostInit
3. Create inner views on host
4. No-op: Deep copy host mirror to host

Does this make sense? Or am I missing sth?

[dalg24]

4. Deep copy host mirror to device (@dalg24 does this transfer the SequentialHostInit property)?

No it does not

[pgrete]

4. Deep copy host mirror to device (@dalg24 does this transfer the SequentialHostInit property)?

No it does not

So does this work then because host mirror is not destructed when it falls out of scope but kept alive due to the inner views are still alive?

[Yurlungur]

@dalg24 in this case, what is the proper way to build a device-side view of views that needs to be recreated multiple times? Should we just be managing the memory ourselves?

[lroberts36]

I wonder if we should just move to views of unmanaged views. I think this is essentially safe in all our use cases, since we use views of views only for pointing at variable data in packs (and for boundary stuff).

That being said, I am not sure I fully understand the implications of 'SequentialHostInit'.