Utility Upload/Download functions need `cull_frees` to be called before reseting the Fence

[Erfan-Ahmadi]

Description

Right now Utility Upload/Download functions need cull_frees to be called before reseting the fence The reason for this is that streaming buffers cull_frees gets the fence statuses to see if it can free a certain block.

We are looking for solutions to avoid this important and "easily missed" requirement for the functions

Solution proposal

Current

Current solution is documentation in the code which specifies the correct usage is for the user to call cull_frees if they need to reset the fence.

Tracking Fence

Track Fence submits manually (Proposed by @devshgraphicsprogramming)

• have a submission timestamp/counter on the IQueue, we increment with each submit
• when submitting slap the timestamp/counter and a QUEUE ID (could be a pointer) into the fence as book keeping to a submitted > variable in the fence
• we keep a completed set of counters in the fence for each queue (yes that means 1 counter per queue, but we know how many queues we have when we start the device)
• when doing getFenceStatus we copy the timestamp from submitted to the completed[] belonging to the queue that did the submit
• a reset does nothing

@devshgraphicsprogramming please edit this text out below and fill in more details:

• How does the streaming buffer know if the fence was completed if the fence was reset before calling cull_frees?
• Waiting for the fence might happen indirectly, for example via wait/signal semaphores and an unrelated fence, Does that change anything related to this solution?
• Example for the case above: a series of submits chained together with wait/signal semaphores, when the last fence is signalled you'll know the first one is in SIGNALLED state as well and you don't need to wait for the first one.

Timeline Semaphores

Wait until we can bump Nabla's minimum required version to Vulkan 1.2 to make use of timeline semaphores.