Improve model load priority

[HifiExperiments]

Closes #834

This contains multiple changes to how we prioritize loading models:

• Avatar model entities have a higher loading priority
• Non-avatar model entities have a dynamically updated priority that changes as you move around and gives priority to things that are in view
• Model entities have a new property, loadPriority, that you can set to override the automatically computed priority. This is a bit experimental, so I have not exposed it to Create.
• In an FST, you can specify waitForWearables=true (e.g. https://gist.github.com/HifiExperiments/fbb063e083129fc04530a9e1dd833443/raw/08ee9c9c3d047c23f5e0e6765b84a953c37a120e/waitForWearablesTest.fst), which will make your avatar not show up until your avatar entity wearables have loaded.

This is all a bit difficult to test, because many factors impact loading performance. For example, the loading priority is only used if we have pending loading requests. So when you first enter a domain, the entity mixer will send you a bunch of entities, and, because we're under the concurrent download limit, some will begin loading right away without checking their priority. Only once we reach the limit are the priorities used to determine subsequent downloads. Also, just because a model starts loading first doesn't mean it will finish first. I'm open to suggestions if there are other recommendations for how to improve this.

• [ ] I also had trouble testing waitForWearables myself, so it's possible it's not working completely, could use some help making sure it works
• [ ] file follow up issue to support specifying avatar entities in FSTs, and specifying that only some are necessary to wait for

Funding

This project is funded through NGI0 Entrust, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.

[JulianGro]

I assume waitForWearables is global in the sense that is affects all wearables an avatar has? I feel like we should be able to toggle this per wearable. I might want to wait for basic clothing to load, but a hat or some other accessory is relatively unimportant and can take its time.

[HifiExperiments]

@JulianGro that's correct, waitForWearables waits for all wearables. hm...making it per-entity might be a bit complicated because clients don't really know how many avatar entities to expect until they've actually arrived. do you envision a property that you set on the entities themselves or some way to specify which ones to wait for in the FST? I also worry that making it too complicated will deter people from using it, seems easier to just have a single flag that needs to be set.

[JulianGro]

I don't really know how you specify wearables in the FST. I assumed it is just a list of other files to load; In that list, you could add that property for every entry of the list.

[HifiExperiments]

oh! I didn’t know you could do that. I think I see it in the code now. yeah I think I can make it so FST-defined entities can add a flag to just those that you actually want to wait for. maybe we also keep the global waitForWearables as a second option, especially since not all entities are defined like that?

[ksuprynowicz]

maybe we also keep the global waitForWearables as a second option, especially since not all entities are defined like that?

I think it's a really good idea.

[HifiExperiments]

I think I see it in the code now.

I was wrong lol; we support loading avatar entities from bookmarks, but not FSTs directly. that would definitely be a nice feature to have, but maybe we handle it in separate PR (along with being able to specify those individual ones to wait for)?

[ksuprynowicz]

I just tested it on Linux and something very weird is happening audio and avatar mixers are crashing:

[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 0 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 1 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 2 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 3 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 4 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 5 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 6 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] unreasonable sequence number: 7 (possible duplicate)
[08/18 16:48:10] [DEBUG] [hifi.networking] [62380] [audio-mixer] re-synced with sequence number sender
assignment-client: /home/ksuprynowicz/overte/overte_v8/overte/libraries/entities/src/EntityTree.cpp:2074: int EntityTree::processEraseMessage(ReceivedMessage&, const SharedNodePointer&): Assertion `!getIsServer()' failed.
Aborted (core dumped)

and

[08/18 16:48:09] [DEBUG] [hifi.networking] [62388] [avatar-mixer] Added "Agent" (I) {53ed13e6-102c-4475-b7c9-4462743e39c1}(30294) "UDP ""89.64.89.53":61311 / "UDP ""192.168.0.15":58793 
[08/18 16:48:09] [DEBUG] [hifi.networking] [62388] [avatar-mixer] Creating new Connection class for "UDP ""192.168.0.15":58793
[08/18 16:48:09] [DEBUG] [hifi.networking] [62388] [avatar-mixer] Creating new Connection class for "UDP ""89.64.89.53":61311
[08/18 16:48:09] [DEBUG] [hifi.networking] [62388] [avatar-mixer] Packet of type 3 (Ping) received from unknown node with Local ID 18111 
[08/18 16:48:09] [DEBUG] [hifi.networking] [62388] [avatar-mixer] Activating local socket for network peer with ID "53ed13e6-102c-4475-b7c9-4462743e39c1"
[08/18 16:48:10] [DEBUG] [hifi.avatars] [62388] [avatar-mixer] Processing first identity packet for QUuid("{53ed13e6-102c-4475-b7c9-4462743e39c1}") - 2
[08/18 16:48:10] [DEBUG] [hifi.avatars] [62388] [avatar-mixer] Giving session display name "74hc595" to node with ID QUuid("{53ed13e6-102c-4475-b7c9-4462743e39c1}")
[08/18 16:48:10] [SUPPRESS] [] [62388] [avatar-mixer] 3 repeated log entries - Last entry: "Packet of type 3 (Ping) received from unknown node with Local ID 18111 "
assignment-client: /home/ksuprynowicz/overte/overte_v8/overte/libraries/entities/src/EntityTree.cpp:2074: int EntityTree::processEraseMessage(ReceivedMessage&, const SharedNodePointer&): Assertion `!getIsServer()' failed.
Aborted (core dumped)

[ksuprynowicz]

I'm not sure if it's a bug , but it shows NaN priority for loading wearables:

[ksuprynowicz]

Next time I tried loading it priority was not NaN anymore. It looks like waitForWearables is not working though. The avatar (https://oaktown.pl/hazraptor_1/avatar_shark_3a_wearables.fst) already shows up while wearable(https://data.moto9000.moe/domain/Tile-59-68-1-1.obj) is still loading.

[HifiExperiments]

thank you @ksuprynowicz! that huge model made it a lot easier to test this, waitForWearables should be working now!

the assert was actually due to a change I made in the sound entities PR, I started marking more of the mixers properly as "servers", but some of the logic actually needed to be specifically for "isEntityServer". I believe I've fixed that

the NaNs were somewhat expected. if we were below the concurrent download limit, we just start downloading right away, and the asset doesn't get a priority at all. but I've made it show it just doesn't show anything, rather than showing NaN, because that was confusing.

[ksuprynowicz]

I just tested it again, and everything works perfectly now :) I think it's ready to merge.