Query 'active' networks without having to check every network in org

[JenkeScheen]

I'm trying to build a function on top of alchemiscale that will allow me to find all weights associated with all networks that currently having 'running' or 'waiting' status. With this function, I plan to build another function that will let me submit networks to alchemiscale with either higher priority than all 'waiting'/'running' networks, or instead with lower priority than all 'waiting'/'running' networks.

Currently (AFAIK) the only way of doing this is by doing something like the below (as attempted in this PR):

for key in AlchemiscaleClient.query_networks():
    n_running = AlchemiscaleClient.get_network_status(
                network=key, visualize=False).get("running", 0)
    n_waiting = AlchemiscaleClient.get_network_status(
                network=key, visualize=False).get("waiting", 0)

which gives us all networks where n_running > 0 and/or n_waiting > 0 (i.e. all networks that will still produce results). However, AlchemiscaleClient.query_networks() will keep querying networks indefinitely until it has queried all networks ever run in our org which is very time consuming (due to back & forth requests to alchemiscale) and unnecessary because these aren't important for this use-case.

We're currently breaking the loop when 'enough' networks have been queried with 0/0 but this is a very janky solution. Ideally there would be a way to only query networks that have a 'finished' status or tag.

[dotsdl]

This issue would be partially addressed by a solution to #237. This would allow users such as @JenkeScheen to reduce the number of active AlchemicalNetworks by marking those no longer of interest to a different status so they no longer appear by default from AlchemiscaleClient.query_networks.

Getting and setting relative weights of all AlchemicalNetworks of interest could then be accomplished quickly using the solution proposed in #243.

How does this sound @JenkeScheen?

[JenkeScheen]

that sounds perfect!

[dotsdl]

@JenkeScheen were you able to make use of this feature in asap-alchemy? Documentation on how to use network state is in the User Guide here: https://docs.alchemiscale.org/en/latest/user_guide.html#marking-alchemicalnetworks-as-inactive-deleted-or-invalid

[JenkeScheen]

we ended up changing our method to adjusting existing network weights, see https://github.com/asapdiscovery/asapdiscovery/blob/main/asapdiscovery-alchemy/asapdiscovery/alchemy/cli/alchemy.py#L664-L684

[JenkeScheen]

I'll have a look around our API to see if we can improve anything with asc.set_network_state(an_sk, state). Thanks!

[dotsdl]

Excellent, thank you!