pynbody
How to use filter/mask with calculate_all() ?
Hi all,
I am in situation where I would like to calculate a property for all halos in a timestep that satisfy a condition on a different property (all halos in a mass range for example).
It feels like this should be feasible since the web interface allows you to filter halos on a given condition very easily but I can't find the right syntax to make it work within python with calculate_all().
An example of this use case is actually provided here https://github.com/pynbody/tangos/blob/7fb60522ee6590bf9a6f817c2c197aa448698c25/tangos/examples/mergers.py#L31-L42
but in a wasteful way (e.g. calculate for all halos and then mask, rather than calculate just for the masked halos)
Is this something that is easily achievable ? Where is the definition of the underlying filtering command used by the web interface ?
Unless the calculation is very expensive, you don't lose much by filtering afterwards. The web interface only filters in the front-end interface, behind the scenes it still calculates every row. But if you have a case where this is causing problems, it wouldn't be too hard to implement pre-filtering. You would want to create a temporary halo list (see https://github.com/pynbody/tangos/blob/master/tangos/temporary_halolist.py) and query against that (see https://github.com/pynbody/tangos/blob/master/tangos/core/halo.py#L300 for an example)
This is for an expensive live-calculation that I wanted to calculate across multiple halos.
The easier is probably to turn it into a written calculation for which filter are implemented through --include-only.
Yes, that seems like a sensible solution although maybe one day it would be worth implementing a way to seamlessly filter before calculating.