[Bug] Assert CompositeSpec order guarantees

[vmoens]

Discussed in https://github.com/pytorch/rl/discussions/2341

^{Originally posted by hesic73 July 31, 2024} I have a question about the order of keys returned by CompositeSpec.keys(include_nested=True, leaves_only=False). Is there any guarantee on the order of the keys, specifically that the iterator iterates from the deepest nodes to the root?

I want to write a function that removes all empty entries from a CompositeSpec:

def _remove_empty_entries(spec: CompositeSpec) -> CompositeSpec:
    for key in list(spec.keys(True, False)):
        if not isinstance(spec[key], CompositeSpec):
            continue
        if spec[key].is_empty():
            del spec[key]
    return spec

As shown in the function above, it assumes that the iterator processes from the deepest nodes to the root. This assumption is crucial for the function to work correctly.

If there is no such guarantee, how can I achieve this goal? A CompositeSpec with empty entries is quite undesirable.

Thank you!