[RLlib] - Check environment numerics during run time

[simonsays1980]

Description

Following the discussion in #30471 (specifically this comment) and much checking over the last years, I would like to see some tools that simplify tracing an error back to its cause. One of such tools is checking for numerics (NaNs/Infs) in the environment during runtime.

I admit that environment checking could be considered as solely the user's responsibility, but in my opinion the framework that RLlib offers for training RL agents on any kind of environment has grown so complex and nested that it is sometimes hard for users that are not that experienced with the source code to trace back an error. In summary I see three reasons why checking numerics in an environment might me advantageous to be done inside of RLlib:

1. We face sometimes error in the environment at runtime that come from some very specific actions - we cannot test for all situation beforehand. It is easier to check then during runtime what happens.
2. RLlib has grown to a large powerful framework that contains many nested classes and complex workflows: a. Most end users are not that familiar with the source code that they could easily check the numerics within the environment. b. Debugging is still complicated and often needs the user to change some settings (e.g. local_mode). c. With RLlib we want to actually lift the heavy-weight away from the user and let her apply RL.
3. Numerics might be checked on the environment side - but even here users face the same problem that they might need to change source code they have not written.

Use case

Having an environment I can set some configuration parameters similar to disable_env_checking that check the numerics in an environment at runtime.

The checking can be done via printing warnings or - if the user wants it - via raising exceptions.