apex
apex copied to clipboard
Add get_world_info()
A method get_world_info()
will be very useful for variety of purposes especially, for testing and debugging.
Additionally, returning all zeros in get_rank_info()
when parallel is not initialized can be misleading.
I think it would be even better to have assert model_parallel_is_initialized()
in future.
cc @eqy @crcrpar
I agree on the warning part, additionally we could perhaps format the output as a log and add it to debug logging in apex.
I agree on the warning part, additionally we could perhaps format the output as a log and add it to debug logging in apex.
I think the log is already present here. ~~I do not why it is not printing it though...~~
It prints the log when running with logging.getLogger("apex").setLevel(logging.INFO)