Megatron-LM
Megatron-LM copied to clipboard
[bugfix] Fix the incorrect with-statement
You may want to enter multiple context managers in one with-statement.
However, with rng_context and fp8_context
is equivalent to with fp8_context
, because rng_context
doesn't rewrite __and__
method.
The correct way is with rng_context, fp8_context:
.
Ref: https://docs.python.org/3/reference/compound_stmts.html#the-with-statement