Mohammad Amin Nabian
Mohammad Amin Nabian
> [@stadlmax](https://github.com/stadlmax) could you please verify that the issue is fixed? Max confirmed this has been fixed. Closing as completed.
Hi @cr-metdev , thanks for posting your question. We started with a simplified version of the loss function, but now in a recent PR, the implementation has improved to include...
Closing as complete
@ktangsali could you please provide an update for this issue?
@akshaysubr we do have unit tests for this model: https://github.com/NVIDIA/modulus/blob/main/test/models/diffusion/test_song_unet_pos_embd.py#L41
/blossom-ci
> @mnabian Since you are revisiting GraphCast now, adding a few comments > > * Can we add the option to use `transformer_engine.LayerNorm`? In AIFS benchmarks, we just could get...
Note to myself: API updates breaks GraphCast tests. Need to update them all.
@stadlmax as far as I remember, we were using fused layernorm and that gave us nice speedup:https://github.com/NVIDIA/modulus/blob/main/modulus/models/gnn_layers/mesh_graph_mlp.py#L157... Did you also compare `transformer_engine.LayerNorm` with fused layernorm?