Martyna Patelka

We already use the new node, which contains: `distributed_first := (self.compile in ("eager", "inductor") or "dynamo" in self.compile):` so from Mixology perspective it should be fine.

Actually it's related to benchmark_litgpt.py script. I know one possible fix for it, so I can prepare PR around Wednesday, but it won't solve missing results from `calculate_model_flops` function.

I think we might get similar problem in another place in the code: > 0: [rank0]: Traceback (most recent call last): 0: [rank0]: File "/workspace/lightning-thunder/thunder/benchmarks/benchmark_litgpt.py", line 974, in 0: [rank0]:...

Right now Mixtral-8x7B-v0.1' works with ThunderFX, so we might close this issue.However it still doesn't work with Thunder.

We still see OOM for these models. I'm not sure if this is relevant, bu right now OOM is preceded by > [ERROR | nvfuser ]: An error occurred while...