tjoymeed
tjoymeed
Thanks! That means Sept of 2025? If so then that's too late. Could you please give some pointers regarding integrating PAI-Megaton-Patch's stage 3 pretraining feature into Nemo-Megatron, since PAI-Megaton-Patch has...
> We're working on better long context training support right now, but I don't have any near term ETA to share with you at this time. > > Qwen3 MoE...
I used: enroot import docker://nvcr.io/nvidia/nemo:25.04, enroot import docker://nvcr.io/nvidia/nemo:dev they both don't have qwen3 recipes inside. I then tried: enroot import docker://nvcr.io/nvidia/nemo:nightly enroot import docker://nvcr.io/nvidia/nemo:latest But got 404 errors: [INFO] Querying...
So the conclusion is? Do we add these two lines, or simply ignore the warning?
I am now trying to convert the lora-adaptor part. After I convert the lora-adapter part, I used "--peft-path" parameter in the training shell script to set the path to the...
Hi @snowmanwwg Thanks a lot for your attention. We are exploring to optimizing training throughput. We start with replicating the results of the performance scripts of Qwen3-30B-A3B ... https://github.com/NVIDIA-NeMo/NeMo/blob/main/scripts/performance/llm/pretrain_qwen3_30b_a3b.py Being...
Note the following had no problem: mike@ln01:~/enroot$ srun --nodes=1 --ntasks=1 --partition=gpu-interactive --account=gpu --container-image=docker://ubuntu:22.04 /bin/echo "Pyxis works" srun: job 1276824 queued and waiting for resources srun: job 1276824 has been allocated...
I have to get the local updated unpacked rootfs folder working (since I did the enroot import and then enroot create and then did some local updates already...) I have...
Okay, could you please provide a fully replicable example (including the sbatch script and the python script and the sample data) for the performance script: https://github.com/NVIDIA-NeMo/NeMo/blob/main/scripts/performance/llm/pretrain_qwen3_30b_a3b.py I am using exactly...
> Our recommended workflow (what we use internally all the time) is just to mount the NeMo directory inside if all that's changing is NeMo. Interesting! Could you please give...