Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard

[BUG] The argument --no-position-embedding should be fixed

Open Hoonly opened this issue 1 year ago • 3 comments

Describe the bug from the file Megatron-LM/megatron/training/arguments.py

    group.add_argument('--no-position-embedding',
                       action='store_false',
                       help='Disable position embedding. Deprecated: use --position-embedding-type',
                       dest='add_position_embedding')

I can see that this argument is Deprecated, but if we only use --position-embedding-type=rope, the add_position_embedding will accept default value True, conflict to the rope

To Reproduce

Expected behavior

Stack trace/logs

Environment (please complete the following information):

  • Megatron-LM commit ID 0d983e64afcd84cab83124e0b7ca89a3d8ec9655
  • PyTorch version 2.3.0a0+ebedce2
  • CUDA version V12.3.107
  • NCCL version 2.19.4

Proposed fix If you have a proposal for how to fix the issue state it here or link to a PR.

Additional context Add any other context about the problem here.

Hoonly avatar May 14 '24 11:05 Hoonly

Marking as stale. No activity in 60 days.

github-actions[bot] avatar Jul 13 '24 18:07 github-actions[bot]

这是来自QQ邮箱的自动回复邮件。   您好,您的邮件我已收到,谢谢。

Hoonly avatar Jul 13 '24 18:07 Hoonly

Marking as stale. No activity in 60 days.

github-actions[bot] avatar Sep 12 '24 18:09 github-actions[bot]