TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard

test:Add Eagle tests with untrained heads

Open brb-nv opened this issue 9 months ago • 10 comments

This MR adds unit tests to validate Eagle support for models with untrained Eagle heads. These are meant to be sanity tests which will find blatant issues such as missing Eagle support or issues with building/running eagle-enchanced models.

brb-nv avatar Mar 23 '25 19:03 brb-nv

/bot run

brb-nv avatar Mar 23 '25 20:03 brb-nv

Thanks for submitting this PR @brb-nv, I noticed that multiple E2E tests have been added. Do you have any rough estimation about the increased pre-merge time? I am asking this since we are paying attention to the CI running time to ensure the dev velocity. For sure it doesn't mean that we should never add new E2E tests, I just want to make sure we are using our CI resource in a “mean" way:)

June

juney-nvidia avatar Mar 23 '25 22:03 juney-nvidia

/bot run

zeroepoch avatar Mar 23 '25 22:03 zeroepoch

Thanks for submitting this PR @brb-nv, I noticed that multiple E2E tests have been added. Do you have any rough estimation about the increased pre-merge time? I am asking this since we are paying attention to the CI running time to ensure the dev velocity. For sure it doesn't mean that we should never add new E2E tests, I just want to make sure we are using our CI resource in a “mean" way:)

June

Hi June, thank you for the comment.

  • The tests in this MR are being added to qa/examples_test_list.txt and not L0. So, they shouldn't add to any pre-merge times.
  • qa/examples_test_list.txt is run with a relatively low frequency (once or twice a week, I believe).

Please let me know if you think we can do things differently. I'm exploring ways to add tests for features instead of individual models.

brb-nv avatar Mar 23 '25 22:03 brb-nv

PR_Github #198 [ run ] triggered by Bot

niukuo avatar Mar 23 '25 23:03 niukuo

PR_Github #198 [ run ] completed with state FAILURE

niukuo avatar Mar 23 '25 23:03 niukuo

Thanks for submitting this PR @brb-nv, I noticed that multiple E2E tests have been added. Do you have any rough estimation about the increased pre-merge time? I am asking this since we are paying attention to the CI running time to ensure the dev velocity. For sure it doesn't mean that we should never add new E2E tests, I just want to make sure we are using our CI resource in a “mean" way:) June

Hi June, thank you for the comment.

  • The tests in this MR are being added to qa/examples_test_list.txt and not L0. So, they shouldn't add to any pre-merge times.
  • qa/examples_test_list.txt is run with a relatively low frequency (once or twice a week, I believe).

Please let me know if you think we can do things differently. I'm exploring ways to add tests for features instead of individual models.

Thanks for the explanation, Balaram. I have no concern now.

Let's wait for the CI to run through now :)

June

juney-nvidia avatar Mar 24 '25 00:03 juney-nvidia

/bot run

kaiyux avatar Mar 24 '25 02:03 kaiyux

PR_Github #205 [ run ] triggered by Bot

niukuo avatar Mar 24 '25 02:03 niukuo

PR_Github #205 [ run ] completed with state SUCCESS /LLM/main/L0_MergeRequest_PR pipeline #217 completed with status: 'SUCCESS'

niukuo avatar Mar 24 '25 05:03 niukuo

/bot reuse-pipeline

kaiyux avatar Mar 31 '25 02:03 kaiyux

PR_Github #721 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd avatar Mar 31 '25 02:03 tensorrt-cicd

PR_Github #721 [ reuse-pipeline ] completed with state FAILURE Can't reuse PR_Github #0 with status: UNKNOWN

tensorrt-cicd avatar Mar 31 '25 02:03 tensorrt-cicd

/bot reuse-pipeline

kaiyux avatar Mar 31 '25 02:03 kaiyux

PR_Github #725 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd avatar Mar 31 '25 03:03 tensorrt-cicd

PR_Github #725 [ reuse-pipeline ] completed with state SUCCESS Can't reuse PR_Github #0 with status: UNKNOWN

tensorrt-cicd avatar Mar 31 '25 03:03 tensorrt-cicd

/bot run

brb-nv avatar Apr 01 '25 00:04 brb-nv

PR_Github #812 [ run ] triggered by Bot

tensorrt-cicd avatar Apr 01 '25 00:04 tensorrt-cicd

PR_Github #812 [ run ] completed with state SUCCESS /LLM/main/L0_MergeRequest_PR pipeline #658 completed with status: 'SUCCESS'

tensorrt-cicd avatar Apr 01 '25 02:04 tensorrt-cicd

/bot reuse-pipeline

kaiyux avatar Apr 01 '25 03:04 kaiyux

PR_Github #835 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd avatar Apr 01 '25 03:04 tensorrt-cicd

/bot reuse-pipeline

kaiyux avatar Apr 01 '25 03:04 kaiyux

PR_Github #835 [ reuse-pipeline ] completed with state SUCCESS Reusing PR_Github #812 for commit 450b024

tensorrt-cicd avatar Apr 01 '25 03:04 tensorrt-cicd

PR_Github #837 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd avatar Apr 01 '25 03:04 tensorrt-cicd

PR_Github #837 [ reuse-pipeline ] completed with state SUCCESS Reusing PR_Github #812 for commit ff2d1d5

tensorrt-cicd avatar Apr 01 '25 03:04 tensorrt-cicd