[SPARK-40231][SQL][TEST] Add 1TB TPCDS Plan stability tests

[mskapilks]

What changes were proposed in this pull request?

Add 1TB (sf=1000) TPCDS plan stability tests.

Why are the changes needed?

1TB scale TPCDS dataset is used frequently to measure performance improvement. This would make it easier to see the impact of any SQL optimization in terms of plan changes.

The table stats are generated from this script https://gist.github.com/maropu/f553d32c323ee803d39e2f7fa0b5a8c3

Does this PR introduce any user-facing change?

No

How was this patch tested?

Tests only

[LuciferYang]

How long does the 1TB TPCDS test take?

[AmplabJenkins]

Can one of the admins verify this patch?

[mskapilks]

How long does the 1TB TPCDS test take?

@LuciferYang The test run in pr build took 34328 ms for 1.4 and 17138 ms for 2.7 TPCDS

[mskapilks]

cc: @dongjoon-hyun

[mskapilks]

cc: @dongjoon-hyun @kazuyukitanimura @wangyum

[wangyum]

Personally I don't think it's meaningful.

[kazuyukitanimura]

Thanks for the ping. Would you mind providing the context a bit more? E.g. what problems we are trying to solve, why the existing plan stability tests are not good enough, etc...

[github-actions[bot]]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!