add Proposal for Large Language Model Edge Benchmark Suite: Implement…

[XueSongTap]

…ation on KubeEdge-lanvs

What type of PR is this? /kind design

What this PR does / why we need it: This pr is a proposal for Large Language Model Edge Benchmark Suite: Implementation on KubeEdge-lanvs Which issue(s) this PR fixes:

Fixes # https://github.com/kubeedge/ianvs/issues/94

[MooreZheng]

@hsj576 might also need to take a look at this proposal

[hsj576]

Overall looks great to me. Just a few questions: Why do you need to add INT8 quantization, FP16 mixed precision, etc to TestCaseController? As far as I know you could load the INT8 quantization, FP16 mixed precision method by set "load_in_8bit=True," or "torch_dtype=torch.bfloat16" in “AutoModelForCausalLM.from_pretrained” function. Therefore, is it necessary to integrate those methods in the Ianvs TestCaseController?

[MooreZheng]

/lgtm

[XueSongTap]

the benchmark suite description was fixed and the edge was highlighted in the nameing of documents https://github.com/kubeedge/ianvs/pull/127/commits/4244a71e11889fd967816b788df8d8670cf528c7

[MooreZheng]

Looks fine to me for the revised version. As soon as the CI issues are fixed, it would be fine for ianvs to merge this proposal.

Seems that pylint message is not valid (as below). @XueSongTap might want to try out https://stackoverflow.com/questions/79019204/too-many-positional-arguments-on-one-machine-but-does-not-know-the-error-on-t

Run pylint '/home/runner/work/ianvs/ianvs/core'
************* Module core.testenvmanager.dataset.dataset
core/testenvmanager/dataset/dataset.py:119:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'too-many-positional-arguments' (unknown-option-value)
core/testenvmanager/dataset/dataset.py:206:0: C0303: Trailing whitespace (trailing-whitespace)
core/testenvmanager/dataset/dataset.py:214:0: C0303: Trailing whitespace (trailing-whitespace)

-----------------------------------
Your code has been rated at 9.9[8](https://github.com/kubeedge/ianvs/actions/runs/11338050517/job/31531990780?pr=127#step:5:9)/10

Error: The operation was canceled.

[XueSongTap]

Need to fix below CI-error messages before merging.

Run pylint '/home/runner/work/ianvs/ianvs/core'
************* Module core.testenvmanager.dataset.dataset 
core/testenvmanager/dataset/dataset.py:119:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value) core/testenvmanager/dataset/dataset.py:207:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value) 
core/testenvmanager/dataset/dataset.py:215:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value) core/testenvmanager/dataset/dataset.py:249:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value) 
core/testenvmanager/dataset/dataset.py:289:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value) core/testenvmanager/dataset/dataset.py:374:0: W0012: Unknown option value for 'disable', expected a valid pylint message and got 'R0917' (unknown-option-value)

-----------------------------------
Your code has been rated at 9.96/10

Error: Process completed with exit code 4.

BTW: @XueSongTap might want to try out this solution to similar error messages, as mentioned above

Hi,I have resolved the pylint CI issues. Please review the changes and consider merging this PR.

[MooreZheng]

/lgtm

[MooreZheng]

/approve

[kubeedge-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hsj576, MooreZheng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [MooreZheng]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment