Optimize benchmark scripts

[chensuyue]

Description

1. Add clean up option for benchmark scripts.
2. Fix setting for chatqna without rerank.

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds new functionality)
• [ ] Breaking change (fix or feature that would break existing design and interface)
• [x] Others (enhancement, documentation, validation, etc.)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

[github-actions[bot]]

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None