allure-python icon indicating copy to clipboard operation
allure-python copied to clipboard

using pytest.mark.parametrize will miss some testcase in report

Open Pegasus-Yang opened this issue 2 years ago • 2 comments

I'm submitting a ...

  • [x] bug report
  • [ ] feature request
  • [ ] support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

when i use pytest.mark.parametrize ,in specific data situations, i got same historyId for two diffent testcase, when report cameout, only one of two testcase show in report.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Code to repeat :

data = [
    ["", "admin", 12345],
    ["admin", "", 12345],
]


@pytest.mark.parametrize("name, account, phone_number", data)
def test_add_member(name, account, phone_number):
    print(name, account, phone_number)

What is the expected behavior?

expected got 2 testcase in report

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Other information

Becourse in allure_pytest.util.get_history_id, only use values in original_values to generate historyId Maybe use both name and value will solve the problem

Pegasus-Yang avatar Dec 05 '23 11:12 Pegasus-Yang

Thank you for your report, @Pegasus-Yang ! There is a bug in how we hash arguments to create historyId. Due to this bug, empty strings don't affect the result, so the actual sequence of arguments that get into historyId is ["admin", 12345] in both cases. The same bug, I believe, causes some other collisions, like 1 and "1". I'll fix it soon.

delatrie avatar Dec 05 '23 12:12 delatrie

Here is a workaround when you don't need the history feature, but just one-off report:

find allure-results -name '*-result.json' -print \
  | xargs -r -n1 -P8 -i -- sh -c 'jq ".historyId=.uuid" {} | sponge {}'

saaj avatar Jun 24 '24 15:06 saaj