Adding ipv6_mgmt_only test case into PR testing

[sdszhang]

Description of PR

Summary: Add ip/test_mgmt_ipv6_only.py into PR pipeline testing.

Type of change

• [ ] Bug fix
• [ ] Testbed and Framework(new/improvement)
• [x] Test case(new/improvement)

Back port request

• [ ] 201911
• [ ] 202012
• [ ] 202205
• [ ] 202305
• [ ] 202311

Approach

What is the motivation for this PR?

Include ip/test_mgmt_ipv6_only.py into PR pipeline testing.

How did you do it?

Add it into the PR test case list.

How did you verify/test it?

Test cases passed after configuring sonic-mgmt container with IPv6 address

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[sdszhang]

pending on https://github.com/sonic-net/sonic-mgmt/issues/12377

[qiluo-msft]

Could you resolve the conflict and check the test failure?

[sdszhang]

Could you resolve the conflict and check the test failure? @qiluo-msft I'm currently checking on the failures.

ERROR ip/test_mgmt_ipv6_only.py::test_telemetry_output_ipv6_only[vlab-01-True] - fixed in PR #12420 ERROR ip/test_mgmt_ipv6_only.py::test_ntp_ipv6_only -- missing "import setup_ntp". Easy fix.

For the rest 3, as they are all broken with error: "paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 10.250.0.101". Seems to be feature related. still checking on that.

ERROR ip/test_mgmt_ipv6_only.py::test_snmp_ipv6_only[vlab-01] - paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 10.250.0.101 ERROR ip/test_mgmt_ipv6_only.py::test_ro_user_ipv6_only[vlab-01] - paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 10.250.0.101 ERROR ip/test_mgmt_ipv6_only.py::test_rw_user_ipv6_only[vlab-01] - paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 10.250.0.101

[qiluo-msft]

May be fixed by Fix unreachable DUT mgmt_ip in IPv6 only test case by maipbui · Pull Request #12492 · sonic-net/sonic-mgmt (github.com)

[qiluo-msft]

/azp run Azure.sonic-mgmt

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[vivekverma-arista]

The RCA is correct and it's a race condition between fixture teardowns but merely changing the order of fixtures requests in test/fixture does not guarantee execution in a certain order.

https://docs.pytest.org/en/stable/reference/fixtures.html#fixture-instantiation-order

Quoting from the document Names of fixtures or tests, where they’re defined, the order they’re defined in, and the order fixtures are requested in have no bearing on execution order beyond coincidence. While pytest will try to make sure coincidences like these stay consistent from run to run, it’s not something that should be depended on.

So this issue is still hitting on our testbeds after this fix. These tests should be skipped until this issue is properly addressed.

[sdszhang]

Thanks for the comment, let's track it further in https://github.com/sonic-net/sonic-mgmt/issues/12705

[mssonicbld]

@sdszhang PR conflicts with 202311 branch

[mssonicbld]

@sdszhang PR conflicts with 202305 branch