azure-cli
azure-cli copied to clipboard
{testsdk} Initialize random config dir in `setUp`
Fix #28848
Description
According to https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown, tearDown
is a counterpart of setUp
, meaning anything that is done in tearDown
should have its equivalent in setUp
.
This PR solves 2 problems:
-
config_dir
leak: By injectingprint(os.getpid(), method_name)
intoazure.cli.testsdk.base.ScenarioTest.__init__
and runningazdev test
in--series
mode , we can see pytest first creates aunittest.TestCase
withmethod_name="runTest"
, and runs no test with it:$ azdev test test_containerapp_manualjob_withsecret_crudoperations_e2e --series ... collecting ... 1098 runTest 1098 test_containerapp_manualjob_withsecret_crudoperations_e2e collected 1 item src/azure-cli/azure/cli/command_modules/containerapp/tests/latest/test_containerappjob_with_secrets. py::ContainerAppJobsSecretsOperationsTest::test_containerapp_manualjob_withsecret_crudoperations_e2e PASSED [100%]
As no test method is run for
runTest
,tearDown
is not called, leading to random config dir not being deleted. -
patch.dict
pollution: From the above log, we can seeazure.cli.testsdk.base.ScenarioTest.__init__
is run at pytest's collecting staging, in the same process (1098
), buttearDown
is executed after test methods have been finished. Therefore, thispatch.dict(os.environ, {'AZURE_CONFIG_DIR': config_dir})
statement pollutes other test classes'__init__
methods while pytest collects them.
Related:
- https://github.com/Azure/azure-cli/pull/25689
- https://github.com/Azure/azure-cli/pull/26475
- https://github.com/Azure/azure-cli/pull/28673
❌AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
❌configure
❌latest
❌3.11
Type Test Case Error Message Line Failed test_configure_global_defaults self = <azure.cli.command_modules.configure.tests.latest.test_configure.ConfigureGlobalDefaultsTest testMethod=test_configure_global_defaults>
def test_configure_global_defaults(self):
# setiing the az configure defaults
> self.cmd('configure --defaults global=global1')
src/azure-cli/azure/cli/command_modules/configure/tests/latest/test_configure.py:41:
self = <azure.cli.command_modules.configure.tests.latest.test_configure.ConfigureGlobalDefaultsTest testMethod=test_configure_global_defaults>
command = 'configure --defaults global=global1', checks = None
expect_failure = False
def cmd(self, command, checks=None, expect_failure=False):
command = self._apply_kwargs(command)
> return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
E AttributeError: 'ConfigureGlobalDefaultsTest' object has no attribute 'cli_ctx'
src/azure-cli-testsdk/azure/cli/testsdk/base.py:200: AttributeErrorazure/cli/command_modules/configure/tests/latest/test_configure.py:38 ❌3.9
Type Test Case Error Message Line Failed test_configure_global_defaults self = <azure.cli.command_modules.configure.tests.latest.test_configure.ConfigureGlobalDefaultsTest testMethod=test_configure_global_defaults>
def test_configure_global_defaults(self):
# setiing the az configure defaults
> self.cmd('configure --defaults global=global1')
src/azure-cli/azure/cli/command_modules/configure/tests/latest/test_configure.py:41:
self = <azure.cli.command_modules.configure.tests.latest.test_configure.ConfigureGlobalDefaultsTest testMethod=test_configure_global_defaults>
command = 'configure --defaults global=global1', checks = None
expect_failure = False
def cmd(self, command, checks=None, expect_failure=False):
command = self._apply_kwargs(command)
> return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
E AttributeError: 'ConfigureGlobalDefaultsTest' object has no attribute 'cli_ctx'
src/azure-cli-testsdk/azure/cli/testsdk/base.py:200: AttributeErrorazure/cli/command_modules/configure/tests/latest/test_configure.py:38 ️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
🔄telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
🔄latest
🔄3.11
🔄3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes
testsdk