certification-tool
certification-tool copied to clipboard
[Bug] SDK Python tests fails when running using cli.sh
Describe the bug
When running the SDK python tests using cli.sh the test raises a Value error ValueError: Level 'PYTHON_TEST' does not exist
. This is observed using test harness version v2.10+spring2024.
Steps to reproduce the behavior
The error can be observed be running the following command on the test harness (version v2.10+spring2024)
- Make the DUT in commissioning mode Eg. Run all-cluster-app, lighting app or any other application.
-
./cli.sh run-tests --project-id 1 -s '{"SDK Python Tests":{"Python Testing Suite":{"TC_ACE_1_3":1}}}'
Expected behavior
Creating new test run with title: 2024-06-13-13:13:28
Starting Test run: Title: 2024-06-13-13:13:28, id: 18
Test Run [EXECUTING]
- Python Testing Suite [EXECUTING]
=======================================
Make sure the DUT is in Commissioning Mode
1: DONE
2: FAILED
Please enter a number for an option above:
1
=======================================
- TC-ACE-1.3 [EXECUTING]
- Start Python test [EXECUTING]
Traceback (most recent call last):
File "/home/ubuntu/certification-tool/cli/app/main.py", line 47, in <module>
root()
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/certification-tool/cli/app/async_cmd.py", line 28, in wrapper
return asyncio.run(f(*args, **kwargs))
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/ubuntu/certification-tool/cli/app/commands/run_tests.py", line 65, in run_tests
await socket_task
File "/home/ubuntu/certification-tool/cli/app/test_run/websocket.py", line 71, in connect_websocket
await self.__handle_incoming_socket_message(socket=socket, message=message_obj)
File "/home/ubuntu/certification-tool/cli/app/test_run/websocket.py", line 82, in __handle_incoming_socket_message
self.__handle_log_record(message.payload)
File "/home/ubuntu/certification-tool/cli/app/test_run/websocket.py", line 128, in __handle_log_record
logger.log(record.level, record.message)
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/loguru/_logger.py", line 1997, in log
__self._log(level_id, static_level_no, False, __self._options, __message, args, kwargs)
File "/home/ubuntu/.cache/pypoetry/virtualenvs/certification-tool-cli-Bd4yKGBQ-py3.10/lib/python3.10/site-packages/loguru/_logger.py", line 1886, in _log
raise ValueError("Level '%s' does not exist" % level_id) from None
ValueError: Level 'PYTHON_TEST' does not exist
Log files
No response
PICS file
No response
Screenshots
No response
Environment
No response
Additional Information
The fix is to add the following lines in cli/app/test_run/logging.py
# Add custom logger for python tests
PYTHON_TEST_LEVEL = "PYTHON_TEST"
logger.level(PYTHON_TEST_LEVEL, no=22, icon="🐍", color="<cyan>")
Happy to create a PR for this. Just want to confirm that are the steps to contribute to this repo same as steps mention in the Matter SDK ?