home-assistant-libs
MatterServer cannot be started more than once
I am interested in the case where there is a (TBD) bug that requires me to shutdown and restart the MatterServer. I have certainly had plenty during my development. It seems that after the server.stop() and regeneration of the MatterServer object, there is a failure regenerating the chip stack. I have been able to verify this by editing scripts/example.py and putting a loop around the code and hitting ^C to cause the termination of the first iteration, stopping the server and then trying to start the second iteration. The log output shows close to where things run amiss (starting during the shutdown of the first iteration):
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Entering shutdown phase.
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Executing provided shutdown_callback.
2024-12-22 13:00:14 Master-198 matter_server.server.server[25935] INFO Stopping the Matter Server...
2024-12-22 13:00:14 Master-198 matter_server.server.stack[25935] INFO Shutting down the Matter stack...
2024-12-22 13:00:14 Master-198 chip.native.CTL[25935] Level 39 Shutting down the stack...
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Waiting for executor shutdown.
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Shutting down async generators
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Closing the loop.
2024-12-22 13:00:14 Master-198 aiorun[25935] INFO Leaving. Bye!
iteration 1
2024-12-22 13:00:14 Master-198 matter_server.server.stack[25935] INFO Initializing CHIP/Matter Controller Stack...
2024-12-22 13:00:14 Master-198 chip.native.DL[25935] Level 39 ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /tmp/chip_kvs
2024-12-22 13:00:14 Master-198 chip.native.DL[25935] Level 39 ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_factory.ini
2024-12-22 13:00:14 Master-198 chip.native.DL[25935] Level 39 ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_config.ini
2024-12-22 13:00:14 Master-198 chip.native.DL[25935] Level 39 ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_counters.ini
2024-12-22 13:00:14 Master-198 chip.CertificateAuthority[25935] INFO Loading certificate authorities from storage...
Traceback (most recent call last):
File "/home/pi/Irricloud-substation-development/python-matter-server/scripts/example.py", line 78, in
Please provide (a lot) more info because this is way too little for us to understand your goals and issue. I have never ever seen this issue before so it must be some edge case you trigger.
What dev environment ? Why are you trying to loop the code around ? What are you trying to accomplish ? etc
This is definitely a dev environment setup and is not impacting me. I just wanted to understand failure cases with the server or client and making sure I could restart them in such a case.
fwiw, the error is in the chip library and, in my case, I do not have that built directly but installed via snap. That may or may not be part of the issue and I could retry (in a while) with a clean build.
But, I tried to give you an easy way to replicate the problem in your environment. In particular, I edited the scripts/example.py which starts the server and client, and when interrupted (ie, ^C) exits. I put that body (shown below) in a stupid loop so after the ^C, it would reestablish the server and client. It was during that reestablishment, that the error is shown.
for i in range(2):
print('iteration',i)
# Init server
server = MatterServer(
args.storage_path,
DEFAULT_VENDOR_ID,
DEFAULT_FABRIC_ID,
...
async def handle_stop(loop: asyncio.AbstractEventLoop):
"""Handle server stop."""
await server.stop()
# run the server
run(run_matter(), shutdown_callback=handle_stop)
Does that make enough sense for you to reproduce?
On Tue, Dec 31, 2024 at 12:45 AM Marcel van der Veldt @.***> wrote:
Please provide (a lot) more info because this is way too little for us to understand your goals and issue. I have never ever seen this issue before so it must be some edge case you trigger.
What dev environment ? Why are you trying to loop the code around ? What are you trying to accomplish ? etc
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
I definitely think this is a bug. Here is another manifestation.
The high level view is:
while True:
self.logger.info('matter run looptop')
self.matter_server = MatterServer(...)
loop = asyncio.new_event_loop()
try:
asyncio.set_event_loop(loop)
loop.set_debug(True)
self.logger.info('MatterController run server and client')
loop.run_until_complete(self.start_server_and_client())
self.logger.info('MatterController stop server')
loop.run_until_complete(self.stop_server())
self.logger.info('MatterController run finish')
finally:
self.logger.info('MatterController finally')
asyncio.set_event_loop(None)
loop.close()
self.client = None
time.sleep(30)
where start_server_and client looks runs for an hour then breaks out of its loop to force the reinstantiation of MatterServer above
await self.matter_server.start()
url = DEFAULT_URL
async with aiohttp.ClientSession() as session:
async with MatterClient(url, session) as client:
self.client = client
asyncio.create_task(client.start_listening())
await asyncio.sleep(10)
while True:
if an hour has gone by since start:
break # completes routine so new MatterServer is created
process client commands
Here is the log showing how the shutdown (and attempted restart) proceeded:
2025-02-02 12:21:22,447 - matter - INFO - MatterController stop server 2025-02-02 12:21:22,449 - matter_server.server.server - INFO - Stopping the Matter Server... 2025-02-02 12:21:22,512 - matter_server.server.stack - INFO - Shutting down the Matter stack... 2025-02-02 12:21:22,515 - chip.native.CTL - Level 39 - Shutting down the stack... 2025-02-02 12:21:22,534 - chip.native.DIS - Level 39 - Failed to advertise records: src/lib/dnssd/minimal_mdns/Server.cpp:344: CHIP Error 0x00000046: No endpoint was available to send the message 2025-02-02 12:21:22,537 - chip.native.DIS - Level 39 - Failed to advertise records: src/lib/dnssd/minimal_mdns/Server.cpp:344: CHIP Error 0x00000046: No endpoint was available to send the message 2025-02-02 12:21:22,573 - matter - INFO - MatterController run finish 2025-02-02 12:21:22,574 - matter - INFO - MatterController finally 2025-02-02 12:21:52,579 - matter - INFO - matter run looptop 2025-02-02 12:21:52,584 - matter_server.server.stack - INFO - Initializing CHIP/Matter Controller Stack... 2025-02-02 12:21:52,586 - chip.native.DL - Level 39 - ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /tmp/chip_kvs 2025-02-02 12:21:52,587 - chip.native.DL - Level 39 - ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_factory.ini 2025-02-02 12:21:52,587 - chip.native.DL - Level 39 - ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_config.ini 2025-02-02 12:21:52,588 - chip.native.DL - Level 39 - ChipLinuxStorage::Init: Attempt to re-initialize with KVS config file: /data/chip_counters.ini 2025-02-02 12:21:52,599 - chip.native.DL - Level 39 - Failed to reset WiFi statistic counts 2025-02-02 12:21:52,600 - chip.CertificateAuthority - INFO - Loading certificate authorities from storage... 2025-02-02 12:21:52,601 - matter - ERROR - no MatterServer Traceback (most recent call last): File "/home/pi/Irricloud-substation-production/plugins/matter.py", line 289, in run self.matter_server = MatterServer( ^^^^^^^^^^^^^ File "/home/pi/Irricloud-substation-production/python-matter-server/matter_server/server/server.py", line 140, in init self.stack = MatterStack(self, bluetooth_adapter_id, enable_server_interactions) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pi/Irricloud-substation-production/python-matter-server/matter_server/server/stack.py", line 130, in init self.certificate_authority_manager.LoadAuthoritiesFromStorage() File "/home/pi/.venv/lib/python3.11/site-packages/chip/CertificateAuthority.py", line 253, in LoadAuthoritiesFromStorage caList = self._persistentStorage.GetReplKey('caList') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'GetReplKey'
fwiw, I also tried to stop the MatterServer and then restart it (without creating a new instance of MatterServer), but that too ran into reinitialization problems in the chip library.
The Matter SDK Python bindings have quite some globals which do not allow to initialize the Matter stack twice.
Our MatterServer class initializes and holds the Matter stack, which can only be initialized once, so that limits things currently.
Technically a single MatterStack can handle multiple fabrics. So if your goal is to have multiple Matter fabrics, that should be possible. It probably needs some changes though. Specifically weprobably should make main() to initialize a single MatterStack, and pass that to the MatterServer. This would allow multiple MatterServer instances.
This makes total sense and would be great.
On Tue, Feb 18, 2025 at 3:25 PM Stefan Agner @.***> wrote:
The Matter SDK Python bindings have quite some globals which do not allow to initialize the Matter stack twice.
Our MatterServer class initializes and holds the Matter stack, which can only be initialized once, so that limits things currently.
Technically a single MatterStack can handle multiple fabrics. So if your goal is to have multiple Matter fabrics, that is possible.
We probably should make main() to initialize a single MatterStack, and pass that to the MatterServer. This would allow multiple MatterServer instances.
— Reply to this email directly, view it on GitHub https://github.com/home-assistant-libs/python-matter-server/issues/1009#issuecomment-2667148512, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADHHYLIEVTRJO4M5YCZZPXT2QO6M5AVCNFSM6AAAAABUB72WRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRXGE2DQNJRGI . You are receiving this because you authored the thread.Message ID: @.*** com> [image: agners]agners left a comment (home-assistant-libs/python-matter-server#1009) https://github.com/home-assistant-libs/python-matter-server/issues/1009#issuecomment-2667148512
The Matter SDK Python bindings have quite some globals which do not allow to initialize the Matter stack twice.
Our MatterServer class initializes and holds the Matter stack, which can only be initialized once, so that limits things currently.
Technically a single MatterStack can handle multiple fabrics. So if your goal is to have multiple Matter fabrics, that is possible.
We probably should make main() to initialize a single MatterStack, and pass that to the MatterServer. This would allow multiple MatterServer instances.
— Reply to this email directly, view it on GitHub https://github.com/home-assistant-libs/python-matter-server/issues/1009#issuecomment-2667148512, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADHHYLIEVTRJO4M5YCZZPXT2QO6M5AVCNFSM6AAAAABUB72WRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRXGE2DQNJRGI . You are receiving this because you authored the thread.Message ID: @.*** com>
