skywire icon indicating copy to clipboard operation
skywire copied to clipboard

Too many dmsg clients

Open 0pcom opened this issue 2 weeks ago • 0 comments

When the dmsg non-transportability / shutdown logic was replaced with logic that restarted the transport manager module for that transport type ; it seems that previously existing dmsg clients were not shut down or closed out. This is the likely source of the ballooning instability for dmsg recently. Here is seems my visor had several hundred dmsg clients due to the non-transportability logic revisions since v1.3.31

[2025-12-10T17:44:52.104053758-06:00] DEBUG [dmsgC]: All sessions closed.
[2025-12-10T17:44:52.104014649-06:00] DEBUG [dmsgC]: On (*ClientSession).serve() return, close client session resulted in error. error="io: read/write on closed pipe" session=0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb
[2025-12-10T17:44:52.104803332-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104865406-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104951231-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104804886-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.105064371-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.105103432-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
...
[2025-12-10T17:44:52.108616688-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.108253311-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.108402661-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107308247-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10798043-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107691311-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107566966-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107492717-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107245993-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106858303-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107024714-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107143888-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106640347-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106586396-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106536104-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106379741-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106755516-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10632137-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.105859517-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104902643-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10488904-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.105002573-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104867007-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104836402-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.104836715-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
...
[2025-12-10T17:44:52.105308589-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.105802848-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106089026-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.1061203-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106187328-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106069645-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106268006-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106230664-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106251925-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106146019-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106332907-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106348161-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106366824-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106312861-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106302585-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106487684-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106607937-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106658638-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106709243-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106725478-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10677252-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106520827-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106571407-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.106983011-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10704682-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107094122-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107110579-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107159336-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107175372-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107191225-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107263262-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107323567-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107340591-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107356489-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10737121-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107293503-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.10721974-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107385277-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107504171-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107514306-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107531518-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107581261-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107595561-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:44:52.107656918-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
...
[2025-12-10T17:44:52.401842118-06:00] WARN [52/356] [visor:shutdown:dmsg]: Module stopped with unexpected result. elapsed=298.679328ms error="entry of public key is not found"
...
[2025-12-10T17:45:07.259169176-06:00] WARN [dmsgC]: Stopped accepting streams. error="io: read/write on closed pipe" session=02a2d4c346dabd165fd555dfdba4a7f4d18786fe7e055e562397cd5102bdd7f8dd
[2025-12-10T17:45:07.259197444-06:00] DEBUG [dmsgC]: On (*ClientSession).serve() return, close client session resulted in error. error="io: read/write on closed pipe" session=02a2d4c346dabd165fd555dfdba4a7f4d18786fe7e055e562397cd5102bdd7f8dd
[2025-12-10T17:45:07.259202356-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.259211115-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.259211309-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.259215705-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.259237288-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.259220538-06:00] DEBUG [dmsgC]: On (*netutil.Porter).CloseAll(), closing contained value resulted in error. error="io: read/write on closed pipe"
[2025-12-10T17:45:07.551488866-06:00] WARN [3/356] [visor:shutdown:dmsg]: Module stopped with unexpected result. elapsed=292.973975ms error="entry of public key is not found"
[2025-12-10T17:45:07.551605143-06:00] DEBUG [2/356] [visor:shutdown:address_resolver]: Shutting down module...
[2025-12-10T17:45:07.616629307-06:00] DEBUG [2/356] [visor:shutdown:address_resolver]: Module stopped cleanly. elapsed=65.029637ms
[2025-12-10T17:45:07.616700389-06:00] DEBUG [1/356] [visor:shutdown:event_broadcaster]: Shutting down module...
[2025-12-10T17:45:07.616831916-06:00] DEBUG [1/356] [visor:shutdown:event_broadcaster]: Module stopped cleanly. elapsed=133.131µs
[2025-12-10T17:45:07.616877615-06:00] INFO [visor:shutdown]: Shutdown complete. Goodbye!

0pcom avatar Dec 10 '25 23:12 0pcom