debugpy: breakpoints not working in core integrations like `tuya`

[auphof]

The problem

Any breakpoints created in vscode debugger in a component such as Tuya don't trigger.
For instance I have a breakpoint set at https://github.com/home-assistant/core/blob/cfd1f7809f6f6dc3a40682719036f4479c28bbb3/homeassistant/components/tuya/init.py#L150. The Tuya debug log is enabled and I can see these in the log records from my Tuya devices, but the breakpoint does not occur.

I have followed instructions from the integration https://www.home-assistant.io/integrations/debugpy. I can remote attach successfully , all the threads are displayed in VSCODE debugger. if i attempt to pause a thread such as the thread Tuya is operating on (Thread ID obtained from Log Record) it does not happen immediately (Possibly async related) but periodically it does pause at some random spot, I can used Debug Console to interrogate the HA objects and step from there, however it never pauses at the intended breakpoint

I note in the Log of the HA container that when i start the DebugPy integration Service it talks about requiring python to be started with -Xfrozen_modules=off

When I apply that in the service run python3 -Xfrozen_modules=off -m homeassistant --config /config then breakpoints function as intended

Expected Outcome

Debug breakpoints in production HA container just work as per integration provided

I realize that might not be achievable so maybe an appropriate Developer Doc is created to support this. see my suggested method below, I am willing to put this as a PR against the develop docs, I am also willing to try other approaches.

A method of Debugging Production HA Containers

To Debug the latest HA Container I built a new container from the following dockerfile.

This Dockerfile is designed to set up a debugpy tuned containerized environment for debugging Home Assistant, The file outlines several steps to customize the container for fixing the debugpy breakpoints issue performance and potential integration improvements. Here's a breakdown of what each part does:

1. Base Image: The Dockerfile starts with FROM homeassistant/home-assistant:latest, which specifies that the container will be built from the latest version of the Home Assistant Docker image. This image contains all the necessary dependencies to run Home Assistant.

2. Customizing Service Script:

   • The first RUN command replaces the default service script for Home Assistant with a custom one. The script is written to the file /usr/src/homeassistant/rootfs/etc/services.d/home-assistant/run.
   • A similar operation is performed again, but this time the script is written to /etc/services.d/home-assistant/run. It's written to both places but i think it would work if only written to /etc/services.d/home-assistant/run. Both scripts:
     • Start with navigating to the /config directory, where Home Assistant's configuration files are expected to be found. If this directory does not exist, the script exits with an error.
     • Check if the DISABLE_JEMALLOC environment variable is not set. If it's not set, the script configures the container to use jemalloc as the memory allocator for Home Assistant, which can offer performance benefits. It does this by setting the LD_PRELOAD environment variable to preload libjemalloc.so.2 and configuring jemalloc settings via MALLOC_CONF.
     • Execute Home Assistant using Python 3, with a specific flag to disable frozen modules for improved performance or compatibility and specifying the configuration directory.

3. Commented Out Code for Tuya SDK:

   • There are commented-out sections that indicate optional steps for testing or using a patched version of the Tuya device sharing SDK. These sections are not active in the current state of the Dockerfile but suggest a way to uninstall the official Tuya SDK and replace it with a version from a specific Git repository. This could be useful for integrating or testing features not available in the official SDK release.
   • Another commented-out command shows how to patch the Tuya component within Home Assistant to use a different version of the Tuya SDK. This is done by modifying the manifest.json file of the Tuya component to require a new version of the SDK.

The commented-out sections offer flexibility for developers/testers working with customized but not released 3rd party python modules. Enabling the live testing of changes developed in the Devcontainer approach

FROM homeassistant/home-assistant:latest

RUN \
  cat > /usr/src/homeassistant/rootfs/etc/services.d/home-assistant/run <<EOF
#!/usr/bin/with-contenv bashio
# ==============================================================================
# Start Home Assistant service
# ==============================================================================

cd /config || bashio::exit.nok "Can't find config folder!"

# Enable mimalloc for Home Assistant Core, unless disabled
if [[ -z "${DISABLE_JEMALLOC+x}" ]]; then
  export LD_PRELOAD="/usr/local/lib/libjemalloc.so.2"
  export MALLOC_CONF="background_thread:true,metadata_thp:auto,dirty_decay_ms:20000,muzzy_decay_ms:20000"
fi
exec python3 -Xfrozen_modules=off -m homeassistant --config /config
EOF


RUN \
  cat > /etc/services.d/home-assistant/run <<EOF
#!/usr/bin/with-contenv bashio
# ==============================================================================
# Start Home Assistant service
# ==============================================================================

cd /config || bashio::exit.nok "Can't find config folder!"

# Enable mimalloc for Home Assistant Core, unless disabled
if [[ -z "${DISABLE_JEMALLOC+x}" ]]; then
  export LD_PRELOAD="/usr/local/lib/libjemalloc.so.2"
  export MALLOC_CONF="background_thread:true,metadata_thp:auto,dirty_decay_ms:20000,muzzy_decay_ms:20000"
fi
exec python3 -Xfrozen_modules=off -m homeassistant --config /config
EOF


# Uncomment for testing against a  patched tuya-device-sharing-sdk
#RUN \
#  pip uninstall -y tuya-device-sharing-sdk && \
#  pip install git+https://github.com/auphof/tuya-device-sharing-sdk.git@dev-au-fix-pir-24w06b#egg=tuya-device-sharing-sdk


# Patch the Tuya component manifest to use the updated version
# file is /usr/src/homeassistant/homeassistant/components/tuya/manifest.json
# find `"requirements": ["tuya-device-sharing-sdk==0.1.9"]` and
# replace with `"requirements": ["tuya-device-sharing-sdk==0.1.10"]`


## Assuming /usr/src/homeassistant is already populated with the Home Assistant code
#RUN sed -i 's/"requirements": \["tuya-device-sharing-sdk==0.1.9"\]/"requirements": ["tuya-device-sharing-sdk==0.1.10"]/g' #/usr/src/homeassistant/homeassistant/components/tuya/manifest.json

Build the modified image on a target platform

# In my case I am targeting a PI4 , so the following is done in a folder on a  pi4

#1. Edit and place a copy of the above dockerfile in a file `dockerfile.update` in the appropriate dir
mkdir -p ha-container-patch && cd ha-container-patch
cat > dockerfile.update <<EOF
#<<<<paste and edit the relevant dockerfile parts from above in place of this line>>>>
EOF

#2. Define the name for the image, ie  repository/tag for the customized HA container Image
# example TARGET_REPO_TAG="10.1.1.155:5101/homeassist:patched1a"  #where this is my local network private docker image registry
TARGET_REPO_TAG="localhost/homeassist:patched-1a" #accessible on this machine

#3. Build new image and push if required
sudo docker build -f dockerfile.update -t $TARGET_REPO_TAG.
#Push If you are using a local private registry (Useful for HA container images being so large > 1.4Gb)
# NB:         Configuration of a local private registry is not covered here
#sudo docker push 10.1.1.155:5101/homeassist:patched1a

#4 Use $TARGET_REPO_TAG in docker run or docker-compose to run with the adjusted and debugable container

Disclosure

Interpretation of the dockerfile.update was suggested and edited with aid of ChatGPT4, my human eyes, brain and fingers then modified appropriately 😁

What version of Home Assistant Core has the issue?

2024.2.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

seen in TUYA

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

System Information

version	core-2024.2.1
installation_type	Unknown
dev	false
hassio	false
docker	false
user	root
virtualenv	false
python_version	3.12.1
os_name	Linux
os_version	6.6.14-200.fc39.aarch64
arch	aarch64
timezone	Pacific/Auckland
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
GitHub API Calls Remaining	5000
Installed Version	1.34.0
Stage	running
Available Repositories	1397
Downloaded Repositories	2
HACS Data	ok

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Dashboards

dashboards	2
resources	1
views	1
mode	storage

Recorder

oldest_recorder_run	February 10, 2024 at 1:45 AM
current_recorder_run	February 15, 2024 at 1:56 PM
estimated_db_size	3.82 MiB
database_engine	sqlite
database_version	3.44.2