Advanced energy scan

[puddly]

To run it:

# Collect the data
pip install 'git+https://github.com/puddly/zigpy-cli@puddly/advanced-energy-scan'
zigpy radio ezsp /dev/cu.SLAB_USBtoUART advanced-energy-scan \
    --num-energy-scans 1000 \
    --num-network-scans 20 \
    advanced-energy-scan.json

# Plot it with the script below
pip install pandas matplotlib
python plot.py advanced-energy-scan.json

---

Inspired by https://github.com/zigpy/zha/issues/51.

I'm thinking of taking a slightly different approach to pick channels. Instead of performing a "long" scan with a high exponent, we instead perform many short scans. The theory is that the one long scan is just max(short_scans) for the same time period so we're just discarding data.

This allows for a much more granular view of the spectrum that can be combined with a beacon scan to identify real noise as opposed to a Zigbee router right next to the coordinator:

Script to generate the above plot:

import sys
import json
import pathlib
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.patches as patches

path = pathlib.Path(sys.argv[1])
data = json.loads(path.read_text())

current_channel = data["current_channel"]
energy_scan = pd.DataFrame(data=data["energy_scan"])
energy_scan.timestamp = pd.to_datetime(energy_scan.timestamp, unit="s")

network_scan = pd.DataFrame(data["network_scan"])

if energy_scan.rssi.isna().any():
    # LQI only scan
    vmin = 0
    vmax = 255
    energy_scan["scan"] = energy_scan.energy
    scan_type = "lqi"
else:
    # Scan with RSSI
    vmin = -100
    vmax = 10
    energy_scan["scan"] = energy_scan.rssi
    scan_type = "rssi"

# Define the colormap for histograms
plt.rcParams["figure.dpi"] = 50
energy_cmap = plt.get_cmap("viridis")
network_cmap = plt.get_cmap("hsv")
norm = plt.Normalize(vmin=vmin, vmax=vmax)

# Set up the plot
num_channels = len(energy_scan.channel.drop_duplicates().sort_values().to_list())
fig, axes = plt.subplots(nrows=num_channels, figsize=(10, 20), sharex=True)
fig.suptitle(
    (
        "Energy Distribution Across 802.15.4 Channels"
        + f" ({len(energy_scan) // num_channels} samples)\n"
        + path.name
    ),
    fontsize=20,
)

for ax, ((channel,), scans) in zip(axes, energy_scan.groupby(["channel"])):
    counts, bins, hist_patches = ax.hist(
        scans["scan"],
        bins=100,
        range=(vmin, vmax),
        color="black",
        edgecolor="black",
    )

    for patch, bin_left in zip(hist_patches, bins):
        patch.set_facecolor(energy_cmap(norm(bin_left)))

    networks = None

    if not network_scan.empty:
        networks = network_scan[network_scan.channel == channel].copy()

        # Draw vertical lines according to each network beacon, colored by extended_pan_id
        if not networks.empty:
            network_indexes = pd.factorize(networks.extended_pan_id)[0]
            networks["color"] = network_indexes / (network_indexes.max() + 1)

            for _, network in networks.iterrows():
                ax.axvline(
                    (network.rssi if scan_type == "rssi" else network.lqi),
                    color=network_cmap(network.color),
                    linestyle="dotted",
                    linewidth=3,
                    alpha=0.5,
                    zorder=0,
                )

    title = "$\\bf{" + f"Channel\\ {channel}" + "}$"

    if networks is not None and not networks.empty:
        title += f"\n{len(networks)} networks"

    ax.set_frame_on(False)

    ax.set_ylabel(title, rotation=0)
    ax.set_ylim(0, None)
    ax.set_xlim(vmin, vmax)

    ax.yaxis.set_label_coords(-0.1, 0.4)
    ax.yaxis.set_ticks([])

    # Show the x axis on the last channel
    if channel == 26:
        ax.set_xlabel("RSSI (dBm)" if scan_type == "rssi" else "LQI")
    else:
        ax.xaxis.set_visible(False)

    # Highlight the plot for the current channel
    if channel == current_channel:
        ax.set_facecolor("lightblue")

plt.tight_layout()
plt.savefig(f"{path.stem}.png", dpi=200)
plt.show()

[Hedda]

Inspired by zigpy/zha#51

Suggest also check out ideas discussed in this related forum thread -> https://community.home-assistant.io/t/i-tracked-channel-utilization-with-zha-to-find-the-best-zigbee-channel/656139

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 0% with 52 lines in your changes missing coverage. Please review.

Project coverage is 2.02%. Comparing base (ab58a1b) to head (ebc953d). Report is 4 commits behind head on dev.

Files with missing lines	Patch %	Lines
zigpy_cli/radio.py	0.00%	52 Missing :warning:

Additional details and impacted files

@@           Coverage Diff            @@
##             dev     #49      +/-   ##
========================================
- Coverage   2.23%   2.02%   -0.21%     
========================================
  Files          8       8              
  Lines        492     543      +51     
========================================
  Hits          11      11              
- Misses       481     532      +51     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Hedda]

For more UI inspiration check out this zigpy-cli based community member built real-time Zigbee Channel Monitoring shell script:

• https://community.home-assistant.io/t/realtime-zigbee-channel-monitoring/691737

He has written a shell script that uses a separate Zigbee Coordinator and existing energy-scan in zigpy-cli to pass data into awk to format the output as json and then stores it using a template (sensor) that executes his shell script and stores the return values for each zigbee channel as an attribute. For ease of graphing a template sensor for each channel is then created. All that enable continuous long-term monitoring to produce a bar graph using a custom bar-card for Home Assistant dashboard:

"This produces the bar graph shown before that gives you readings for each zigbee channel updated every 10 seconds. You can access each channels historical data by clicking on the icon at the bottom of each channel."