PlummersSoftwareLLC
Socket server: Unknown command in packed received: [some number]
Bug report
Problem
I get the error "Unknown command in packet received: 31202" when running the ledstrip example with the python audioserver.py sample script. After that the connection is closing and the server as well as the python client keep retrying to establish a connectionn. I've looked into the socket server and the command16 variable is only supposed to have the values 3 or 4, is that right? Steps
- Install python audioserver.py dependencies, setup correctly with esp32 ip.
- compile ledstrip example with correct ssid and password in secrets.h
- Upload example and run python script
Example Is something here not configured correctly?
#!/usr/bin/python
##+--------------------------------------------------------------------------
##
## audioserver - (c) 2023 Dave Plummer. All Rights Reserved.
##
## File: audioserver.py - Pushes Audio FFT data to NightDriverStrip
##
## Description:
##
## Samples the system audio, splits it into bands using an FFT, and then
## sends is over WiFi to a NightDriverStrip instance
##
## History: Feb-20-2023 davepl Created
##
##---------------------------------------------------------------------------
import pyaudio
import numpy as np
import socket
import struct
import time
import sys
# NightDriver ESP32 wifi address - update to your ESP32 WiFi
client = '192.168.178.72'
# Set up audio input stream. 512@24000 gives a nice framerate. And 512
# is what I run on the ESP32 if connected via hardware mic, so at least it matches
chunk_size = 512
sample_rate = 44100
max_freq = 20000
num_bands = 12
# Correction I apply to get a mostly linear response across the bands.
if num_bands == 16:
band_scalars = [ 0.35, 0.20, 0.125, 0.1, 0.5, 1.2, 1.7, 2.0, 2.1, 2.75, 2.0, 8.0, 8.0, 8.0, 8.0, 8.0 ]
else:
if num_bands == 12:
band_scalars = [ 1.0, 1.0, 1.0, 1.0, 0.01, 0.01, 0.01, 0.1, 0.1, 0.1, 0.1, 1.0 ]
# Open the audio stream. I'm reading from the mic here because I could not find a system independent
# way to read from the default output device, which would normally require a loopback instance be
# installed and for simplicity sake, I don't want that.
p = pyaudio.PyAudio()
#
# Set up FFT parameters:
#
fft_size = 2**12 # Choose the size of the FFT (power of 2 for efficiency)
# Calculate the frequencies corresponding to each FFT bin.
freqs = np.fft.rfftfreq(fft_size, d=1.0/sample_rate)
# Divide the frequency range into frequency bands of equal logrithmic width
# `20` is the minimum frequency of human hearing, `max_freq` is the maximum frequency of interest,
# and `num_bands` is the desired number of frequency bands.
bands = np.logspace(np.log10(20), np.log10(max_freq), num_bands+1).astype(int)
# Compute the width of each frequency band. This returns the detla between band (n, n+1)) for each band
band_widths = np.diff(bands) # Take the difference between adjacent frequency band limits
# The socket we will open to the ESP32 to send our band data to
print("Connect to " + client + "...")
sock = None
stream = p.open(format=pyaudio.paFloat32, channels=1, rate=sample_rate, input=True, frames_per_buffer=chunk_size)
# Loop to continuously sample audio and compute spectrum analyzer bands
while True:
# Connect to the socket we will be sending to if its not already connected
if sock == None:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
address = (client, 49152)
sock.connect(address)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
sock.setblocking(True);
# Read the raw audio data. We ignore overflow exceptions from not accepting every bit, it's ok if
# miss a few in betweeen samples
audio_data = np.frombuffer(stream.read(chunk_size, exception_on_overflow=False), dtype=np.float32)
# Compute the FFT to put the samples into frequency
fft_data = np.abs(np.fft.rfft(audio_data, n=fft_size))
# Compute band values
band_values = []
for i in range(len(bands)-1): # BUGBUG RANGE stops one before
band_start = np.searchsorted(freqs, bands[i])
band_stop = np.searchsorted(freqs, bands[i+1])
band_value = np.median(fft_data[band_start:band_stop])
band_values.append(band_value)
band_values = np.multiply(band_values, band_scalars)
# Scale band values to [0, 1]
band_values = np.clip(band_values, 0.000001, None) # Avoid div by zero
max_band_value = np.max(band_values) + 0.000001; # Avoid zero maximumum
if (max_band_value > 1):
scaled_values = np.divide(band_values, max_band_value)
else:
scaled_values = band_values
# Convert scaled values to ASCII bargraph
bargraphs = []
for scaled_value in scaled_values:
asterisks = "*" * int(round(scaled_value * 8))
bargraph = f"{asterisks:<8}"
bargraphs.append(bargraph)
# Print ASCII bargraphs
#breakpoint()
print(bargraphs)
sys.stdout.write('*')
# Compose and send the PEAKDATA packet to be sent to the ESP32 NightDriverStrip instance
packed_data = struct.pack('f' * len(scaled_values), *scaled_values)
command = 4
length32 = 4 * num_bands
seconds = int(time.time())
micros = time.perf_counter() - seconds
header1 = (command).to_bytes(2, byteorder='little') # Offset 0, command16
header2 = (num_bands).to_bytes(2, byteorder='little') # Offset 2, num_bands
header3 = (length32).to_bytes(4, byteorder='little') # Offset 4, length32
header4 = (seconds).to_bytes(8, byteorder='little') # Offset 8, seconds
header5 = struct.pack('d', micros) # Offset 16, micros
complete_packet = header1 + header2 + header3 + header4 + header5 + packed_data
try:
sock.send(complete_packet)
except socket.error as e:
#breakpoint()
print("Socket error!");
sock.close()
sock = None
time.sleep(0.015);
sock.close()
The globals.h section for the ledstrip demo:
#elif LEDSTRIP
// The LED strips I use for Christmas lights under my eaves
#ifndef PROJECT_NAME
#define PROJECT_NAME "Ledstrip"
#endif
#ifndef ENABLE_WEBSERVER
#define ENABLE_WEBSERVER 0 // Turn on the internal webserver
#endif
#define ENABLE_WIFI 1 // Connect to WiFi
#define INCOMING_WIFI_ENABLED 1 // Accepting incoming color data and commands
#define WAIT_FOR_WIFI 1 // Hold in setup until we have WiFi - for strips without effects
#define TIME_BEFORE_LOCAL 5 // How many seconds before the lamp times out and shows local content
#define COLORDATA_SERVER_ENABLED 1 // Also provides a response packet
#define NUM_CHANNELS 1
#define MATRIX_WIDTH (1*144) // My maximum run, and about all you can do at 30fps
#define MATRIX_HEIGHT 1
#define NUM_LEDS (MATRIX_WIDTH * MATRIX_HEIGHT)
#define ENABLE_REMOTE 0 // IR Remote Control
#define ENABLE_AUDIO 0 // Listen for audio from the microphone and process it
#ifndef LED_PIN0
#define LED_PIN0 5
#endif
#define DEFAULT_EFFECT_INTERVAL (1000*20)
#define RING_SIZE_0 1
#define RING_SIZE_1 2
#define RING_SIZE_2 4
#define RING_SIZE_3 8
#define RING_SIZE_4 16
Notes I have connected to led strips to port 5 of my esp32.
Thank you very much for your help! :)
I cannot seem to get this to work. When I do not set ENABLE_AUDIO to 1 it does not process the peak_data, right? Hence the example audioserver.py won't work? But if I do set it to to 1 it tries to read from a non existing mic. What is the correct way to continue further? Thank you in advance.
31202 is 0x79E2, so it's not like it's wrong wndian or off one byte.
I do lots of builds without enable audio set and no mic attached. Mine never spawn that adc thread or socketswrvwr. Threads or block on them.
Just debug it methodically. Does a scope show a solid sine wave when you whistle to it? Do you have the types of boards that are actually supported (I think the list is long in M1 stuff and short on "solder and $1 mic to pins X and Y". Does your audio gear and the config.h mess March? (E.g., not soldering an i2s mix to code reading analong inputs...) Does something resembling an audio wave appear in that big buffer if you graph it with gplot or Numbers or whatever? Then proceed upstream: is the stream of audio bytes. Basically sane but perhaps missing a synchronization primitive or printing an wrong thing in the debug message or something?
There's just not a lot of collected knowledge that I know of about audio debugging here.
On Sun, Nov 17, 2024, 3:02 PM Julius @.***> wrote:
I cannot seem to get this to work. When I do not set ENABLE_AUDIO to 1 it does not process the peak_data, right? Hence the example audioserver.py won't work? But if I do set it to to 1 it tries to read from a non existing mic. What is the correct way to continue further? Thank you in advance.
— Reply to this email directly, view it on GitHub https://github.com/PlummersSoftwareLLC/NightDriverStrip/issues/666#issuecomment-2481552457, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD3YDA25ENWTA5SG2JB32BD77RAVCNFSM6AAAAABRPGVV5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBRGU2TENBVG4 . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>
@robertlipe Thank you for your detailed elaboration, but my goal is to only send and receive the data via a socket to the esp32. I will try more things here and there.
I don't think the microseconds are supposed to be negative, right? Regarding
micros = time.perf_counter() - seconds in audioserver.py. My seconds and micros basically have the values 1736709405 and -1736695268.2918289.
I'm running into this error when I open a WebSocket, but a different number. It happens when running my Kotlin code or when I attempt to open a WebSocket using default settings in Postman.
Logs
(W) (ProcessIncomingConnectionsLoop)(C1) Unknown command in packet received: 17735
(W)
(W) (SocketServerTaskEntry)(C1) Socket connection closed. Retrying...
Kotlin code sample
val uri = UriBuilder.of(websocketAddress).port(clientEntity.wsPort!!).build()
val future = CompletableFuture<LedStripWebSocketClient>()
val clientPublisher = webSocketClient.connect(LedStripWebSocketClient::class.java, uri)
clientPublisher.subscribe(object : Subscriber<LedStripWebSocketClient> {
override fun onSubscribe(s: Subscription?) {
s?.request(1)
}
override fun onError(t: Throwable?) {
state = WebSocketState.DisconnectedIdle
future.completeExceptionally(t)
}
override fun onComplete() {}
override fun onNext(client: LedStripWebSocketClient?) {
state = WebSocketState.ConnectedIdle
future.complete(client)
}
})
client = future.get(5, TimeUnit.SECONDS)
@Cyborg-Squirrel You're implying a connection between setting up a web socket to listen for events coming from the ESP32, and errors being shown by the code that accepts commands and color data going into the ESP32.
I'm pretty confident that on the ESP32 side, those code paths are pretty solidly separated. That makes me believe that any errors you see being generated by the socket server code is caused by something sending bogus data to the ESP32 socket server port (49152), which may or may not be related to whatever is trying to negotiate the web socket connection with the ESP32 web server port (80).
To help confirm or reject this theory, could you provide the full source code that is involved with opening the web socket? The code you shared so far is partial, in the sense that it refers to Kotlin/Java classes for which the code is not provided.
@rbergen thank you for the quick response!
The code snippet is from this class
Here are the values stored in the clients table in the Postgres database to give you an example of the values loaded into the LedStripClientEntity used for the WebSocketJob.
The goal of the WebSocketJob class is to connect to stream color data to the ESP32.
I think the repo link you sent me points to a private repo, because GitHub won't allow me access to it.
In any case, the table you included in your last message seems to show you're trying to set up a web socket connection to port 49152. That's not going to work. In short:
- Port 49152 will accept color data (and associated commands), but only in raw network packets sent to a "raw" TCP port. There is no way to send color data to the ESP32 via a web socket.
- It is possible to set up a web socket with the ESP32 on-board web server on port 80, which will allow you to retrieve color data that the ESP32 is showing on the connected LEDs. Again, this is color data coming from the ESP32, not the other way around.
I hope this clarifies why what you're trying is not working, and won't.