khusmann
improve audio support
Current audio support is awkward because there's no good SBC encoder / decoder for Python. (So example uses pyav instead).
I'd like to get python bindings for google's libsbc -- this will need to be its own separate project...
Do you think it would be possible to run VARA FM or other digital modes over BT audio if software was written to do that? It looks like you have to mess with audio cables, add-on cards, virtual audio devices, etc to make some digital modes work. Seems with audio support in BT, this would be nice and direct without any user configs.
Do you think it would be possible to run VARA FM or other digital modes over BT audio if software was written to do that? It looks like you have to mess with audio cables, add-on cards, virtual audio devices, etc to make some digital modes work. Seems with audio support in BT, this would be nice and direct without any user configs.
@Ylianst 100% -- this is in fact how the HT app does all its digital image modes, etc. It just generates the audio clip, converts to SBC and then transmits over RFCOMM
I imagine that for APRS send/receive, it uses it's internal TNC... but yes, for the rest, it's a great way to the app to be super flexible and you can probably get better speeds.
@Ylianst yes, the app uses the internal TNC for APRS.
I imagine if you did your own APRS decoding via audio (e.g. via direwolf) you could get better decoding than whatever they're doing onboard the radio (at the cost of more power / processing on the device you're using)
You are right that full software decoding would likely be best, especially if you can add error correcting codes. Do you have an idea what is the sampling rate of the Bluetooth audio? also, is any compression applied to the BT audio link? Obviously, if the BT link is compressed, that would be a problem.
@Ylianst audio is at 32khz compressed with the SBC codec. I have a working send and receive example in benlink here: https://kylehusmann.com/benlink/benlink/audio.html check out benlink.examples.audiotransmit and benlink.examples.audiomonitor
The python code will be much simplified with libsbc bindings for python as I mention above, for now I'm using pyav for decoding. But you can get the idea...
@khusmann Ha! Got it. Yes, the SBC codec will add some noise to the digital signal for sure. This is interesting stuff, I am going to be reading up on this.
@Ylianst yes, not ideal but good enough for basic use of the image modes it seems.
I haven't experimented with it much but it did seem there was some flexibility for the radio to play different bitrates (via different bitpool settings). So you could potentially go higher than the default audio compression quality it uses.
So yeah, definitely worth experimenting with. I suppose a good first step would be to do a side-by-side comparison of direwolf decoding of signals at different SBC compression rates.
allow me to throw data modes of m17 into the mix, i.e. https://github.com/xssfox/freedvtnc2
@gretel - Oh! That is interesting! I got plenty on my hands just getting basic packet radio working in HT Commander, but this is very interesting work that will be useful when I start working on audio.
@khusmann - I am trying to get your audiomonitor.py to work, but I am not very familiar with python and when I run it, I get:
Can I get really dumb down instructions on running it? I want to try to get audio frames going in/out and needs to figure out the basics.
Ah, you need to install the package -- from the root repo dir:
pip install -e .
Then run (from anywhere):
python -m benlink.examples.audiomonitor
Thanks you, now I get this:
Ok, I needed to also do:
pip install pyaudio
pip install av
Now, I get this:
If I use 1 as the channel, I get this:
Apologies for the really basic questions.
Yup, I don't include those deps in the package level bc they're only needed to run the example. (And I'll remove pyav once I make my own python sbc bindings)
Hmm so the first error is because we can't scan the rfcomm channels to know which has the audio stream in python (pybluez is no longer maintained).
Are you sure you're using the right rfcomm channel? Its not always the first one, sometimes the second or third, try doing a sdp scan -- in linux you can use sdptool, i dunno about windows.
Also could be something about the bluetooth socket / asyncio in windows... I havent tested bt serial in python on windows... Might have a chance to look at tomorrow
I tried channels 0, 1, 2, 3... 12. It always returns the same. I would love to start experimenting with audio, lots of wild things to do.
Hmm yeah looks like an issue with windows connecting to the rfcomm socket in asycio -- do you have a linux box to try on? (Raspberry pi? Boot a liveusb?)
Another question, looking at link.py there is RfcommAudioLink and RfcommCommandLink both seem to be identify and they create a RfComm channel using the device UUID. I can't spot what would make them different? - Just curious.
Another question, looking at
link.pythere isRfcommAudioLinkandRfcommCommandLinkboth seem to be identify and they create a RfComm channel using the device UUID. I can't spot what would make them different? - Just curious.
Main difference there is AudioLink types send AudioMessages, whereas CommandLink types send benshi protocol messages (which it wraps the protocol message into a gaiaframe). Receiving is also different for the same reason I think. AudioLink and CommandLink give me then the abstract interface to use with any audio or command implementation - right now i have rfcomm and ble impl for command and rfcomm for audio, but could also compress the frames and send over tcp, or create a dummy device for testing, or whatever, and still use the same abstract interfaces.
Thanks, but when connecting, both seem to use the same device_uuid to connect, so, how does the bt stack/radio know you are connecting for audio or command?
By the way, I just got my own Windows code receiving audio now! I put channel 2 and that caused it to work. Any ideas what the audio channels are? Do I also transmit on channel 2?
Also, if you send audio to the radio, does it automatically push the PTT?
Thanks, but when connecting, both seem to use the same device_uuid to connect, so, how does the bt stack/radio know you are connecting for audio or command?
They run on separate rfcomm channels on the same uuid. (Basically think of each rfcomm channel as a serial port you can connect to) You can use sdptool in linux to see which is channel is for which, not sure how in windows. (The command channel is labeled SPP, audio is labeled something else)
By the way, I just got my own Windows code receiving audio now! I put channel 2 and that caused it to work. Any ideas what the audio channels are? Do I also transmit on channel 2?
Nice! If channel 2 is your audio rfcomm, then everything happens there, both transmit and receive.
Also, if you send audio to the radio, does it automatically push the PTT?
Sort of -- there are different audio frame types that indicate when you're about to send audio packets and when you are done so it can turn off the ptt, but some seemed to be optional. There was also a frame type that I didnt figure out... Anyway, it should all be there in the transmit / receive examples.
@khusmann Ha. Thank you. So, right, the channel number is where the magic is. My next step is going to be to bind to a SBC codec and see if I can play audio in real-time.
So, you have BleCommandLink and RfcommCommandLink, what is the difference between the two? I currently use the BleCommandLink but to use audio, I have to switch from using a BluetoothLE to a classic Bluetooth library. Can I do everything I was doing on LE using classic? Do BleCommandLink and RfcommCommandLink handle the same commands?
The difference between ble and rfcomm commands is that when youre sending commands over rfcomm you wrap your command in a gaiaframe. (Look for the gaiaframe definition in the protocol folder to see its structure). So its all the same commands, just one more layer of encapsulation.
That said you dont need to switch to rfcomm for your commands. You can still use BLE for commands while simultaneously using rfcomm for audio... No issue there.
Nice. I was able to fully decode and listen to the radio in real-time and it's working well. I did notice that I occasionally have a 0x03 command in the audio stream. It looks like it's also an audio frame, so I decode it just like a 0x00 frame, but would be interesting to know what the difference is.
Also, for frame 0x01 (Audio End), it's 9 bytes long, I am looking to see what is in that. I never got a 0x02 (Audio Ack), if you know what that is, let me know. I will work on optimizing the code more and add it to HT Commander this week.
Nice work! Sounds you're up to where I left off - I'd also like to know the significance of 0x03 and the body of the 0x01 (audio end). (Let me know if you find anything)
0x02 I can explain though -- the HT app sends that in response to each received audio frame, but it appeared to be optional in my testing - everything seemed to work fine without it.
Thank you so much @khusmann. I would have never guessed for 0x02, very interesting. So now, I need to work on some visualization and doing to transmit path. I feel the most difficult is over now. In looking at the SBC codec and the frequency response, it does seem like it's going to have an impact on any soft-modem attached to the radio in the future, however, we will see.
Glad I could help you get up to speed! That's been my guiding goal for the benlink project, to reverse engineer these protocols and setup so folks like you can run with it.
Looking forward to see where you'll take this!
@khusmann - Interesting. I started adding audio support to HT Commander and noticed that if I connect BTLE channel first, I can't connect the audio channel anymore. But if I do audio first, then BTLE still works. This may be a consequence of the libraries I use and not the radio, but I did want to ask if you encountered such situation? I am thinking of switching the data channel over to BT Classic and see what happens.
Also, for command frames, is it channel id 1? With audio being 2?