esp-idf-svc
esp-idf-svc copied to clipboard
esp-rs
ESP Modem for Rust (LTE / GSM / Sim Card)
The ESP-Modem Component is not supported.
It would enable SMS+GSM +LTE connections to work with the current Netif interface.
I have started calling into the the esp-modem component, and figured it would be good to wrap this up for use in esp-idf-svc.
The code in [netif.rs](https://github.com/esp-rs/esp-idf-svc/blob/bf47d2bcee6e1bc71530a23264eae63a7af84aae/src/netif.rs#L34C5-L36C9) suggests support for PPP mode, but currently there are no ways to create that interface.
I am happy to start development, but I think this may be a pretty big project and would appreciate some help fitting this into current components. The official esp-modem component is quite chunky: https://github.com/espressif/esp-protocols/tree/master/components/esp_modem.
Unfortunately I haven't played with these modems yet, so below my very limited understanding:
- First of all, you need to talk to the modem via some peripheral. Most often than not, that's via UART, so your future modem (let's call the Rust wrapper of it for now
EspModem) needs to take aUartperipheral and TX+RX pins and/or an instantiatedUartDriver. Probably the former, as you are wrapping a native component which would like to take raw pins and UART number (i.e. a "peripheral") - If communication via other protocols than UART is also supported, we need to support it in future as well, but I don't see that in the C code
- Seems that
EspModemwould need to have a public API for configuring the modem (as in LTE/GSM pin code, APN etc.); I assume the C esp modem exposes a similar API that you need to wrap, that talks to the modem over UART with "AT" commands - Once the modem is configured successfully and the LTE/GSM connection is up, I guess you need to somehow (with an API/AT command) switch it to "data" mode, where you can start sending/receiving PPP packets over UART. How exactly this is done is unclear to me, and you probably need to check the ESP Modem C examples
- Also unclear to me how to switch back to AT commands mode
Probably this is a start?
Also, are you only interested in modems for internet connectivity? I assume modems can also be used for simple SMS send/receive? Not sure how calling / receiving voice calls work over these, if supported at all... Anyway, these are probably corner cases...
Yes the project is large-ish, but maybe not that much after all...
Oh. One more thing but maybe important - given that ALL communication with the modem is basically over a UART channel (sending and receiving bytes of data) - is there any pure-Rust crate that implements this? Even if this crate (obivously) does not talk to ESP Netif's PPP out of the box, it can be made to do so.
I'm mentioning this because if there exists a well-supported Rust crate that does the AT comm with the modem, it might be easier and cleaner to integrate with it instead, rather than going down the ESP IDF C route again.
Also this is interesting. Given that @diondokter is active in the Embassy Matrix channel, why don't you ping him in there? Given his library, I'm sure he had dealt with a modem or two and their pesky AT-command interface. :)
@diondokter Sorry maybe even better to also ping you here (as I actually already did, from above):
The TL;DR is - we are contemplating wrapping in Rust an ESP IDF C library called "ESP Modem" that does the heavy lifting of configuring a modem via AT commands so that eventually it is switched to PPP at the end (this always happens, right?) and then once the UART channel is in "PPP mode" so to say, we we can further hook it up with the ESP IDF networking layer (this is not embassy-net but this is not really important) so that we can do IP networking.
However - this all might be unnecessary, and might just bring more unsafe C code and a lot of work for us, I fear.
- Given that we already have a UART HAL driver (of course)
- We'll surely figure out how to "plug" the PPP stream flowing on top of UART into ESP IDF so that its networking stack powers up
... the one remaining challenge is how to configure the modem via AT commands. Given your crate I referenced above, you've surely dealt with this in the past. Do you have any link / example code how this is typically done? (Ideally, with your "at-commands" crate :) ?
If the example is with - say - embassy-net + embassy-net-ppp we can still understand that, and we'll translate it all to the ESP IDF context. It is the "AT part" which is interesting to us...
Hi, it really depends on your modem! I used the at-commands crate in https://github.com/diondokter/nrf-modem which is a library that wraps the Nordic libmodem C library. I believe you can do the thing you propose here where you can setup the modem and then drop into a uart stream for TCP data or something lower level.
With nrf-modem however, there are actual APIs that you can use. So a socket is just a socket and that doesn't use the at commands. At-commands are used only for things where no API is available.
Example for building a command: https://github.com/diondokter/nrf-modem/blob/ec738c3762974bfd3559becdc785907cafff07e4/src/lib.rs#L305-L314 Example for parsing a command: https://github.com/diondokter/nrf-modem/blob/ec738c3762974bfd3559becdc785907cafff07e4/src/lib.rs#L178-L184
I'm not familiar with the ESP modem though, so the only tip I can give is to make your interface to the modem nice to use. Modems are already notoriously frustrating at times, so don't add to that with a cumbersome interface. I'm not quite sure what the embassy-net-ppp interface is like, but I'd try to make the modem device struct be able to split into two channels:
- An AT command channel
- The raw PPP channel
The AT channel you can then use to setup the modem and query it for information. The PPP channel can then be handed to the embassy-net-ppp library. The channels themselves then coordinate the state of the uart and the switching of modes of the modem.
Not sure what else I can add... But feel free to ask anything
Not sure what else I can add... But feel free to ask anything
That was already plenty, thanks a lot!
Regarding PPP, this is just a way to tunnel IP traffic (IP packets) over a point-to-point connection, which in our case would be between the host (The ESP MCU) and the modem itself (which is whatever modem is connected over UART to the ESP).
I was imagining that's all there is to it (1- AT commands to setup the modem and then 2- a PPP channel for encapsulating the IP traffic/packets)... but then, looking at your Nordic modem wrapper, you have a lot of code in there, including stuff like TCP socket struct, UDP socket struct and whatnot.
... which brings the question... Why is any of that necessary actually? If indeed the IP traffic between the modem and the host (ESP or whatever it is, could be even Linux) is just flowing over a PPP tunnel, then you can use any IP/TCP/UDP library that can packetize/depacketize IP over PPP. As in:
embassy-net(or evensmoltcpbarebones)LwIP(the one used in ESP IDF)- Linux
- etc. etc.
... so in a way you won't need to model your own TCP or UDP socket. That would come from whatever IP/TCP/UDP stack library you choose to use over the PPP tunnel...? So where is the complexity in your case coming from?
I was imagining that's all there is to it (1- AT commands to setup the modem and then 2- a PPP channel for encapsulating the IP traffic/packets)... but then, looking at your Nordic modem wrapper, you have a lot of code in there, including stuff like TCP socket struct, UDP socket struct and whatnot.
In your case that is indeed all there is to it.
... which brings the question... Why is any of that necessary actually? If indeed the IP traffic between the modem and the host (ESP or whatever it is, could be even Linux) is just flowing over a PPP tunnel, then you can use any IP/TCP/UDP library that can packetize/depacketize IP over PPP.
Yep, you're right
... so in a way you won't need to model your own TCP or UDP socket. That would come from whatever IP/TCP/UDP stack library you choose to use over the PPP tunnel...? So where is the complexity in your case coming from?
While it's possible to use PPP on the nRF91, it's not needed. It provides its own socket APIs with which you can do TCP, UDP and more. This TCP/IP stack then runs on the modem itself. This has two advantages:
- The modem core is already running, so why not offload the compute to it?
- The modem has its own flash memory, so you get the TCP/IP stack for free.
If I ran the TCP/IP on the host core, then that takes away compute and it makes the firmware bigger. There still might be good reason to want to run your own stack, for example when the built-in stack is buggy or lacks some feature, but I've seen no issue in using the existing socket APIs.
For you though, you don't seem to have a choice since the ESP modem doesn't list anything about a built-in TCP/IP stack.
@diondokter Crystal clear, thanks a ton!
@DaneSlattery I am somehow leaning towards seriously evaluating first if we can implement the modem in pure Rust. (I'm already regretting a bit that for the one-wire we did not instead expose the new RMT driver in Rust and then just write on top a pure-Rust one-wire impl that can simply operate on top of two APIs: send_bit and recv_bit. As that would've allowed us to easily re-target the pure Rust code to operate over - say - UART, as you can do send_bit / recv_bit over UART as well.)
Also in general, my feeling w.r.t. ESP IDF is that:
- The lower-level code works OK (as in drivers, the IP stack and so on). It also gets a lot of banging as ESP IDF is very popular and the base stuff is used a lot in the open
- But the higher you climb the stack... maybe better to implement in Rust itself because its equivalent in C (if it exists) is either usually not used so much because it is... higher level and thus more of niche use case OR is often not a good fit for Rust. Hence why for example I'm pushing
edge-netas an alternative for the built-in ESP-IDF app-level networking protocols (HTTP client, server, mDNS etc.)
@ivmarkov I agree that the onewire rmt approach could be based on this API, and I would like to get to deprecating the IDF-v4 RMT driver in favour of IDF-v5, that would have simplified my approach. That said, working code is better than analysis paralysis 👍 , and I wanted this feature for my own projects. Nothing wrong with refactoring later. (I do wonder if this project has an overall direction and who is steering the wheel, would love to hear more about that working group).
As for a pure-rust implementation, I think given the information:
- The existing
esp-idf-halUART is a suitable base - AT commands can be written in pure rust in a similar fashion to this: https://github.com/espressif/esp-protocols/blob/master/components/esp_modem/include/generate/esp_modem_command_declare.inc . I think even a small subset of these can be supported. I would use
https://github.com/diondokter/at-commandsas a base - I would ignore cmux and vfs support for now (mostly because I can't test them, and they seem like an advanced edge case).
- PPP Mode should probably be based on the idf-lwip stack for now, although I agree that
edge-netis a really solid library.
@ivmarkov I agree that the
onewirermt approach could be based on this API, and I would like to get to deprecating theIDF-v4RMT driver in favour ofIDF-v5, that would have simplified my approach. That said, working code is better than analysis paralysis 👍 , and I wanted this feature for my own projects. Nothing wrong with refactoring later.
If I recall correctly, the approach where the new RMT driver needed to be exposed was simply deemed much more effort to implement by you. Hence why we ended up where we are. Which is OK, we still did a good progress! But we definitely did not have analysis-paralysis, I believe.
(I do wonder if this project has an overall direction and who is steering the wheel, would love to hear more about that working group).
There is no "steering committee" of sorts, because Espressif is taking a rather opportunistic / tactical approach towards its Rust-related portfolio. This is also valid for the bare-metal crates which do have paid developers working on those.
esp-idf-* is a community effort, and I guess I'm the most active one here, who historically contributed the largest portion of the code, but we also now have @Vollbrecht as an active committer and a few (now inactive) committers who contributed large portions as well (as in the esp-idf-sys "native" build and components that you are using - these originally came from @N3xed).
Do I miss a more "architectural" / "bigger-horizon" approach which can be discussed among more community members? I certainly do, but this requires a lot of prior exposure to the esp-idf-* crates (a huge time investment) and in the absence of Espressif steering it, it is what it is - i.e. the ones who sticked around for the longnest (and are vocal) tend to define the direction.
As in, there are bi-weekly meetings of the esp-rs community (this include both bare-metal and ESP-IDF crates) which are open for everyone that you might want to attend. But I fear these are more of a "sprint ceremony" "what was done last sprint" kind of a thing - primarily for the bare-metal team, rather than a forum where direction is being discussed. We can change that of course, but to change it, there must be more people that feel the change is useful and necessary, and I feel I'm in the minority there. :)
Anyway, I digress.
As for a pure-rust implementation, I think given the information:
- The existing
esp-idf-halUART is a suitable base- AT commands can be written in pure rust in a similar fashion to this: https://github.com/espressif/esp-protocols/blob/master/components/esp_modem/include/generate/esp_modem_command_declare.inc . I think even a small subset of these can be supported. I would use
https://github.com/diondokter/at-commandsas a base- I would ignore cmux and vfs support for now (mostly because I can't test them, and they seem like an advanced edge case).
Can you elaborate what cmux and vfs support in the ESP Modem C driver is about, if you've looked into it?
- PPP Mode should probably be based on the idf-lwip stack for now, although I agree that
edge-netis a really solid library.
I did not mention edge-net as an alternative to the ifd-lwip stack as these are two different animals:
edge-netis "application" protocols that sit above IP/TCP/UDP, like http server and client, dhcp server and client, mdns responder and so on. It is just I have found out that the native "C" ESP IDF http, wensockets, mdns etc. services to be suboptimal in certain aspectslwip(and esp-netif) is your IP(/TCP/UDP) stack and is the right place we need to plug the esp modem
Thank you for the detail on the management structures. It is interesting to see. It seems like no_std is a key focus for espressif, and I wonder at what point it will supercede the rust-idf approach.
Here's what I have on cmux, which seems useful for keeping the UART in AT and PPP mode simultaneously. https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/applications/serial_lte_modem/doc/CMUX_AT_commands.html
And VFS:
/* The VFS terminal is just a demonstration of using an abstract file descriptor
* which implements non-block reads, writes and selects to communicate with esp-modem.
* This configuration uses the same UART driver as the terminal created by create_uart_dte(),
* so doesn't give any practical benefit besides the FD use demonstration and a placeholder
* to use FD terminal for other devices
*/
Here are some more relevant crates:
- https://github.com/embassy-rs/ppproto/tree/main
- https://github.com/FactbirdHQ/atat/tree/master (usage here : https://github.com/atlas-aero/rt-esp-at-nal/blob/main/src/commands.rs) going to start here.
- https://github.com/technocreatives/sim7000/blob/main/samples/nrf52840/src/main.rs
Thank you for the detail on the management structures. It is interesting to see. It seems like
no_stdis a key focus for espressif, and I wonder at what point it will supercede the rust-idf approach.
It is. The ESP-IDF is viewed as a stop-gap solution, and obviously the solution if you already have a large C codebase that you can't or don't want to migrate - in one go or at all.
With that said, it's been like that for many years.
What I'm personally doing is trying to abstract from the underlying "OS" as much as possible - with e-hal, or by rolling your own traits etc. And by using async IO as much as possible as no other embedded platform besides esp-idf-* has threads (and STD). So that I can migrate between ESP-IDF and baremetal, or even to other MCUs.
Hence - the more code we have in Rust itself - the better for everyone!
Here's what I have on cmux, which seems useful for keeping the UART in AT and PPP mode simultaneously. https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/applications/serial_lte_modem/doc/CMUX_AT_commands.html
And VFS: /* The VFS terminal is just a demonstration of using an abstract file descriptor * which implements non-block reads, writes and selects to communicate with esp-modem. * This configuration uses the same UART driver as the terminal created by
create_uart_dte(), * so doesn't give any practical benefit besides the FD use demonstration and a placeholder * to use FD terminal for other devices */
Agreed that none of these is essential. Even CMUX - at least initially.
Here are some more relevant crates:
- https://github.com/embassy-rs/ppproto/tree/main
Yes this is the PPP impl for embassy. If your AT-commands code is in pure Rust (you can even abstract the UART by using just the async IO Read/Write traits), we can even write connectors for your code to embassy-PPP.
https://github.com/FactbirdHQ/atat/tree/master (usage here : https://github.com/atlas-aero/rt-esp-at-nal/blob/main/src/commands.rs) going to start here.
Sounds like a plan!
See, if everything goes the way we hope it goes, we might not even have to PR anything in esp-idf-svc. It might turn out that your code can live as an external crate, and just needs 3 lines of code or so for the PPP glue!
I have started writing out some AT commands over a UART device and getting replies. I will borrow a few commands from https://github.com/technocreatives/sim7000/tree/main but implement them using the atat traits.
I think it makes sense to impl atat::blocking::AtatClient and atat::asynch::AtatClient for the esp-idf-hal uart.
I do think that whatever code we write here should also target something that impls embedded_io::Read + Write , that way it can be quite general.
What I am not sure of is how to glue the modem when it is in ppp mode to a netif.
The docs say the general flow is:
Create a DCE instance (using esp_modem_new())
Call specific functions to issue AT commands (Modem commands)
Switch to the data mode (using esp_modem_set_mode())
Perform desired network operations (using standard networking API, unrelated to ESP-MODEM)
Optionally switch back to command mode (again esp_modem_set_mode())
Destroy the DCE handle (sing esp_modem_destroy())
I see this in the c-api: and this)
I have started writing out some AT commands over a UART device and getting replies. I will borrow a few commands from
https://github.com/technocreatives/sim7000/tree/mainbut implement them using theatattraits.
Great progress!
I think it makes sense to impl atat::blocking::AtatClient and atat::asynch::AtatClient for the
esp-idf-haluart.
I don't think this would be necessary, because the machinery around AtatClient (both blocking and async) just expects you pass in something which does implement embedded_io::Read + embedded_io::Write (for the blocking case) or embedded_io_async::Read + embedded_io_async::Write (for the async case). And the key is, that UartDriver already implements embedded_io::Read + embedded_io::Write, while AsyncUartDriver already implements embedded_io_async::Read + embedded_io_async::Write.
With that said, your mileage may vary, and we have to see how it is shaping up. Two reasons for that:
-
The blocking
AtatClient's "reading" seems overly complicated, with stuff like ingress and whatnot. I suspect this is because it was created with baremetal "blocking" UART implementations, where the "read" call is actually NOT blocking but also not "async". It is something, where if you try to read and there are no bytes to read, it returnsErr(nb::WouldBlock), and then you have to somehow re-try. However,UartClient::readis already true blocking. In other words, if there are currently no incoming bytes, it will IO-wait inside the "read" method until bytes arrive. It will never returnErr(nb::WouldBlock). With ESP-IDF this is possible - as unlike baremetal - ESP-IDF runs a true preemptive task scheduler and thus tasks/threads which wait on IO are just not scheduled for execution, until "the bytes arrive" so to say. -
Note how in contrast, the async
AtatClientjust usesembedded_io_async::Readwithout any "ingress" complexities. This is because async allows even baremetal code to "wait" on IO (asynchronously). And BTW that's why Embassy (in its widest interpretation which is "EMBedded ASSYnc" - async-for-embedded) is such a big deal - as it allows - with async - to use normal, "sequential looking" code for IO, just like if you had a regular preemptive multithreading task scheduler underneath (like FreeRTOS) except you don't have one and you don't need one. However, the implementation of the async write part ofAsyncUartDriveris currently cheating a bit (because the underlying ESP IDF UART does not allow for a "true" async), so we'll see whether the existing cheat would be good enough, or we have to fix something.
What I am not sure of is how to glue the modem when it is in
pppmode to a netif. The docs say the general flow is:Create a DCE instance (using esp_modem_new()) Call specific functions to issue AT commands (Modem commands) Switch to the data mode (using esp_modem_set_mode())
It should boil down to the following:
- Once you send
ATD*99#and you get an "ok" reply from the modem, you should assume that what is now flowing over UART read/write immediately after that is PPP traffic, not AT-commands text traffic - So after the above ^^^ had happened, you need to:
- Instantiate a new
EspNetifof type PPP withEspNetif::new_with_conf(&NetifConfiguration::ppp_client())(for now, to keep it simple). - Then the non-trivial stuff happens, as we need to follow the ESP modem C code and we'll do some unsafe calls into
esp_idf_sysAPIs as we don't have safe wrappers for these (yet). The "non-trivial stuff" is essentially attaching to your just-created PPP netif a netif "driver", which seems to be nothing else but a way to instruct your PPP netif what to call when it needs to send/transmit data. (for receiving, we'll just manually callesp_netif_receive(<your-ppp-netif-handle>)to "ingest" data in the ppp-netif) - The key seems to be to call
esp_netif_attachon your ppp-netif, and supply to this call also a prepared C struct which describes your "driver" - the C struct for the driver is this. - Without going into further level of details, as I might mess it up without actually implementing it, you need to translate to Rust three things:
- The code in this constructor
- The code in this "post-attach" function, which the esp-idf PPP netif call will calback for you, as you pass a reference to this function when you call
esp_netif_attach. This will take care of sending data from the netif into the UART. - For receiving data from UART and ingesting it into the netif, you need to spin a
loop {}which callsUartDriver::readand then callsesp_netif_receive(I think)
- Instantiate a new
Perform desired network operations (using standard networking API, unrelated to ESP-MODEM)
As per above, once you do the above ^^^ then you have a netif (network interface) which is attached to the ESP IDF LwIP stack and to/from which network traffic is routed. So you can use regular Rust STD API to open sockets and whatnot. That is, as long as you keep the netif instance and the driver attached to it alive.
Optionally switch back to command mode (again esp_modem_set_mode())
Let's wait with this.
Destroy the DCE handle (sing esp_modem_destroy())
I suggest not to use the ESP Modem component for anything but inspiration (copy-paste) of C code and then translation of that code to Rust unsafe calls into esp-idf-sys. Let's not think for now how to destroy our "netif" instance and the driver which is attached into it. We'll get there.
Yes, exactly! This is from where you need to "copy-paste" code into Rust. :)
By the way... (and no offense to the atat crate), but you might have a better luck and easier life by using the lower-level crate described here.
After all, all that you need is a utility, that converts an AT command to a sequence of bytes that you can then send via UartDriver::write (= embedded_io::Write) OR AsyncUartDriver::write (= embedded_io_async::Write) if you are using async; and the other way around - a utility that converts an sequence of bytes to an AT command response, where the sequence of bytes is coming from UartDriver::read (= embedded_io::Read) OR AsyncUartDriver::read (= embedded_io_async::Read) if you are using async.
All of these extra layers of abstractions in atat would either help you, or get in the way, especially considering the complicated "ingress" thing which you don't need and this is really a thing of the past: :D
- Baremetal just uses async these days anyway, so the "ingress" thing is not necessary
- With ESP IDF, you have a true blocking read and write, so the "ingress" thing is also not necessary. And with ESP IDF you can still use async read/write
So what is the point?
By the way... (and no offense to the atat crate), but you might have a better luck and easier life by using the lower-level crate described https://github.com/esp-rs/esp-idf-svc/issues/468#issuecomment-2282004218.
One thing I really like from atat are the atat::AtatCmd and atat::AtatResp traits and their derive macros. This allows for more "static" definitions of the commands, rather than building them on the fly.
for example:
/// 4.1 Manufacturer identification +CGMI
///
/// Text string identifying the manufacturer.
#[derive(Clone, AtatCmd)]
#[at_cmd("+CGMI", ManufacturerId)]
pub struct GetManufacturerId;
/// 4.1 Manufacturer identification
/// Text string identifying the manufacturer.
#[derive(Clone, Debug, AtatResp)]
pub struct ManufacturerId {
pub id: String<64>,
}
impl<'d, T> EspModem<'d, T>
where
T: embedded_svc::io::Read + embedded_svc::io::Write,
{
pub fn new(serial: &'d mut T) -> Self {
Self {
serial,
_d: PhantomData,
}
}
pub fn send_cmd<CMD: AtatCmd>(&mut self, cmd: &CMD) -> Result<CMD::Response, atat::Error> {
let mut buff = [0u8; 64];
// flush the channel
while self
.serial
.read(&mut buff)
.map_err(|_err| atat::Error::Read)?
> 0
{}
// write the command to the uart
let len = cmd.write(&mut buff);
self.serial
.write(&buff[..len])
.map_err(|_err| atat::Error::Write)?;
// now read the uart to get the response
let len = self
.serial
.read(&mut buff)
.map_err(|_err| atat::Error::Read)?;
cmd.parse(Ok(&buff[..len]))
}
}
This gives us strong typing of the commands and responses, but the trait is a bit bloated:
pub trait AtatCmd {
/// The type of the response. Must implement the `AtatResp` trait.
type Response: AtatResp;
/// The size of the buffer required to write the request.
const MAX_LEN: usize;
/// Whether or not this command can be aborted.
const CAN_ABORT: bool = false;
/// The max timeout in milliseconds.
const MAX_TIMEOUT_MS: u32 = 1000;
/// The max number of times to attempt a command with automatic retries if
/// using `send_retry`.
const ATTEMPTS: u8 = 1;
/// Whether or not to reattempt a command on a parse error
/// using `send_retry`.
const REATTEMPT_ON_PARSE_ERR: bool = true;
/// Force client to look for a response.
/// Empty slice is then passed to parse by client.
/// Implemented to enhance expandability of ATAT
const EXPECTS_RESPONSE_CODE: bool = true;
/// Write the command and return the number of written bytes.
fn write(&self, buf: &mut [u8]) -> usize;
/// Parse the response into a `Self::Response` or `Error` instance.
fn parse(&self, resp: Result<&[u8], InternalError>) -> Result<Self::Response, Error>;
}
I feel like the at-commands library could perhaps get it's own traits.
Kept lurking...
Probably not the place to discuss it, but the at-commands doesn't do anything with the transport of the commands.
I'm not sure what it would need a trait for? Or are you saying it should model the transport layer?
I'd be open to make it use the embedded-io read and write traits instead of operating on a buffer directly. That could also help with hooking it up to a uart stream.
Beauty is in the eye of the beholder, I guess, because I like this builder pattern much more. You can just call it from your modem code, and it will just serialize the commands directly in the buff you supply. Incrementally. No intermediate (potentially large-ish) objects, no nothing.
But it is you who are implementing it, so it is your choice of course.
Kept lurking...
Probably not the place to discuss it, but the
at-commandsdoesn't do anything with the transport of the commands. I'm not sure what it would need a trait for? Or are you saying it should model the transport layer?I'd be open to make it use the
embedded-ioread and write traits instead of operating on a buffer directly. That could also help with hooking it up to a uart stream.
I - personally - like the at-commands crate as-is. I.e. it does not deal with IO. Hooking it with embedded_io(_async) is so trivial, that it is not worth it to be part of the crate. After all, it is just 2-3 lines of code.
Now, if communicating with the AT modem requires a complex re-try logic, then it might make sense, as this can't be modeled without introducing embedded_io(_async) into the picture, but I don't know whether that's true or not. atat does seem to model some sort of retry logic with each command...
I have worked it for the day and come up with an idea for the at-commands @diondokter. The builders are actually quite ergonomic.
The traits I was talking about was not about the transport layer, more along these lines. I like the fact that commands and their responses are linked in atat, so perhaps a higher level API can wrap them:
pub trait Cmd {
/// The type of the response. Must implement the `Resp` trait.
type Response: Resp;
/// Write the command and return the number of written bytes.
fn write(&self, buf: &mut [u8]) -> usize;
/// Parse the response into a `Self::Response` or `Error` instance.
fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error>;
}
pub trait Resp {}
pub struct GetSignalQuality;
pub struct SignalQuality{
ber: i32,
rssi: i32,
};
impl Resp for SignalQuality {}
impl Cmd for GetSignalQuality {
type Response = SignalQuality;
fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error> {
let (rssi, ber) = CommandParser::parse(&buff[..len])
.expect_identifier(b"\r\n+CSQ: ")
.expect_int_parameter()
.expect_int_parameter()
.expect_identifier(b"\r\n\r\nOK\r\n")
.finish()
.unwrap();
Ok(SignalQuality{rssi,ber})
}
fn write(&self, buf: &mut [u8]) -> usize {
CommandBuilder::create_execute(&mut buf, true)
.named("+CSQ")
.finish()
.unwrap();
}
}
The part I seem to be after is more const functions, especially for builders. I don't think I should build a new signal quality command every time if the named value never changes. The command I build then is also not so easily shared with another device that might have the same command name but a different terminator (serde could help with this).
It might also be interesting to have the ability to configure delimiters for parsers and builders. For example, every response on the simcom7600 series replies with \r\n<REPLY>\r\n\r\nOK\r\n, so I expect that identifier with every call, but I must write every parser to expect that.
It might also be interesting to support a serde style serializer/deserializer with the above traits to account for more complex commands/responses.
That said, none of this is really hindering me. I will create a PR for this so that we may see some code
It should boil down to the following:
Once you send ATD*99# and you get an "ok" reply from the modem, you should assume that what is now flowing over UART read/write immediately after that is PPP traffic, not AT-commands text traffic So after the above ^^^ had happened, you need to: Instantiate a new EspNetif of type PPP with EspNetif::new_with_conf(&NetifConfiguration::ppp_client()) (for now, to keep it simple). Then the non-trivial stuff happens, as we need to follow the ESP modem C code and we'll do some unsafe calls into esp_idf_sys APIs as we don't have safe wrappers for these (yet). The "non-trivial stuff" is essentially attaching to your just-created PPP netif a netif "driver", which seems to be nothing else but a way to instruct your PPP netif what to call when it needs to send/transmit data. (for receiving, we'll just manually call esp_netif_receive(
) to "ingest" data in the ppp-netif) The key seems to be to call esp_netif_attach on your ppp-netif, and supply to this call also a prepared C struct which describes your "driver" - the C struct for the driver is this. Without going into further level of details, as I might mess it up without actually implementing it, you need to translate to Rust three things: The code in this constructor The code in this "post-attach" function, which the esp-idf PPP netif call will calback for you, as you pass a reference to this function when you call esp_netif_attach. This will take care of sending data from the netif into the UART. For receiving data from UART and ingesting it into the netif, you need to spin a loop {} which calls UartDriver::read and then calls esp_netif_receive (I think)
So I am completely stuck here with post_attach and the ppp_netif_driver translation, I don't know enough about FFI for this to make a netif glue. I did manage to get the modem into data mode successfully though, but hooking up the uart send and receive is out of reach for me right now. I will continue hacking away at it, but welcome input from anyone with more experience on FFI and rust-c interop.
So I am completely stuck here with
post_attachand theppp_netif_drivertranslation, I don't know enough about FFI for this to make a netif glue. I did manage to get the modem into data mode successfully though, but hooking up the uart send and receive is out of reach for me right now. I will continue hacking away at it, but welcome input from anyone with more experience on FFI and rust-c interop.
I had to shoot in the dark a bit as I weren't sure what exactly is the problematic aspect. Hopefully I was right and this feedback would be helpful. If not (or if you have additional questions), I would gladly address those.
Also see this (NOTE: not typechecked!) which is what I now believe we are ultimately trying to achieve over here! :)
@DaneSlattery I decided it would be fair to at least typecheck my code, so you can (try to) use my EspNetifDriver idea from... esp-idf-svc master.
(We have to prohibit direct pushes to master; instead of git checkout -b netif-driver I did git checkout -p netif-driver and... here you go, my untested code ended up on master directly... oh well.)
To use the EspNetifDriver, you need to put the following in the Cargo.toml of your binary crate:
[patch.crates-io]
esp-idf-hal = { git = "https://github.com/esp-rs/esp-idf-hal" }
esp-idf-sys = { git = "https://github.com/esp-rs/esp-idf-sys" }
...and then in your binary crate you need to replace the ref to esp-idf-svc from crates.io with a ref from esp-idf-svc GIT
... or just also add
esp-idf-svc = { git = "https://github.com/esp-rs/esp-idf-svc" }
... to your [patch.crates-io] section, but since I assume my code is not bug-free, you might want to fork esp-idf-svc master and then bugfix it, and as such you should use your own GIT fork of esp-idf-svc master.
====
Basically, you need to call EspNetifDriver::new_ppp(tx), where tx is a callback so that the driver can call you back when it had produced a PPP packet that you need to send to UART. You of course need to supply the driver with a valid EspNetif which is created with PPP in mind (look at NetifStack::Ppp::default_configuration()).
To push RX packet into the driver, you need to call EspNetifDriver::rx with stuff you read from UART. Now, whether rx would expect complete PPP packets, or can work with fractional packets (I assume the latter, or else when reading from UART you need to understand where a packet ends/stops...?) I don't know...
Unfortunately, I cannot test this code for real, as I'm away from home, so I'm not having access even to my measly GPRS modem which is lurking around (actually not even sure this cheapo thing supports PPP, as I never tried it but...).
@DaneSlattery If you are already using my changes, please do cargo update in the root of your binary crate. I think I found a bug, where the PPP netif does not have an important flag raised upon creation, that I just added.
Hi @ivmarkov
Thank you for the assistance on this! The stuff you did boggles the mind, and I will continue to look around at the various comments. Just wanted you to know I have attempted this now, and I'm currently debugging this error.
Please note that my binary is actually the lte_example in my PR. I have pushed some updates there, and did a bit of life-time modification on the EspNetifDriver.
Update, ran this without free(buffer) and it at least started. Now I need to subscribe to network events I think.
I (1498) esp_idf_svc::modem: got response [65, 84, 90, 48, 13, 13, 10, 79, 75, 13, 10]
I (2498) esp_idf_svc::modem: got response [65, 84, 69, 48, 13, 13, 10, 79, 75, 13, 10]
I (3498) esp_idf_svc::modem: got response [13, 10, 43, 67, 71, 82, 69, 71, 58, 32, 48, 44, 48, 13, 10, 13, 10, 79, 75, 13, 10]
I (3498) esp_idf_svc::modem: CGREG: n: 0stat: 0, lac: None, ci: None
I (4508) esp_idf_svc::modem: got response [13, 10, 79, 75, 13, 10]
I (5508) esp_idf_svc::modem: got response [13, 10, 67, 79, 78, 78, 69, 67, 84, 32, 49, 49, 53, 50, 48, 48, 13, 10]
I (5508) esp_idf_svc::modem: connect Some("115200")
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x40380194 PS : 0x00060133 A0 : 0x8038000a A1 : 0x3fca5af0
0x40380194 - tlsf_free
at ??:??
A2 : 0x3fc971c4 A3 : 0x3fca7c60 A4 : 0x0000002d A5 : 0x00060123
A6 : 0x002d0018 A7 : 0x3fca7c58 A8 : 0x00010083 A9 : 0x00010082
A10 : 0x3fca7c58 A11 : 0xfffffffc A12 : 0x3fca7c5c A13 : 0x00000002
A14 : 0x3fca4e84 A15 : 0x00000000 SAR : 0x0000001f EXCCAUSE: 0x0000001c
EXCVADDR: 0x002d001c LBEG : 0x40056f5c LEND : 0x40056f72 LCOUNT : 0x00000000
Backtrace: 0x40380191:0x3fca5af0 0x40380007:0x3fca5b10 0x40377246:0x3fca5b30 0x40381425:0x3fca5b50 0x42002f92:0x3fca5b70 0x42071137:0x3fca5b90 0x4205dce5:0x3fca5bb0 0x4205516c:0x3fca5bd0 0x420552dc:0x3fca5bf0 0x42070c51:0x3fca5c20 0x4205b733:0x3fca5c40 0x4205b78f:0x3fca5c70 0x4205a80f:0x3fca5c90 0x42054de9:0x3fca5cb0 0x42055055:0x3fca5cd0 0x42054bbf:0x3fca5cf0 0x42054c5d:0x3fca5d10 0x4205ddba:0x3fca5d30 0x4205c6f1:0x3fca5d50 0x4205c19b:0x3fca5db0 0x42047241:0x3fca5dd0
0x40380191 - tlsf_free
at ??:??
0x40380007 - multi_heap_aligned_free
at ??:??
0x40377246 - heap_caps_free
at ??:??
0x40381425 - cfree
at ??:??
0x42002f92 - esp_idf_svc::netif::driver::EspNetifDriverInner<T>::raw_tx
at ??:??
0x42071137 - esp_netif_transmit
at ??:??
0x4205dce5 - pppos_low_level_output
at ??:??
0x4205516c - pppos_output_last
at ??:??
0x420552dc - pppos_write
at ??:??
0x42070c51 - ppp_write
at ??:??
0x4205b733 - fsm_sconfreq
at ??:??
0x4205b78f - fsm_lowerup
at ??:??
0x4205a80f - lcp_lowerup
at ??:??
0x42054de9 - ppp_start
at ??:??
0x42055055 - pppos_connect
at ??:??
0x42054bbf - ppp_do_connect
at ??:??
0x42054c5d - ppp_connect
at ??:??
0x4205ddba - esp_netif_start_ppp
at ??:??
0x4205c6f1 - esp_netif_start_api
at ??:??
0x4205c19b - esp_netif_api_cb
at ??:??
0x42047241 - tcpip_thread
at ??:??
ELF file SHA256: 000000000
Rebooting...
���ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40376cc0
0x40376cc0 - esp_restart_noos
at ??:??
SPIWP:0xee
mode:DIO, clock div:2
load:0x3fce3818,len:0x16f8
load:0x403c9700,len:0x4
load:0x403c9704,len:0xc00
load:0x403cc700,len:0x2eb0
entry 0x403c9908
Please note that my binary is actually the
lte_examplein my PR. I have pushed some updates there, and did a bit of life-time modification on theEspNetifDriver.
Yes, I commented on it.
Update, ran this without
free(buffer)and it at least started.
Let's keep free commented out for now, until we figure out (by examining the C ESP Modem) if we need to call it or whoever is calling us calls it for us once we return control to it.
Now I need to subscribe to network events I think.
Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our eventloop module.
One inconvenience is that we don't have the IP_EVENT_PPP_GOT_IP / IP_EVENT_PPP_LOST_IP mapped. Maybe you can map them to Rust structures following the lead of what had been done for the (very similar) IP_EVENT_STA_GOT_IP / IP_EVENT_STA_GOT_IP and IP_EVENT_ETH_GOT_IP and IP_EVENT_ETH_GOT_IP?
You can also spawn another thread which is examining the EspNetif::is_connected / ::get_ip_info in a loop + sleeping for a while if you don't want to deal with mapping ESP_EVENT_PPP_* to Rust immediately...
Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our
eventloopmodule.
Now that I think of it, I'm not sure what protocol is used over PPP so that the ESP will get an IP address (that is, assuming you don't just assign a fixed IP). It can't use DHCP, as DHCP lives on the border between Ethernet and IP, and does assume the medium below IP is Ethernet.
So what is it? Maybe something within PPP itself?