feature: Move shell features like history/tab completion to the host machine (and other goodies!)

[jasta]

trafficstars

First of all, thank you for this crate, and I'm excited to start using it in a new embedded project of mine! One thing jumped out to me though is that I think we can greatly improve ergonomics (and optimize!) the crate by leveraging the host machine to do more of the heavy lifting job that the shell normally takes over. Specifically, we could ship a special CLI tool that runs on the host machine with the embedded-cli acting as a kind of CLI tool server with the communication pipe being, e.g. the USB serial at /dev/ttyACM0. Advantages of this approach include:

1. Advanced shell features like history, tab completion, etc can be easily offloaded to the much more powerful host machine (and thus embedded-cli uses less memory)
2. Embedded commands are scriptable on the host, able to fully support stdin, stdout, stderr, and command exit status transported through e.g. /dev/ttyACM0
3. Agnostic to USB serial transport (i.e. we could just as easily support transport over Wi-Fi, BLE, LoRa, whatever)
4. Possibly easier to support async I/O since we could implement the frame reading/writing separate from the embedded-cli handling of those frames.

I've done something like this in the past for Android applications, very similar to the use cases I think you're targetting. The use case there was plug in your phone over USB, then run a custom command on your laptop to interact with your Android application with the core CLI heavy lifting happening on the Android side. See: https://github.com/facebook/stetho/blob/main/scripts/dumpapp for that custom command and you can imagine that the embedded side of this is the server that implements this simple framing protocol.

I'm interested in pursuing support for this with a PR, but I wanted to start the discussion at a higher level first to gauge interest and see if there are gotchas I'm not seeing

[funbiscuit]

Hi! Thanks for the idea.

That makes sense. On the embedded side, you will need to run a very small RPC server that only responds to commands from the client. You just need to ensure that the client and server understand each other correctly.

The main challenge would be the requirement for special software on the client machine. Currently, you're able to communicate with the CLI via any terminal application. However, with this approach, you would need special software for each version of the CLI.

I don't think this would be possible with a PR since it would require significant changes to the library. The whole approach would need to be different. So, I think it makes sense to treat this as a separate project, as there are no overlapping parts.

I think that if I thought about such approach in the beginning I might have chosen it over parsing everything on the embedded side. It really is more robust and efficient. But since there are already published versions of crate I don't think I can change the approach in this library.

[jasta]

Ack'd, I like the idea of at least still re-using some of the same command structure and syntax, perhaps I'll take a crack at this using only embedded-cli-macros and post my results as an RFC so we can see if there's any opportunity to refactor and re-use more common parts. For example one potential outcome would be that the CliBuilder can be customized to use either my proposed custom framing or the current unframed/plain text approach, something like:

let cli = CliBuilder::default()
  .framed() // <-- gives you a CliBuilderFramed that drops the command_buffer/history_buffer
  .writer(...)
  .build()
  .ok()?;  

Oh and re a separate CLI tool, this can be distributed through cargo cargo install embedded-cli-tool, so you can keep everything nice and tidy for installation in the README with virtually no effort. Anyway, just thinking out loud a little bit, I'll go ahead and proceed to get something working for my upcoming project and follow-up here with the specifics.

[funbiscuit]

I think this might work. Since rust will remove all unused code it will be okay to have two separate implementations of parsing input stream of bytes. One to do actual parsing with history and all that (current impl) and another to just parse incoming commands in some byte represantation (the one you're proposing).

Wire protocol is one more thing that needs to be established.