RFC: Consider rolling our own project generation tool

[jessebraham]

Since this templates inception we have been using cargo-generate as the project generation tool. This has worked for the most part, however we have encountered a number of limitations and rough edges which have made continued development of this template more difficult that it probably needs to be.

Our team has discussed this topic in various meetings to some degree, and have come to the conclusion that perhaps we should just roll our own project generation tool for this. We can be incredibly opinionated about things and cater the tool to our specific needs, instead of trying to coerce a generic tool into doing what we want.

We will need to have some further discussion to nail down the requirements for such a tool, and make a final decision if we do indeed want to go down this path, however I am opening this issue now for the sake of tracking progress here.

[bjoernQ]

I made some experiments here: https://github.com/bjoernQ/esp-generate

[SergioGasquez]

Ill start doing some investigation on this issue! Ill start by collecting a list of requirements/ideas and then take a look at esp-generate

[SergioGasquez]

Requirements

• The user should be able to select:
  • Project name
  • Target chip
  • Enable esp-alloc
  • Enable Wifi/bluettoth/esp-now via esp-wifi
  • Enable esp-openthread
  • Use embassy
  • Enable stack protection? https://github.com/esp-rs/esp-generate/issues/45. Maybe if the user selects nightly, enable this by default?
  • Other options:
    • Devcontainer support
    • Wokwi simulation support
    • GHA files
    • Initialize git repo?
    • Toolchain channel (nightly/stable)?
    • RA support for Vscode?
• Output code should be formatted
• We need to have some CI that check the different combinations (or at least, the most simple and the most complex scenarios)
• https://github.com/esp-rs/esp-generate/issues/46?

@esp-rs/espressif feel free to add/modify anything to the list. esp-generate already does a bunch of those things, which is a excellent starting point!

[MabezDev]

I think this is a great initial set of requirements for the end-user experience!

I think we should also think about the requirements of the maintainer side of things (us :D), for example managing code snippets - is there a better way to do it? How might we manage generating multiple templates, for instance we may want an RTIC, Embassy and normal blocking template (maybe more?). These kinds of requirements are why we're exploring not using cargo-generate, so I think they're important to think about.

[bjoernQ]

I thought about some of that in my POC

The most annoying thing to me with cargo-generate was the fact that it's mostly trial-and-error - so my idea was to have a "template" which is just a normal project which works in VSCode (or any other IDE) and can be compiled just as is

t.b.h. the result is not as great as I hoped for but much better than what we had before - the code needs a lot of weird hints for replacements - while that would remain the case there are probably ways to improve the experience

For "multiple templates": could be done as multiple independent templates but in the POC I opted for having multiple binaries (which are "filtered" out during generation)

We need to have some CI that check the different combinations (or at least, the most simple and the most complex scenarios)

Haven't implemented that but I briefly thought about a way to tell the tool to generate all valid permutations which then could be tested if they compile - maybe it's not a great idea since that might be a lot of combinations. That idea could be adapted by having a way to give you the cmd-line invocations for all the valid combinations and we can copy-paste the relevant ones to a workflow

[bjoernQ]

BTW I would really love a have an optional TUI like in the POC

[SergioGasquez]

How might we manage generating multiple templates, for instance we may want an RTIC, Embassy and normal blocking template (maybe more?)

For "multiple templates": could be done as multiple independent templates but in the POC I opted for having multiple binaries (which are "filtered" out during generation)

My idea would be to have an extra argument which selects the template (blocking, embassy....)

Haven't implemented that but I briefly thought about a way to tell the tool to generate all valid permutations which then could be tested if they compile - maybe it's not a great idea since that might be a lot of combinations.

I guess this should be explored later on, but I think there gonna be too many possible combinations.

BTW I would really love a have an optional TUI like in the POC

Agree! I really like using the TUI

[SergioGasquez]

Just discovered https://github.com/AdinAck/cargo-embassy, which we can get some inspiration from

[SergioGasquez]

I created https://github.com/bjoernQ/esp-generate/issues/2 to track the esp-generate improvements that we planned

[jessebraham]

While I'm not sure whether or not we're ready to promote esp-generate to our recommended tool, might be nice to come up with a game plan here. I think we're close to the point where we can consider retiring this repository.

@SergioGasquez @MabezDev @bjoernQ do you have any thoughts on this? Are there any missing functionality and/or blocking issues in esp-generate preventing us from doing this currently?

[bjoernQ]

I think esp-generate is already better than esp-template.

Maybe we could do a "soft launch" by advertising it on Matrix and ask users to give it a try. There is always something to improve but at this point getting feedback from real users might be very valuable

[jessebraham]

@MabezDev what's the plan here? We've closed most of the issues in esp-generate, if there is any missing functionality or blocking issues remaining please let us know so that we can take care of it. Otherwise, I'd like to finally be done with this repo.

[MabezDev]

I think @bjoernQ's suggestion of a soft launch on matrix is a good idea. If all goes well we can retire this a promote esp-generate everywhere.

[jessebraham]

I've created an issue in esp-generate to make sure we're happy with the state of things prior to doing our first release, if anybody has any input there: https://github.com/esp-rs/esp-generate/issues/38

Probably next week we can pull the trigger on this.