Promote tier 3 riscv32 ESP-IDF targets to tier 2

[SergioGasquez]

Proposal

I would like to propose promoting the RISC-V 32bit ESP-IDF targets, riscv32imc-esp-espidf, riscv32imac-esp-espidf, & riscv32imafc-esp-espidf to tier 2.

Tier 2 target requirements

A tier 2 target must have value to people other than its maintainers. (It may still be a niche target, but it must not be exclusively useful for an closed group.)

In general, RISC-V is an open specification, used and accessible to anyone, including individuals.

The *-espidf targets are specific for the Espressif line of MCUs and their ESP-IDF operating system (which is largely POSIX compatible). With that said, the ESP IDF is: Open source Having a high popularity (the “de facto” SDK for developing C-based firmware on top of the Espressif MCUs)

…so we believe it is not the case at all that it is used by a closed group.

A tier 2 target must have a designated team of developers (the "target maintainers") available to consult on target-specific build-breaking issues, or if necessary to develop target-specific language or library implementation details. This team must have at least 2 developers.

@MabezDev, @ivmarkov, and I (@SergioGasquez) will maintain the targets.

The target must not place undue burden on Rust developers not specifically concerned with that target. Rust developers are expected to not gratuitously break a tier 2 target, but are not expected to become experts in every tier 2 target, and are not expected to provide target-specific implementations for every tier 2 target.

I don't foresee this being an issue. The maintainers will ensure we avoid undue burden for the general Rust community.

The target must provide documentation for the Rust community explaining how to build for the target using cross-compilation, and explaining how to run tests for the target. If at all possible, this documentation should show how to run Rust programs and tests for the target using emulation, to allow anyone to do so. If the target cannot be feasibly emulated, the documentation should explain how to obtain and work with physical hardware, cloud systems, or equivalent.

There is a QEMU fork of Espressif that supports ESP32-C3. Also there is an experimental feature for running ESP-IDF applications on Linux, which is available for all the RISC-V targets, Wokwi simulation could also be an option as its supports all the targets.

The target must document its baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar.

We already have some documentation, see *-espidf chapter of the The rustc book

If introducing a new tier 2 or higher target that is identical to an existing Rust target except for the baseline expectations for the features or versions of CPUs, operating systems, libraries, runtime environments, and similar, then the proposed target must document to the satisfaction of the approving teams why the specific difference in baseline expectations provides sufficient value to justify a separate target.

The riscv32XXX-esp-espidf targets are indeed very similar to their bare-metal counterparts (riscv32XXX-unknown-core-elf). Where they do differ (but this is key) is in having the following additional properties set on the -espidf targets:

• Family (“unix”)
• target_os (= “espidf”)
• target_env (= “newlib”)
• target_vendor (= “espressif”)

This allows the STD code to branch where necessary for these targets, as done by all others that do support the Rust Standard Library.

Tier 2 targets must not leave any significant portions of core or the standard library unimplemented or stubbed out, unless they cannot possibly be supported on the target.

• core, alloc, and panic_abortare fully implemented.
• std is fully implemented as well, except for support for processes and signals, which are stubbed out.

We do understand that not implementing processes (and therefore signals) is a bit controversial, but with that said, an MCU target cannot feasibly implement those, as often there is just no notion of “multiple processes” there (as is the case with ESP-IDF).

So we do believe we cannot possibly support processes and signals indeed, and thus we fall in the same category as other Tier 2 targets, like wasm, where neither processes, nor even other core aspects of the Standard Library, like multi-threading, BSD sockets, or even mutexes are implemented.

The code generation backend for the target should not have deficiencies that invalidate Rust safety properties, as evaluated by the Rust compiler team. (This requirement does not apply to arbitrary security enhancements or mitigations provided by code generation backends, only to those properties needed to ensure safe Rust code cannot cause undefined behavior or other unsoundness.) If this requirement does not hold, the target must clearly and prominently document any such limitations as part of the target's entry in the target tier list, and ideally also via a failing test in the testsuite. The Rust compiler team must be satisfied with the balance between these limitations and the difficulty of implementing the necessary features.

RISC-V is a well-established and well-maintained LLVM backend. To the best of my knowledge, the backend won't cause the generated code to have undefined behavior.

If the target supports C code, and the target has an interoperable calling convention for C code, the Rust target must support that C calling convention for the platform via extern "C". The C calling convention does not need to be the default Rust calling convention for the target, however.

The C calling convention is supported.

The target must build reliably in CI, for all components that Rust's CI considers mandatory.

This should not be a problem at all. The only dependency the *-espidf targets need is on libc which is already a listed dependency in the Standard Library and which does support ESP-IDF.

There is no need for any native libraries to be present in the CI, and as such building the Standard Library for ESP-IDF only requires the Rust compiler and nothing else.

The approving teams may additionally require that a subset of tests pass in CI, such as enough to build a functional "hello world" program, ./x.py test --no-run, or equivalent "smoke tests". In particular, this requirement may apply if the target builds host tools, or if the tests in question provide substantial value via early detection of critical problems.

The targets will not build host tools. With that said:

• If there is a need to have a build that ends up with an Espressif MCU ELF binary (as opposed to just building the Rust Standard Library), Espressif does have an infrastructure to do this by downloading all dependencies automatically during the build (including checking out the ESP IDF repository which is built from source code, a bit like the Linux kernel)
• The above dependencies can also be installed in a “vendored” way if necessary

Building the target in CI must not take substantially longer than the current slowest target in CI, and should not substantially raise the maintenance burden of the CI infrastructure. This requirement is subjective, to be evaluated by the infrastructure team, and will take the community importance of the target into account.

To the best of my knowledge, this will not induce a burden on the current CI infra.

• Building the Standard Library itself for ESP-IDF is very fast
• Building an Espressif MCU ELF binary is also fast (in the order of a couple of minutes), yet there is extra time necessary (also in the order of a few minutes) to download the Espressif-specific C toolchains. If this is a concern and if building an ELF binary is actually even necessary, we can consider vendoring the toolchains and the ESP-IDF itself to cut on internet traffic

Tier 2 targets should, if at all possible, support cross-compiling. Tier 2 targets should not require using the target as the host for builds, even if the target supports host tools.

Cross-compilation is supported and documented in the platform support document. In fact, cross-compilation is the only way to build for these targets.

In addition to the legal requirements for all targets (specified in the tier 3 requirements), because a tier 2 target typically involves the Rust project building and supplying various compiled binaries, incorporating the target and redistributing any resulting compiled binaries (e.g. built libraries, host tools if any) must not impose any onerous license requirements on any members of the Rust project, including infrastructure team members and those operating CI systems. This is a subjective requirement, to be evaluated by the approving teams.

There are no additional license issues to worry about.

Tier 2 targets must not impose burden on the authors of pull requests, or other developers in the community, to ensure that tests pass for the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on tests failing for the target. Do not send automated messages or notifications (via any medium, including via @) to a PR author or others involved with a PR regarding the PR breaking tests on a tier 2 target, unless they have opted into such messages.

Agree.

The target maintainers should regularly run the testsuite for the target, and should fix any test failures in a reasonably timely fashion.

Agree.

All requirements for tier 3 apply.

They apply, they currently are tier 3 targets.

Tier 2 with host tools

Given that the target runs on MCUs, it will not have any host tools.

Mentors or Reviewers

We asked @davidtwco and @Amanieu, but no response from their side yet. Anyone is welcome, but they do have some context already.

Process

The main points of the Major Change Process are as follows:

• [x] File an issue describing the proposal.
• [ ] A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• [ ] Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

[!CAUTION]

Concerns (2 active)

• ~~ask libs consensus on limited std support~~ resolved in this comment
• ~~fd duplication~~ resolved in this comment
• continued-maintainer-interest
• documentation

Managed by @rustbot—see help for details.

[rustbot]

[!IMPORTANT] This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rfcbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rfcbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler

[jieyouxu]

@rustbot concern ask libs consensus on limited std support

We should ask lib team consensus on promoting RISC-V 32bit ESP-IDF targets, riscv32imc-esp-espidf, riscv32imac-esp-espidf, and riscv32imafc-esp-espidf to tier 2 regarding limited std support re.:

• core, alloc, and panic_abort are fully implemented.
• std is fully implemented as well, except for support for processes and signals, which are stubbed out.

We do understand that not implementing processes (and therefore signals) is a bit controversial, but with that said, an MCU target cannot feasibly implement those, as often there is just no notion of “multiple processes” there (as is the case with ESP-IDF).

So we do believe we cannot possibly support processes and signals indeed, and thus we fall in the same category as other Tier 2 targets, like wasm, where neither processes, nor even other core aspects of the Standard Library, like multi-threading, BSD sockets, or even mutexes are implemented.

EDIT: library team nomination at https://github.com/rust-lang/rust/issues/140042.

[riking]

@rustbot concern fd duplication

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

Upstream issue: https://github.com/espressif/esp-idf/issues/7412

[ivmarkov]

@rfcbot concern fd-duplication

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms

Would you elaborate on those fundamental Unix-like file handling algorithms?

and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

If you mean BorrowedFd::try_clone_to_owned, this API is: (a) Fallible (b) The documentation is pretty clear that - in its own words - it is not supported on all platforms, i.e. "To obtain an OwnedFd, you can use BorrowedFd::try_clone_to_owned instead, but this is not supported on all platforms.".

Am I missing something?

My point of view is that (a) cloning file handles with dup / dup2 might also fail for other reasons anyway, even if these syscalls are supported (as in, you are running out of file descriptors). So higher-level APIs should guard for these failures anyway; (b) originally I thought not supporting duplication of file handles is going to be a big deal, but I've yet to find serious use cases, where this is a show stopper. Hence the request above to provide a few concrete examples.

[jieyouxu]

@rustbot resolve ask libs consensus on limited std support

Library team has some more questions (see MCP thread itself), but resolving the initial question part.

[rustbot]

Only organization members can add or resolve concerns.

[Urgau]

@rustbot concern fd duplication

From https://github.com/rust-lang/compiler-team/issues/864#issuecomment-2850245632

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent.

Upstream issue: espressif/esp-idf#7412

[ivmarkov]

@rustbot concern fd duplication

From #864 (comment)

The idea that an OwnedFd can be duplicated is fundamental in several Unix-like file handling algorithms and Rust in general (BorrowedFd as ToOwned). It's the exact kind of stdlib blind spot we're trying to prevent. Upstream issue: espressif/esp-idf#7412

Would you be so kind to also address my feedback from https://github.com/rust-lang/compiler-team/issues/864#issuecomment-2850346082?

[apiraino]

@ivmarkov @riking the tagging about concerns is meant to "block" proposals until they are resolved. The technical discussion is on Zulip (see comment). Thank you!

[ivmarkov]

@apiraino I wonder whether the "fd duplication" concern is still active, given this comment in the Zulip thread?

[apiraino]

@rustbot resolve fd duplication

Yes, I think so. I will lift the concern on behalf of him.

[apiraino]

Closing MCP as not accepted, see MCP process.

If the topic of discussion in this MCP should be reignited, feel free to reopen this proposal or a new one.

[ivmarkov]

@apiraino Just for my understanding - the MCP was not accepted just because of the fd-duplication concern, or was there more to it?

Because the statement of T-Libs in the Zulip thread is, to quote: " We discussed this again during today's T-libs meeting.

wrt. dup and openat, we do lose some of functionality here, but since this is an embedded target without processes the impact should be acceptable.

cfg(unix): The situation isn't great, but we don't have an exact line where something stops being unix. That crates like socket2, nix and mio work without special-casing esp-idf is encouraging. @Amanieu d'Antras was interested in pthread support, does parking_lot work out of the box?

So we're not exactly happy about it, but we're not going to block the promotion on these issues. "

[bugadani]

I think this was closed due to the following and we can just re-open it:

MCPs can be closed:

[2 options cut]
- by folks doing triage, if there have been three months of inactivity. In this case, people should feel free to re-open the issue if they would like to “rejuvenate” it.

I'm personally sad that "we forgot about this" is a basis to close the proposal, but I guess this is also a way to ping the participating people.

[ivmarkov]

I think this was closed due to the following and we can just re-open it:

MCPs can be closed:

[2 options cut]
- by folks doing triage, if there have been three months of inactivity. In this case, people should feel free to re-open the issue if they would like to “rejuvenate” it.

Fair enough. Let's maybe discuss the status quo during the next esp-rs meeting as if us - Espressif as well as the community - would like to continue pushing for that.

[apiraino]

@ivmarkov as per my comment on Zulip.

I had the feeling that the discussion stalled with what I think are two open questions:

• Does T-libs actually replied in full to the concern (comment)
• Do we have a policy for this level of support for Tier 2 targets? (comment)

Questions that I think needed a complete answer from any of the T-* involved there but AFAICS none chimed in.

(Trying to get the context without fully appreciating the technical nitty-gritty details 😉 , so feel free to comment there)

[ivmarkov]

• Does T-libs actually replied in full to the concern (comment)

My interpretation of where T-libs stands is that they actually replied in full. See my comment where I quoted their reply. And specifically, this actionable sentence of theirs (again): " So we're not exactly happy about it, but we're not going to block the promotion on these issues. "

(emphasis mine)

In other words, if they are not blocking the promotion, what is the other state? I assume they are "OK" or "accepting" the promotion.

• Do we have a policy for this level of support for Tier 2 targets? (comment)

Questions that I think needed a complete answer from any of the T-* involved there but AFAICS none chimed in.

I would again refer to the answer of T-libs here: " wrt. dup and openat, we do lose some of functionality here, but since this is an embedded target without processes the impact should be acceptable.

cfg(unix): The situation isn't great, but we don't have an exact line where something stops being unix. That crates like socket2, nix and mio work without special-casing esp-idf is encouraging. "

My interpretation again: no, there is no clear policy. But should that block promotion until there is a clear policy (if ever)? No.

So the situation is a little bit IMO - nobody is blocking the promotion - and to my understanding - (a) I have addressed all concerns to the best of my knowledge and (b) there are no outstanding concerns anyway.

And the conversation stalled on my (our) side simply because I was expecting a final decision from the Rust maintainers as I did not see anything else actionable that I could do besides pinging "when is the decision to be taken".

===

With that said, let me now internally talk to Espressif and other maintainers first whether we would like to further pursue that. It had been a bit too long in this state of flux.

[tgross35]

The maintainers have been pretty thorough in answering all questions, and there don't seem to be any remaining blockers here (in particular, T-libs was okay with it per @the8472 on Zulip). So for the procedural bit,

@rustbot second

However,

With that said, let me now internally talk to Espressif and other maintainers first whether we would like to further pursue that. It had been a bit too long in this state of flux.

@rustbot concern continued-maintainer-interest

I have a few nonblocking questions but I'll bring them up on Zulip.

[rustbot]

[!IMPORTANT] This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rustbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rustbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler

[tgross35]

@rustbot concern documentation

Per https://github.com/rust-lang/rust/issues/140042#issuecomment-2824863184, https://doc.rust-lang.org/nightly/rustc/platform-support/esp-idf.html should get a section covering what std functionality is missing.