xtensa-esp-elf: how to influence endianness ?

[gopakumarce]

trafficstars

Maybe a question for @ivmarkov - so in the below commit, if the IDF version is "new", then the gcc used is just xtensa-esp-elf - that gcc has __XCHAL_HAVE_BE set to 1 .. So if I am compiling for esp32, what exactly is the magic going on behind the scenes to compile as little endian ? xtensa-esp32-elf has it set to 0 nicely. I see a lot of discussions online which says an "xtensa-overlays.h" or "xtensa-config.h" or "core.h" or "core-isa.h" can be used to influence gcc to produce BE/LE, but I cant find exactly how that works - any pointers will be highly appreciated

The reason to ask is because I am trying to compile a bunch of C files generated by/during bindgen such that it can be linked with the final image. But if I compile with the gcc got by the call to chips.gcc_toolchain() it produces BE code whereas the rest of the code is obviously LE. So how exactly do I go about solving this - in bindgen phase, how can I make sure that my C files are compiled as per the target CPUs endianness (LE for esp32 etc..) ?

commit 31d27d748acb6252f15069d36873411e4bb47529 (tag: v0.33.5)
Author: ivmarkov <[email protected]>
Date:   Fri Oct 27 21:10:53 2023 +0000

    New release

<snip>
+            Self::ESP32 => {
+                if new {
+                    "xtensa-esp-elf"
+                } else {
+                    "xtensa-esp32-elf"
+                }
+            }

[ivmarkov]

During compilation... are you using and passing to the gcc compiler all supplied include files and in general all Cargs?

[ivmarkov]

For compilation we use cmake btw, not invoking the C compiler directly...

[gopakumarce]

@ivmarkov thx for the response, yes I am passing in all the flags I get from build_output - the code is here https://github.com/gopakumarce/esp-idf-sys/commit/d919224f581a82283f67f4f7ae4584db74196f4b#diff-37bbeb3d6061a16a1b78c746bdf8b0c514d591ccdbf4ab731502d5d844b2c6c5R136

And yes I was trying to dig into the code as to where exactly or how exactly an xtensa gcc is told to use the right endianness - where in the cmake or configuration is that mentoned, ive been unsuccesful so far to dig that deep. Once I know how its done, then I can try doing the same when invoking gcc during bindgen

[gopakumarce]

Note that in the diff above I tried including xtensa/config/core.h into the bindgen generated C file, thats actually silly of me because including that into a C file should not have any influence on the endianness gcc produces - but at this point im just clueless and ready to try anything :)

[gopakumarce]

Ok, so as far as I can see, there really is no magic gcc flag, it uses xtensa-esp<model>-elf-gcc for the actual compilation in Cmake, the xtensa-esp-elf-gcc (without the model number in there) seems to be some "generic" compiler ? At any rate the cargo_driver is using chip.gcc_toolchain() just to say "also add this to the list of toolchains installed" - the list of toolchains installed by default seems to have all the models of xtensa-esp<model>-elf-gcc, so this one just says "also install xtensa-esp-elf-gcc"

So for my purpose, where I really want to compile a bindgen C file, I guess chips.gcc_toolchain() is not the right choice, I really need to use the exact xtensa-esp<model>-elf-gcc, ill probably make a chips.gcc_compiler() or some such to get the exact gcc for my use case

[ivmarkov]

Ok, so as far as I can see, there really is no magic gcc flag, it uses xtensa-esp<model>-elf-gcc for the actual compilation in Cmake, the xtensa-esp-elf-gcc (without the model number in there) seems to be some "generic" compiler ? At any rate the cargo_driver is using chip.gcc_toolchain() just to say "also add this to the list of toolchains installed" - the list of toolchains installed by default seems to have all the models of xtensa-esp<model>-elf-gcc, so this one just says "also install xtensa-esp-elf-gcc"

So for my purpose, where I really want to compile a bindgen C file, I guess chips.gcc_toolchain() is not the right choice, I really need to use the exact xtensa-esp<model>-elf-gcc, ill probably make a chips.gcc_compiler() or some such to get the exact gcc for my use case

No I don't think your assessment is correct. What actually happened, is that some time ago, there was only xtensa-esp32-elf - for esp32, and then separately, xtensa-esp32s2-elf, xtensa-esp32s3-elf for esp32s2 and esp32s3. Approx 6 months or so, the ESP IDF folks united all "xtensa" toolchains into a single one - xtensa-esp-elf - just like the "riscv" toolchain is a single one from the very beginning, for all of esp32c2/esp32c3/esp32h2/esp32c5/esp32c6/esp32p4, even though there are variances in the riscv instruction set of each chip.

So now we have to use xtensa-esp-elf with ESP IDF 5.1+ and xtensa-espXX-elf for eariler ESP IDFs which use earlier toolchains.

Or to summarize, there MUST be a way to use the generic xtensa-esp-elf toolchain, and instruct it about the exact chip number (= xtensa instruction set) for which you need to compile. And... I thought this is a flag that you pass to the compiler, and is also one of the flags we give as a build-output...

[gopakumarce]

I see, thx for the context. xtensa-esp-elf-gcc --target-help has a 'dynconfig' option and xtensa-esp-elf-gcc --help also says a "spec" option, I was wondering if either of those are influencing what the target cpu is, but I cant see any of those options used in the esp-idf repo / make files. Maybe I will ask on the esp-idf channel itself and see how it works

[ivmarkov]

I see, thx for the context. xtensa-esp-elf-gcc --target-help has a 'dynconfig' option and xtensa-esp-elf-gcc --help also says a "spec" option, I was wondering if either of those are influencing what the target cpu is, but I cant see any of those options used in the esp-idf repo / make files. Maybe I will ask on the esp-idf channel itself and see how it works

Surely there is a way how they pass the chip type (or rather, the target arch). I.think there are differences between esp32/esp32s2/esp32s3.

[gopakumarce]

@ivmarkov circling back to this thread - pls see https://github.com/espressif/crosstool-NG/issues/50 - the esp-idf still uses the wrappers to compile/link, its just that the actual compiler is just built once, and the espXXX versions are just wrappers to the one built compiler.

Now coming to 31d27d748acb6252f15069d36873411e4bb47529 - I guess I understand what the intent was, so the chips.gcc_toolchain() API can basically be thought of as chips.gcc_toolchain_to_be_installed() - the API is just returning what has to be installed via esp-idf.py, and esp-idf.py is smart enough to know that if its told to install xtensa-esp-elf-gcc, it installs that AND all the wrappers.

In my case, I was trying to interpret chips.gcc_toolchain() as chips.gcc_toolchain_used_to_compile() - and that should really be the wrappers. Would you like me to add two separate APIs clearly indicating the difference in intent ?

[gopakumarce]

@ivmarkov opened https://github.com/esp-rs/esp-idf-sys/pull/283

[Vollbrecht]

The original question about endianness was discussed. If you have further issues or want to provide separate context please create a separate new issue. Closing this for now.