Add convenience proc-macros

[Luro02]

Thank you for your contribution!

We appreciate the time and effort you've put into this pull request.
To help us review it efficiently, please ensure you've gone through the following checklist:

Submission Checklist 📝

• I have updated existing examples or added new ones (if applicable).
• I have used cargo fmt command to ensure that all changed code is formatted correctly.
• I have used cargo clippy command to ensure that all changed code passes latest Clippy nightly lints.
• My changes were added to the CHANGELOG.md in the proper section.

Pull Request Details 📖

Description

After spending the time writing the issue #552, I wanted to see if it works (yes it does), given that it should be trivial to implement, but it wasn't trivial (like every time I try to implement something for the esp).

Here are my findings:

• The ESP-IDF IRAM_ATTR will link code to .iram1, which will end up in .iram0.text of the final binary
• The SRAM is split between IRAM and DRAM -> If the code in IRAM references static data, that should be placed in DRAM (= .dram1, .dram0.data in the final binary) which is used for data.
• The C attribute uses __COUNTER__ to put each element into a unique subsection, in the final binary they will all end up in .iram0.text/.dram0.data. These unique subsections seem to be for some linker issue that I could not reproduce in rust. I added code that tries to put each one in a distinct subsection. If there are multiple items in the same section, it shouldn't be in an issue in rust, because all items are mangled by default.
• It is very annoying that #[link_section = ...] will not place literals in the section. See Place &str in specific linker sections rust-lang/rust#70239, I added some workaround code for this issue that took up the majority of time.
• Rewriting the function body to automatically store literals in ram is not feasible. It is difficult to replace code based on context in proc-macros, which would be necessary here.
• If I understood it correctly, it is okay if some items ends up in flash and some in the sram. It will only be a problem if the code is called in an interrupt while the flash is written to (which would prevent reading from it), but in that case the CPU will fault?
• The C code technically supports placing static data in iram, but I am not a fan of this, given that it is executable. This would be useful on the ESP32, because of a bug the ratio of iram/dram seems fixed and not all of it can be statically assigned? On the ESP32-S3 the docs don't mention this issue.

I also added an internal helper macro that should help with documenting the features required for a piece of code #547.

What could be added in the future?

• rtc and tcm support
• a psram macro for placing code there

Testing

Simply use the crate in the project, then you can list all symbols in the final binary with
xtensa-esp32-elf-objdump.exe -x your_binary (if mangling is on, search with the regex \.iram0\.text.*? _ZN). To look at what was stored there, I used xtensa-esp32-elf-readelf -x .dram0.data and looked up the address there.

[ivmarkov]

• If I understood it correctly, it is okay if some items ends up in flash and some in the sram. It will only be a problem if the code is called in an interrupt while the flash is written to (which would prevent reading from it), but in that case the CPU will fault?

This is also my understanding. But whether "the CPU will fault" = "ok" is a big question?

Again, the example with the SPI driver. It is absolutely not "ok" to rely on coincidence and have "some items in flash" and "some in sram". Because then, the async SPI driver will end up crashing, and people with start complaining that "it does not work". And then go explain the whole saga that putting stuff in IRAM is best effort only. And they might even not know about "IRAM" in the first place. Which is why the SPI driver's async code-path is compiled out when the ESP-IDF config CONFIG_SPI_ISR_HANDLER_IN_IRAM_BLAH_BLAH is enabled. Which folks find non-intuitive but that's another topic: #486

For other cases - like the just-merged PCNT driver - not having those counting functions in IRAM only means that they will be executed more slowly, and might report big count gaps if somebody is locking the flash and operating on it at the same time (i.e. an OTA update or suchlike). But this is anyway also valid for the user code which calls the driver count functions - if it is not in IRAM (and this is very difficult to assert) it will also be delayed when counting, even if the driver API itself is not.

===

So yeah, we can have the macros you suggest, but their usefulness might be a bit limited, as per above?

[Luro02]

It would be ideal if the macro could be used to put the function and everything it requires into SRAM, but I don't know how that could be accomplished without placing the link_section (in this case the ram attribute) on everything the function is using. This would sadly have to be done manually? because proc-macros are not powerful enough. Maybe a compiler plugin could work? But I have never worked with those and I think that will introduce new problems.

I thought about going the linker route mentioned in the docs, which would circumvent some of the problems, but I am not knowledgeable enough about the way esp projects are built in rust.

Regarding the fallout of partially having items in flash and sram:

This is the issue the docs describe. By default the interrupts seem to be not IRAM-safe and they will be suspended while the flash is accessed. To let the interrupt execute while it is accessing the flash, one must set the ESP_INTR_FLAG_IRAM flag.

[ivmarkov]

It would be ideal if the macro could be used to put the function and everything it requires into SRAM, but I don't know how that could be accomplished without placing the link_section (in this case the ram attribute) on everything the function is using. This would sadly have to be done manually? because proc-macros are not powerful enough. Maybe a compiler plugin could work? But I have never worked with those and I think that will introduce new problems.

That's the point exactly. You have to place it on the transitive closure of what the function calls, and (a) it might not work for some functions (generics, who knows what, etc.) or (b) it might even be something you don't have access to (3rd party crates etc.).

This is the issue the docs describe. By default the interrupts seem to be not IRAM-safe and they will be suspended while the flash is accessed. To let the interrupt execute while it is accessing the flash, one must set the ESP_INTR_FLAG_IRAM flag.

Sigh. I know that. But - not all drivers give you the freedom to say - with an API call - "hey, driver, your interrupt handlers must be iram-safe". For some, this is a conf setting. Hence the SPI async driver CONF_ drama from above. Look at the issue I've linked.

====

With that said, if you want to push this to completion - sure. Just don't be fooled that this is a universal, generic solution to the "your code shall be in IRAM, completely" problem.

[Luro02]

Just don't be fooled that this is a universal, generic solution to the "your code shall be in IRAM, completely" problem.

Wasn't my intention. The rust compiler is not even placing string literals in the right section, only with some const code hacks. The magic solution for this, is likely far away.

It is supposed to be an initial attempt at being able to place things in iram, similarly to what the IRAM_ATTR in the C code does, cleaning up the uses in the code where currently link_section is manually specified.

If someone has the ambition to improve upon this in the future, sure, but I hope I am not the one who needs it.

[Luro02]

After thinking about this, and doing some research, I want to spend some more time exploring the linker router. (I will make a draft PR out of this)

Main idea would be to do it like this:
https://docs.rust-embedded.org/embedonomicon/memory-layout.html

but I will have to think a bit about how it could work out

[Luro02]

Okay, I spent the time looking at the problem.

The ESP-IDF linker fragments allow putting a symbol, object or entire (static) library into ram. This behavior can be replicated in rust through build.rs scripts and using separate crates that would become a dylib or staticlib.

Assuming this would be implemented, one would have to separate the codebase into one for ram code and one for all the other code.

I think this is quite inconvenient, and a proc-macro wouldn't help here. This seems like it should be a template, with maybe a crate that contains all the build.rs code.

But implementing this is not as easy as it sounds, given that the standard library exists and will likely be invoked. If it is compiled as a static library, the standard library will be fully exported, preventing it from being compiled with the flash code (that would also have the standard library in it -> symbol conflicts). Compiling as a dynamic library is another option, but here I am wondering how (assuming one knows which functions should be in ram) one would put the std functions in ram.

Transitively putting code in some sections sounds like a tricky thing to accomplish and I think the people who really need that should write these interrupt handlers in c code or spend the time looking at the assembly and ensuring that it is not accessing flash.

[ivmarkov]

Right. This is why I gave up on this long ago.
And also why esp-hal (the baremetal cousin) does not try either. They have the #[ram] thing, but it is only used for performance (not for correctness) and performance is "not guaranteed", due to the transitive closure thing. @bugadani correct me if I'm wrong.

[ivmarkov]

But implementing this is not as easy as it sounds, given that the standard library exists and will likely be invoked. If it is compiled as a static library, the standard library will be fully exported, preventing it from being compiled with the flash code (that would also have the standard library in it -> symbol conflicts). Compiling as a dynamic library is another option, but here I am wondering how (assuming one knows which functions should be in ram) one would put the std functions in ram.

Dynamic libraries are not supported on MCUs.