Implement `cargo espidf install`

[ivmarkov]

For cases where users would like to have all of ESP-IDF (the repo itself, GCC toolchain, pip deps) pre-installed - rather than downloaded on-demand during cargo build, it might be useful to have such a command.

The existing alternative is for users to do

git clone --recursive https://github.com/espressif/esp-idf
cd esp-idf
./install.sh esp32

(... and also a few pre-requisites that we don't automate either, as per here).

Not super-difficult, but if we would like to isolate the C world even for offline, such a cargo sub-command might be useful.

Possible syntax:

cargo espidf install [--idf-ver <branch-or-tag>] [--tooling-mcu <esp32|esp32c3|...|all>]

The above would just do what we do at build time w.r.t. downloading and installing ESP-IDF.
Given that most of the ESP-IDF download & installation code is in embuild (where cargo espidf itself resides) I don't think that would be such a big lift and shift.

Eventually, we might have an update variant that updates an already existing installation. OR maybe install can do that automatically, if it detects an existing installation.

[benthair]

Any news?

Why it was removed from esp-rs?

We also want to separate programming and compilation from abusive online checks and downloads, by simple commands, wirhout doing extra work manually.

Rust natively allows users to create new project, develop, check for errors with clippy, rust-analyzer, compile, run and test without mandatory internet access on every action. Without having to do extra manual things to allow it.

[ivmarkov]

Why it was removed from esp-rs?

What was removed from esp-rs?

We also want to separate programming and compilation from abusive online checks and downloads, by simple commands, wirhout doing extra work manually.

If you mean, "I want this functionality", perhaps you could contribute a PR?
The changes should actually be in the embuild crate (and thus much easier to test).

All esp-idf-* projects are community-driven (as is now written everywhere in big bold letters).

Rust natively allows users to create new project, develop, check for errors with clippy, rust-analyzer, compile, run and test without mandatory internet access on every action. Without having to do extra manual things to allow it.

Sure about that? cargo build does - in general - need internet access.
From where do you think your crate dependencies get downloaded?

[benthair]

What was removed from esp-rs?

tom-borcin removed this from esp-rs

I would love to contribute, but I'm not the right person for the job. Is everyone contributing because they needed a new feature? This is serious. It's about user security and privacy.Most people don't worry about those things because otherwise they wouldn't use Macintosh or Windows.

I am already working hard to keep the environment safe, and when I have the opportunity, I share my knowledge. But unfortunately, I am new to the world of microcontrollers

All esp-idf-* projects are community-driven (as is now written everywhere in big bold letters).

Does that mean the community doesn't understand security and privacy?

Sure about that? cargo build does - in general - need internet access.
From where do you think your crate dependencies get downloaded?

Yes. I am completely sure because I use it every day.

I use NeoVIM (Arch Linux), with coc-nvim. The coc configuration says: “extensions.updateCheck”: false.

When I start nvim, it does not request internet access.

When I start a new project with cargo new, if I add libraries with cargo add that I don't already have downloaded, that's when it accesses crates.io (with my knowledge and consent in an isolated environment with bwrap and exclusive permission to crates.io and a few other subdomains: index.crstes.io and ststic.crates.io).

At compile time, it only needs internet access the first time, that is, if the library does not exist on my system. And if it gives an error because it tries to download but does not have internet access (because on my systems there is no internet access without explicit permission the a whitelist domains/subdomains/IP Addresses), then I start the secure environment (a simple snvim -g crates.io command) and it starts downloading.

Once that's done, I can program and compile for months without a single internet connection.

I find it ridiculous to have to allow internet access every time I open nvim to program on ESP32.

Even if I have given it permission before and it has downloaded, every time I open nvim it insists on making requests and downloading whatever it wants, whenever it wants. We leave Windows to avoid backdoors, and now it seems that backdoors are pursuing us.

I don't know if this community is aware that there are malicious crates out there. Or that there are native Linux applications created in Golang that incorporate telemetry (e.g., rclone).

It is also possible to have a network service with a zero-day vulnerability. If the system has free access to the internet, it does not matter how isolated it is because it can be used as a proxy and attack other servers around the world, or accessing the rest of the network (pivoting).

[ivmarkov]

What was removed from esp-rs?

tom-borcin removed this from esp-rs

Yes. That's because the team of paid developers who work for Espressif are putting all their efforts into esp-hal.
esp-hal = baremetal SDK. The "official" Rust tolling by Espressif
esp-idf-hal = Rust wrappers around the ESP IDF C SDK. Maintained by myself (and I'm overwhelmed) and those folks who are willing to put extra effort.

I would love to contribute, but I'm not the right person for the job. Is everyone contributing because they needed a new feature? This is serious. It's about user security and privacy.Most people don't worry about those things because otherwise they wouldn't use Macintosh or Windows.

I am already working hard to keep the environment safe, and when I have the opportunity, I share my knowledge. But unfortunately, I am new to the world of microcontrollers

All esp-idf-* projects are community-driven (as is now written everywhere in big bold letters).

Does that mean the community doesn't understand security and privacy?

You are not winning friends here with your... attitude.

Again
(a) I'm overwhelmed
(b) Open a PR if this matters to you so much. It is a chance to learn Rust and microcontrollers too

Sure about that? cargo build does - in general - need internet access.
From where do you think your crate dependencies get downloaded?

Yes. I am completely sure because I use it every day.

I use NeoVIM (Arch Linux), with coc-nvim. The coc configuration says: “extensions.updateCheck”: false.
...(and so on)...

Sigh. Sure. If you take measures, you can make it work without constant internet access.
This is also possible right now, with ESP IDF if you are so security-sensitive.
As per the subject of this issue:

git clone --recursive https://github.com/espressif/esp-idf
cd esp-idf
./install.sh esp32

If you want convenience - you have to work on that, sorry. It is just not the most important issue (from my POV). If it is for you - PR please.

[benthair]

Are you overwhelmed? I'm very sorry for taking up your time with my requests and long texts. I'm in the same boat as you with my work. I don't have enough time.

But I didn't know how things were going internally. From the outside, all you can see is that time is passing and it seemed that no one was concerned about this request that @mvasi90 opened, with whom I share the same problems.

You are not winning friends here with your... attitude.

If I wanted to make friends, I would create social media accounts.

This is also possible right now, with ESP IDF if you are so security-sensitive.
As per the subject of this issue[...]

If there is no other option for now, I will have to do that.

Thank you, and I apologize for taking up your time.

[ivmarkov]

Sigh.
I don't mind reviewing PRs or discussing an issue. But I just cannot do all PRs folks could possibly ask for.

That's why I create those issues with such long descriptions - in the hope that somebody could be willing to help by doing the work, if the feature is deemed important to them. I thought this context is clear? Now you turn this into "this guy does not even want to discuss the issue". Anyway - proceed as you see fit.

[benthair]

Hahaha, you made me laugh. Nothing to do with it, I understand perfectly.

Update:
It's clear: a community of bearded children who don't understand security. Take the mouse out of their hands and they don't know how to use the computer anymore. The keyboard is for typing print “Hello world” and playing games the rest of the day. You tell them to create a multilevel policy in selinux and they say you're not their friend.

And the team leader whines when someone asks for security features because no one else helps him and he can't handle everything.

I'm joking. I know it's a lot of work, and it's already important enough as it is, without having to complicate it further by creating barriers that take up your working time even after you've finished creating them.

I have a hard time maintaining my systems (including my phone without Google and without internet access, and also with native Linux in chroot). There are days when I waste a lot of time just keeping it stable. Due to lack of time, I migrated from Gentoo to Arch Linux.

But even so, I prefer security and privacy.
You'd be scared if you saw the log of unauthorized internet connection attempts. And it's from reliable software... Not malicious.

[MabezDev]

@benthair Please observe and follow Rust's Code of Conduct being rude because someones priorities don't align with yours is not acceptable. esp-idf-* is community supported, and development is done on a volunteer basis.