Hashless assets

[ealmloff]

This adds a new option to the AssetOptions struct to opt out of asset hashes. It also introduces a new external_asset! macro which keeps the asset even if the asset link is unused in the rust code.

Closes #4115

[jkelleyrtp]

Haven't looked at the code, but do you think it's possible to have external_asset be just a flag on the regular asset builder options?

[ealmloff]

Haven't looked at the code, but do you think it's possible to have external_asset be just a flag on the regular asset builder options?

No, there is a flag that controls if the hash is added, but it can't modify if the symbol is kept. external_asset just enables that flag and adds a #[used] attribute to the asset static so that it is kept even if the rust symbol is never used (which will be the case for most hashless assets). We can't read the contents of the asset options inside the macro because it is an arbitrary const expression which could be an ident. We could accept a third argument of literal-only values we parse in the macro like this, but without other options we need there I'm not sure it is a better api: asset!("path", ImageOptions::new(), { hashless = true })

[ealmloff]

Another API we could support is this with the #[used] in user code:

#[used]
static _EXTERNAL_ASSET: Asset = asset!(
    "/66571940.jpeg",
    ImageAssetOptions::new()
        .into_asset_options()
        .with_hash_suffix(false)
);

[jkelleyrtp]

Another API we could support is this with the #[used] in user code:

#[used]
static _EXTERNAL_ASSET: Asset = asset!(
    "/66571940.jpeg",
    ImageAssetOptions::new()
        .into_asset_options()
        .with_hash_suffix(false)
);

From what I remember this might have issues with LTO since the static won't escape the module. Only extern "C" symbols make it out if they're private, but that also has some complications with duplicate symbols being defined during some edge cases in version unification.

[Distortedlogic]

"From what I remember this might have issues with LTO since the static won't escape the module." - J.K.

Note: this isnt an "unused" image but a used one

I beleive this happens with dx bundle --release on 0.7.0-alpha.1

[ealmloff]

From what I remember this might have issues with LTO since the static won't escape the module. Only extern "C" symbols make it out if they're private, but that also has some complications with duplicate symbols being defined during some edge cases in version unification.

It looks like that is a general caveat to the used attribute in the reference. Since we now do the asset intercept after the linker, the linker can modify/remove symbols.

In practice, this doesn't seem to be the case on macos with the default linker. If you mark the asset symbol as used, it is kept even if it is private and unused in a library as long as that library is built. extern "C" functions can't use the used attribute and don't show up in the symbol table after lto if you objdump the final binary. This is the setup I used to test: https://github.com/ealmloff/linker-test-kept

Note: this isnt an "unused" image but a used one

I beleive this happens with dx bundle --release on 0.7.0-alpha.1

I can reproduce this issue in the alpha, but not with the git version of the CLI + dioxus. It might have already been fixed?

Building on web with this profile:

[profile.release]
opt-level = "z"
debug = false
lto = true
codegen-units = 1
panic = "abort"
strip = true
incremental = false

[Distortedlogic]

@ealmloff got some very bad news in this situation. Idk that it possible to force __LINK_SECTION static to persist in dx bundle --release in a desirable way

I cloned dioxus repo, in asset macro, I added #[used] .. I added pub .. seems the only way to get it to persist is to have __LINK_SECTION actually used in the code somewhere outside the scoping of the macro expansion

I beat the various LLMs but they all telling me there isnt another way

Here is my dioxus repo with the used n pub
https://github.com/Distortedlogic/dioxus/tree/asset-link-section-preservation

Here is the repo I am bundling
https://github.com/Summit-Sailors/developer-portfolio

I did fork dioxus-free-icons and bumped the dioxus version in it as I was using it in the site but thats the only dep that lso deps on dioxus
https://github.com/Distortedlogic/dioxus-free-icons/tree/forked

I can reproduce this issue in the alpha, but not with the git version of the CLI + dioxus. It might have already been fixed?

Did u use the --release flag with dx bundle? --release isnt default now for dx bundle .. without release works for me but also has the dev server trying to run .. with --release does not work for me even with latest git commit of dioxus

here is it working without --release

[ealmloff]

@ealmloff got some very bad news in this situation. Idk that it possible to force __LINK_SECTION static to persist in dx bundle --release in a desirable way

I cloned dioxus repo, in asset macro, I added #[used] .. I added pub .. seems the only way to get it to persist is to have __LINK_SECTION actually used in the code somewhere outside the scoping of the macro expansion

I beat the various LLMs but they all telling me there isnt another way

Here is my dioxus repo with the used n pub https://github.com/Distortedlogic/dioxus/tree/asset-link-section-preservation

Here is the repo I am bundling https://github.com/Summit-Sailors/developer-portfolio

I did fork dioxus-free-icons and bumped the dioxus version in it as I was using it in the site but thats the only dep that lso deps on dioxus https://github.com/Distortedlogic/dioxus-free-icons

I can reproduce this issue in the alpha, but not with the git version of the CLI + dioxus. It might have already been fixed?

Did u use the --release flag with dx bundle? --release isnt default now for dx bundle .. without release works for me but also has the dev server trying to run .. with --release does not work for me even with latest git commit of dioxus

here is it working without --release

You have strip = true defined in your cargo.toml which is not compatible with the symbol based asset system when building native binaries like the server. Try removing it or only enabling it in web builds:

https://github.com/Summit-Sailors/developer-portfolio/blob/main/Cargo.toml#L26

[Distortedlogic]

yes, I tried without strip and i tried without lto

with strip = false

44.629s DEBUG dx::build::request: Build completed successfully in 44349857us: "/home/entropybender/repos/developer-portfolio/target/wasm32-unknown-unknown/release/app.wasm"
  44.629s  INFO dx::build::builder: Bundling app...
  44.629s DEBUG dx::build::request: Running wasm-bindgen dx_src=bundle
  44.629s  INFO dx::build::builder: Running wasm-bindgen...
  44.629s DEBUG dx::wasm_bindgen: wasm-bindgen: [
    "--target",
    "web",
    "--debug",
    "--no-demangle",
    "--keep-debug",
    "--keep-lld-exports",
    "--out-name",
    "app",
    "--no-typescript",
    "--out-dir",
    "/home/entropybender/repos/developer-portfolio/target/dx/app/release/web/public/wasm",
    "/home/entropybender/repos/developer-portfolio/target/wasm32-unknown-unknown/release/app.wasm",
]
  44.931s DEBUG dx::build::request: wasm-bindgen complete in 301.972793ms dx_src=bundle
  44.982s DEBUG dx::build::builder: Setting builder to failed state
  44.982s ERROR dx::build::builder: err=Other(Failed to write main executable

Caused by:
    0: Failed to read module
    1: [parse exception: invalid code after misc prefix: 17 (at 0:559040)]) 
Failed to write main executable

with lto=false it builds but issue persists

with strip=false and lto=false

39.731s  INFO dx::build::builder: Bundling app...
  39.731s DEBUG dx::build::request: Running wasm-bindgen dx_src=bundle
  39.731s  INFO dx::build::builder: Running wasm-bindgen...
  39.731s DEBUG dx::wasm_bindgen: wasm-bindgen: [
    "--target",
    "web",
    "--debug",
    "--no-demangle",
    "--keep-debug",
    "--keep-lld-exports",
    "--out-name",
    "app",
    "--no-typescript",
    "--out-dir",
    "/home/entropybender/repos/developer-portfolio/target/dx/app/release/web/public/wasm",
    "/home/entropybender/repos/developer-portfolio/target/wasm32-unknown-unknown/release/app.wasm",
]
  40.226s DEBUG dx::build::request: wasm-bindgen complete in 495.435904ms dx_src=bundle
  40.283s DEBUG dx::build::builder: Setting builder to failed state
  40.283s ERROR dx::build::builder: err=Other(Failed to write main executable

Caused by:
    0: Failed to read module
    1: [parse exception: invalid code after misc prefix: 17 (at 0:662914)]) 
Failed to write main executable

I tried a few other various combos and the doixosu toml wasm opt settings :/