Installed desktop app with self-hosted instance: all images are black

[roughnecks]

If I hover over a black image box, a denied icon shows up: tried screenshotting it but it isn't displayed.

Opened debug logs and I see:

[rndr] [info] request failed - no response (get from my proxied domain) and also
[rndr] [info] request failed - no response (put to our S3 storage)

ente-museun container reports no warning or errors, just info while I'm trying to upload some pictures from the desktop app.
from my phone's app (Android 9) it looks like photos were uploaded and this is a shot from S3 browser, my bucket:

This is another shot from desktop app:

Some other logs from desktop app:

[2024-07-15T15:58:12+02:00] [rndr] [error] get DownloadManager preview Failed: AxiosError: Network Error
    at c.onerror (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:41:45992)
    at ty.request (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:41:50891)
    at async o.request (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:250299)
    at async l (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:257784)
    at async E.downloadThumbnail (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:202611)
    at async y.downloadThumb (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:199779)
    at async y.getThumbnail (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:196271)
    at async y.getThumbnailForPreview (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:196685)
    at async ente://app/_next/static/chunks/4681-92f32afb893b4847.js:1:55632
[2024-07-15T15:58:12+02:00] [rndr] [error] preview card useEffect failed: AxiosError: Network Error
    at c.onerror (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:41:45992)
    at ty.request (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:41:50891)
    at async o.request (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:250299)
    at async l (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:257784)
    at async E.downloadThumbnail (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:202611)
    at async y.downloadThumb (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:199779)
    at async y.getThumbnail (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:196271)
    at async y.getThumbnailForPreview (ente://app/_next/static/chunks/pages/_app-eb8207445e2bd7c7.js:25:196685)
    at async ente://app/_next/static/chunks/4681-92f32afb893b4847.js:1:55632
    ```

It says "network error" and since my home network and vps network (where museum is) both look fine, is it our S3 failing?

Thanks

[mnvr]

Did this start happening after you updated to the recent version? It sounds like it might be a CORS issue.

Previously, the desktop app would always override the ACAO ("Access-Control-Allow-Origin") header in the response to "*". This unfortunately broke other things (like payments), so now it only does the override if the actual response does not have a ACAO header.

Code: https://github.com/ente-io/ente/blob/main/desktop/src/main.ts#L546

If you open dev tools, you should see an OPTIONS network request for your thumbnail. This would be the request that is failing (thus preventing the thumbnail from being fetched). Can you see what is the value of the "Access-Control-Allow-Origin" value returned by your S3 bucket in this request? Or you could add a screenshot of the OPTIONS response too, that will help diagnose the issue.

[roughnecks]

I see CORS errors indeed, does this help?

[mnvr]

Folks have been using it with self hosted instances earlier too, but yes, it's the first release where self hosted instances can be used without building from source.

So the issue here is that your S3 bucket is likely configured to return some explicit domain for the Access-Control-Allow-Origin. This wouldn't work with the desktop app, and you'll need to configure your S3 bucket to return Access-Control-Allow-Origin "*".

If you're curious as to why we need to set it to "*", you can see this explanation in the code -

ente/desktop/src/main.ts

Line 504 in b2e4362

* Allow uploading to arbitrary S3 buckets.

This is in addition to allowing the 'X-Auth-Token' and 'X-Client-Package' headers, but I assume those you've already allowed since those are also needed by the web app (which I presume is working with your existing S3 buckets).

Summary:

Your S3 bucket needs to be configured with the following CORS settings:

• Access-Control-Allow-Origin: *
• Access-Control-Allow-Headers: X-Auth-Token, X-Client-Package

If the CORS preflight request still fails after configuring these, then look in the dev tools console - there will be a message there mentioning why the CORS request is not being allowed.

[roughnecks]

Your S3 bucket needs to be configured with the following CORS settings:

do you mean upstream by the S3 service provider?

[mnvr]

Yes. Your S3 provider will provide a way to configure CORS settings.

If you click on the "preflight" requests in dev tools, a panel will be shown with details about the request. Here you can see the response headers that are currently being sent:

[mnvr]

Here are some example CORS configurations:

• From a community guide - https://help.ente.io/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket
• From a previous discussion - How do I configure a Backblaze bucket for self-hosting? #1764 (comment)

[roughnecks]

This worked: https://help.ente.io/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket

thanks

[roughnecks]

Thank you,
will let you know if we manage to fix this.

[ggpepp41]

The Access-Control-Allow-Origin header from Backblaze is returning NULL for me. I believe this may be the same issue. I have set my CORS rules to allow all origins but still am getting null. Any ideas?

[mnvr]

Summary: The null is expected, but it shouldn't be causing any problems? But if it is, do let me know and I can add another workaround.

---

Details:

The reason you're getting a null is because there is a redirection involved, and browsers don't preserve the Origin when redirecting

The Fetch spec requires browsers to set the origin to a “globally unique identifier” (essentially the same thing as > “opaque origin”, which basically means null…) in one case: [Redirects across origins(https://fetch.spec.whatwg.org/#ref-for-concept-request-tainted-origin%E2%91%A0)

source

The redirect goes this way: the client asks museum for the file, and the museum redirects to your S3 provider. So your S3 provider will be getting null as the Origin in the request it gets. If you're just echoing it back, you will get back a null in the response.

To avoid these issues, you can set "*" as the ACAO in your S3 config. But you're saying you've already done that, so I'm not sure why this is happening.

We also have a (limited) workaround to make it easier for people to self host - https://github.com/ente-io/ente/blob/main/desktop/src/main.ts#L504 - essentially, we intercept the responses and set ACAO to "*" when it matches the default origin (rendererURL, "ente://app"). If your setup is correct and you're still getting a null as the ACAO response, and if that's what's preventing your images from showing up, I can add another case here - https://github.com/ente-io/ente/blob/main/desktop/src/main.ts#L546:

                    headers["Access-Control-Allow-Origin"] =
-                        value[0] == rendererURL ? ["*"] : value;
-                        (value[0] == rendererURL || value[0] == "null") ? ["*"] : value;

i.e. transform null ACAO to *. But I'd like to do this only if we're we think null is indeed the issue.

[ggpepp41]

Let me back up and change my response here. I am able to upload and download files while using the mobile app.
Thumbnails and Downloads work correctly when using the desktop app.
I am having a problem when trying to upload a file on the desktop application. I am receiving the following error:

Since everything works correctly on mobile, I assume that the CORS settings on my bucket are correct. Either way I will attach them below because you understand this a lot better than I do.

[ { "corsRuleName": "entephotos", "allowedOrigins": ["*"], "allowedHeaders": ["*"], "allowedOperations": [ "b2_download_file_by_id", "b2_download_file_by_name", "b2_upload_file", "b2_upload_part", "s3_get", "s3_post", "s3_put", "s3_head" ], "exposeHeaders": [ "ETag", "Access-Control-Allow-Origin" ], "maxAgeSeconds": 3600 } ]

[mnvr]

tl;dr; Can you try with the latest nightly build? That should fix it. You can download the nightly from here: https://github.com/ente-io/photos-desktop/releases

---

Details: Thanks for the clarifications. So uploads have a different flow. We get a URL from museum, and then the client directly uploads to that (pre-signed) URL, and there is no redirection involved.

The fact that it works on the mobile device is not relevant, because mobile apps don't use CORS.

From your screenshot we can see that the CORS request is failing because the ACAO header in the response is null, when it should be "*" since that's what you configured in your bucket CORS settings ("allowedOrigins": ["*"]).

I did a bit of digging. It seems that Backblaze returns null as the ACAO response if the scheme is not http(s), even if we have set allowedOrigins to "*". The desktop app has a custom scheme, "ente://app", which is thus causing B2 to return null.

To fix this, I've add the workaround I mentioned previously (translate null => "*").

The fix is in the nightly build. Let me know if it still doesn't work.

[ggpepp41]

This is perfect. Thank you for fixing it so fast.

[reorx]

Thanks! The nightly build also fixes the problem where the desktop with the self-hosted API could not upload files, even though existing photos could be displayed.

This is how the PUT request looks like now, previously there's no "Origin" in the headers, causing Electron to block the request with CORS error

[meysam81]

For anyone else getting here and still haven't had a working solution...

I was using Hetzner object storage as an External S3 Bucket.

At the time of writing, they didn't support modifying the CORS policy on the bucket unless you change the visibility to public first:

In order for Cross-Origin Resource Sharing (CORS) to work properly with the S3 Bucket, you must set the Bucket visibility to public. source

I switched to Backblaze but that didn't work for me either 🤷

I finally switched to Wasabi and now I can finally see the thumbnail of my albums in the Ente desktop app.
Apparently, even without modify the CORS policies on Wasabi buckets, the default CORS policy works just fine. ✅