feat (web/server) 360 degrees Web panoramas [attempt 2] #3412

dmitry-brazhenko · 2023-07-24T09:38:08Z

Initial PR: #3098

In this PR I aded

Support for 360 Panoramas as it was discussed here: [Feature]: Panoramas: Use PhotoSphere Viewer to display photos with equirectangular projection #1630

How it was implemented

Once an images is uploaded, in metadata-extraction.processor.ts it checks aspectRatio. We check exif (ProjextionType) property to determine if it is a panorama.
If it is a panorama, we add a property projectionType as Enum (none/EQUIRECTANGULAR/...)
We use library @egjs/svelte-view360 to show the panorama
In the library we use a special symbol to display if a thumb is a panorama.

Potential further improvements:

Probably we can switch library and use smth else. But this liibrary works OK both on desktop and mobile-web
Support panoramas in Flutter mobile app
There 360 videos. Later we can support them as well

Here is thumb that shows a panorama:

Panorama after is opened:

vercel · 2023-07-24T09:38:18Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
immich	⬜️ Ignored (Inspect)			Jul 27, 2023 6:16pm

dmitry-brazhenko · 2023-07-24T09:56:13Z

@alextran1502 here is a new PR as you asked here #3098

Need your help to merge it.

I added fix: changed "isPanorama" property to ProjectionType enum property.

server/src/infra/migrations/1690140002097-Panoramas.ts

web/src/lib/components/asset-viewer/asset-viewer.svelte

web/src/lib/components/asset-viewer/panorama-viewer.svelte

server/src/infra/entities/exif.entity.ts

server/src/infra/entities/asset.entity.ts

server/src/domain/asset/response-dto/asset-response.dto.ts

server/src/infra/migrations/1690140002097-Panoramas.ts

dmitry-brazhenko · 2023-07-24T18:27:03Z

@brighteyed Hope I fixed all suggestions :)

dmitry-brazhenko · 2023-07-24T18:52:43Z

Need help with creating documentation and merging the change

web/src/lib/components/assets/thumbnail/thumbnail.svelte

web/src/lib/components/asset-viewer/asset-viewer.svelte

server/src/domain/asset/response-dto/exif-response.dto.ts

server/src/infra/entities/exif.entity.ts

server/src/microservices/processors/metadata-extraction.processor.ts

Co-authored-by: Sergey Kondrikov <[email protected]>

Co-authored-by: Jason Rasmussen <[email protected]>

…sor.ts Co-authored-by: Jason Rasmussen <[email protected]>

dmitry-brazhenko · 2023-07-27T15:15:37Z

@jrasm91 I changed column to varchar

Hope it is ready to be merged now :)
@alextran1502 could you help to regenerate docs properly and merge the change?

Thanks

dmitry-brazhenko · 2023-07-27T15:46:07Z

There is a problem with code prettiffier locally. Once I save file in visual studio, it applies some format that is not accepted by checker :(

dmitry-brazhenko · 2023-07-27T15:48:40Z

Only one migration file is necessary

server/package.json

server/src/infra/entities/exif.entity.ts

jrasm91 · 2023-07-27T16:29:05Z

server/src/microservices/processors/metadata-extraction.processor.ts

+    const exifProjectionType = getExifProperty('ProjectionType');
+    let projectionType: ProjectionType = ProjectionType.NONE;
+
+    if (exifProjectionType) {
+      const exifPropertyUpper = exifProjectionType.toUpperCase();
+
+      if (exifPropertyUpper in ProjectionType) {
+        projectionType = exifPropertyUpper as ProjectionType;
+      }
+    }
+
+    newExif.projectionType = projectionType;


Suggested change

const exifProjectionType = getExifProperty('ProjectionType');

let projectionType: ProjectionType = ProjectionType.NONE;

if (exifProjectionType) {

const exifPropertyUpper = exifProjectionType.toUpperCase();

if (exifPropertyUpper in ProjectionType) {

projectionType = exifPropertyUpper as ProjectionType;

}

}

newExif.projectionType = projectionType;

const projectionType = getExifProperty('ProjectionType');

if (projectionType) {

newExif.projectionType = String(projectionType).toUpperCase();

}

Let's just store the value always, regardless if it matches the enum. We can update the enum in the future if we discover a new valid value.

newExif.projectionType is a enum. What will happen in case String(projectionType).toUpperCase(); does not exist in this enum?

The database type is a varchar, right? The entity type should be ProjectionType | string.

If we only accept valid values we should use pg enum. If we accept other values we should use a varchar and store the raw value (maybe transformed to uppercase). IMO the projection type is just a hint of what valid types are available to be used, but the value is just a string. I don't know if it is possible/easy to use enum in the web and flutter without getting an error for other values.

I think everything would be quite a bit easier if you just stored it as a string everywhere in the server codebase, and had the property in the response dto be a string.

You can define the enum in the web to use for your if statements. Happy to discuss more in discord if you want to chat there instead of back and forth here.

Actualy i liked enum . I would suggested to keep enum | str. I changed it here: a06ee84

I would suggest to keep enum | str.

As far as I understood, there are just 2 most popular projection types:equirectangular and cylinder

…into web_panoramas_2

jrasm91

There was some back and forth when it came to using a projection type enum server-side. In short, the server doesn't even use the value anywhere. The only place it is used currently is in the UI. Everything is a lot simpler if this is stored and treated as a string and the enum exists in the web for this specific panorama case. Javascript isn't typed and can load arbitrary values, but flutter is.

brighteyed

This PR doesn't check the aspect ratio anymore. @dmitry-brazhenko, please update the PR description

dmitry-brazhenko · 2023-07-28T06:12:50Z

🎉

MingchenZhang · 2024-04-25T17:54:01Z

Is there a way to add the property projectionType to post upload? I think there might be some issue with the metadata of my original panorama image that it is not showing up as panorama in immich. It would be nice to (force) show them as panorama in immich.

dmitry-brazhenko · 2024-04-25T18:14:33Z

I would say this could make sense.

Most likely we could add it here:

dmitry-brazhenko added 4 commits July 23, 2023 22:40

commit 1 (isPanorama: boolean)

ee7828c

working solution for projectiontypeenum

4724c0e

fix

26f2c23

format fix

ccf8eac

fix

a356399

dmitry-brazhenko changed the title ~~Web panoramas 2~~ feat (web/server) 360 degrees Web panoramas [attempt 2] Jul 24, 2023

dmitry-brazhenko marked this pull request as ready for review July 24, 2023 09:42

fix

2aac848

dmitry-brazhenko mentioned this pull request Jul 24, 2023

feat (web/server) 360 degrees Web panoramas #3098

Closed

fix

f41cde2

bo0tzz reviewed Jul 24, 2023

View reviewed changes

server/src/infra/migrations/1690140002097-Panoramas.ts Outdated Show resolved Hide resolved

web/src/lib/components/asset-viewer/asset-viewer.svelte Outdated Show resolved Hide resolved

web/src/lib/components/asset-viewer/panorama-viewer.svelte Show resolved Hide resolved

brighteyed reviewed Jul 24, 2023

View reviewed changes

server/src/infra/entities/exif.entity.ts Show resolved Hide resolved

dmitry-brazhenko added 2 commits July 24, 2023 14:04

fix

6ffb2de

enum projectiontype

fe2ae21

brighteyed reviewed Jul 24, 2023

View reviewed changes

dmitry-brazhenko added 5 commits July 24, 2023 19:29

working solution with exif

c0ca82d

fix

c8d9031

reverted >

bd42563

fix format

4c06187

reverted auto-magic api.ts prettification

2605e23

dmitry-brazhenko added 2 commits July 24, 2023 20:32

fix

59972ca

reverted api.ts autogenerated

5761bb2

api ts regenerated

86e879d

brighteyed reviewed Jul 24, 2023

View reviewed changes

dmitry-brazhenko and others added 2 commits July 24, 2023 21:41

Update web/src/lib/components/assets/thumbnail/thumbnail.svelte

45c393a

Co-authored-by: Sergey Kondrikov <[email protected]>

Update web/src/lib/components/asset-viewer/asset-viewer.svelte

4bd289c

Co-authored-by: Sergey Kondrikov <[email protected]>

dmitry-brazhenko and others added 3 commits July 27, 2023 16:41

Update server/src/domain/asset/response-dto/exif-response.dto.ts

484f9f3

Co-authored-by: Jason Rasmussen <[email protected]>

Update server/src/microservices/processors/metadata-extraction.proces…

df7ab47

…sor.ts Co-authored-by: Jason Rasmussen <[email protected]>

enum -> varchar;projectiontypeenum->projectiontype

f164f6b

asset-viewer fixed prettiffier

a8bc93f

jrasm91 reviewed Jul 27, 2023

View reviewed changes

dmitry-brazhenko and others added 11 commits July 27, 2023 18:36

@column({}) single line

af4d808

enum | string

a06ee84

make api

cc1b9e1

enum | string

67c2611

enum | str fix

3fb39fb

fix

9b32ed7

chore: merge main

0e47f8d

Merge branch 'web_panoramas_2' of github.com:dmitry-brazhenko/immich …

644f7f4

…into web_panoramas_2

chore: use string instead of enum

321d07a

chore: open api

815b1a8

fix: checks

c5222fc

jrasm91 approved these changes Jul 27, 2023

View reviewed changes

jrasm91 requested review from alextran1502, bo0tzz and brighteyed July 27, 2023 18:28

brighteyed approved these changes Jul 27, 2023

View reviewed changes

alextran1502 merged commit e071b82 into immich-app:main Jul 28, 2023

dmitry-brazhenko deleted the web_panoramas_2 branch July 28, 2023 06:12

dmitry-brazhenko mentioned this pull request Feb 9, 2024

fix (web): panorama / 360 view - partial panorama: use photo-sphere-viewer #6992

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat (web/server) 360 degrees Web panoramas [attempt 2] #3412

feat (web/server) 360 degrees Web panoramas [attempt 2] #3412

dmitry-brazhenko commented Jul 24, 2023 •

edited

Loading

vercel bot commented Jul 24, 2023 •

edited

Loading

dmitry-brazhenko commented Jul 24, 2023

dmitry-brazhenko commented Jul 24, 2023 •

edited

Loading

dmitry-brazhenko commented Jul 24, 2023

dmitry-brazhenko commented Jul 27, 2023

dmitry-brazhenko commented Jul 27, 2023

dmitry-brazhenko commented Jul 27, 2023

jrasm91 Jul 27, 2023

jrasm91 Jul 27, 2023

dmitry-brazhenko Jul 27, 2023

jrasm91 Jul 27, 2023

jrasm91 Jul 27, 2023

jrasm91 Jul 27, 2023

dmitry-brazhenko Jul 27, 2023

dmitry-brazhenko Jul 27, 2023

jrasm91 left a comment

brighteyed left a comment

dmitry-brazhenko commented Jul 28, 2023

MingchenZhang commented Apr 25, 2024

dmitry-brazhenko commented Apr 25, 2024

feat (web/server) 360 degrees Web panoramas [attempt 2] #3412

feat (web/server) 360 degrees Web panoramas [attempt 2] #3412

Conversation

dmitry-brazhenko commented Jul 24, 2023 • edited Loading

vercel bot commented Jul 24, 2023 • edited Loading

dmitry-brazhenko commented Jul 24, 2023

dmitry-brazhenko commented Jul 24, 2023 • edited Loading

dmitry-brazhenko commented Jul 24, 2023

dmitry-brazhenko commented Jul 27, 2023

dmitry-brazhenko commented Jul 27, 2023

dmitry-brazhenko commented Jul 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrasm91 left a comment

Choose a reason for hiding this comment

brighteyed left a comment

Choose a reason for hiding this comment

dmitry-brazhenko commented Jul 28, 2023

MingchenZhang commented Apr 25, 2024

dmitry-brazhenko commented Apr 25, 2024

dmitry-brazhenko commented Jul 24, 2023 •

edited

Loading

vercel bot commented Jul 24, 2023 •

edited

Loading

dmitry-brazhenko commented Jul 24, 2023 •

edited

Loading