Feature: extension support

[bpatrik]

This a tracking bug and "design doc" for the next mayor update that will be about extension support.

Requirement:

1. Users should be able extend the app without rebuilding the application
   • app restart after a new extension install is acceptable
   • "extending the app" mean business logic changes on both server and client-side
   • extensions should also support at least some UI modifications
2. Extensions should be configurable (should show in the settings)
3. There should be a repository where users can easily download extensions from. Preferably integrated in the app.

Design ideas

Extension will live in their own folder, next to the config.json file. config folder is by default writable from the docker setup, so it should be easy to add there more files.
It will be written in Javascript (I hope its not a question :) ).

server side extension

Creating a server side extension is the easy part. The app will assume a presence of an server.js that has a function init(app) function. App will call the init function of all extensions on startup.
If the folder has package.json, the app will run an npm isntall so that the extension developers can use external libraries (like if someone wants to develop an extension to support bmp files).
The appobject in the init function will represent an interface to the app, where extension developers can subscribe to certain events of the app. These events will be, like: before directory scanning or after directory scanning. With this developers can override the input and the output of the directory scanning event. (So things like also indexing doc files or skipping files that start with _ can be added).
Furthermore app object will also export low level api. Low level API will be less stable than the exported events, but with that developers can override any aspect of the app without restriction.
app object will also export the currently used config.

client side extensions

This is the tricky one as the client side is built and minified and localized AOT with webpack. Therefore there is one build app for all language. Exposing the full client app to the extension is not an option as its is not possible to know in extension development time how webapck will minify the function names.
I also could not find a way to add new angular component to the app after compile. Eg.: So I do not see a way to add UI for supporting docx files on UI through extensions.

I was thinking about exposing callback for the client extension, where developers can add icons or buttons in a very restrictive way to the app. And also here could the developer inject some business logic to the client.

repository

I do not want to host a custom repository. I think I will just "hack" it into a readme file. Developers will maintain extensions on github and add a link to their repo to to a readme file in the repository. This way the app will be able to auto discover the list of extensions.

Future work

Security can be implemented in the future where the app makes it explicit what resources an extension uses. Like it changes the config or touches low-level APIs.

Open questions

I don't really know how to solve the client side extension support properly

Sub tasks

• Create server side extension API #753
• Create extension repo #783
• Create settings panel to add extensions #784
• Create client side extension API

[mblythe86]

I like this idea a lot, and I think it has a lot of potential to serve as a bridge between what you're developing pigallery2 for (as you lay out in #738) and what other users are wanting.

As far as I can tell, the biggest philosophical difference I have is with this concept in #738:

Source of truth is the disk: DB is only a cache

In my particular case, there is an additional source of truth: the DigiKam DB. This is where I mark individual directories to be "Public" (i.e. displayed on pigallery2), and where I manually select directory covers/previews/thumbnails/highlights.

Allowing me to code up a custom extension that intervenes in the "Directory Scanning" and "Preview Filling" jobs would be pretty much a perfect solution.

[bpatrik]

Added backend extension support (in alpha version). So far the app looks for extension subfolders in the /extensions folder.
to add an extension you can:

<pigallery src root>/extensions/myextension/package.js <- this is optional. You can add extra npm packages here
<pigallery src root>/extensions/myextension/server.js <- this is needed

sample server js:

/* eslint-disable @typescript-eslint/no-inferrable-types */
import {IExtensionObject} from '../../src/backend/model/extension/IExtension';
import {PhotoMetadata} from '../../src/common/entities/PhotoDTO';
import {Column, Entity, Index, PrimaryGeneratedColumn} from 'typeorm';
import {SubConfigClass} from 'typeconfig/src/decorators/class/SubConfigClass';
import {ConfigProperty} from 'typeconfig/src/decorators/property/ConfigPropoerty';
@Entity()
class TestLoggerEntity {
  @Index()
  @PrimaryGeneratedColumn({unsigned: true})
  id: number;

  @Column()
  text: string;
}


@SubConfigClass({softReadonly: true})
export class TestConfig {
  @ConfigProperty({
    tags: {
      name: `Test config value`,
    },
    description: `This is just a description`,
  })
  cleanUpUnusedTables: boolean = true;
}

export const init = async (extension: IExtensionObject<TestConfig>): Promise<void> => {
  extension.Logger.debug('Setting up');

// override metadata loader example
  extension.events.gallery.MetadataLoader
    .loadPhotoMetadata.before(async (input, event) => {
    extension.Logger.debug('skipping');
    event.stopPropagation = true;
    return {
      size: {width: 1, height: 1},
      fileSize: 1,
      creationDate: 0
    } as PhotoMetadata;
  });
  extension.config.setTemplate(TestConfig);
  console.log(JSON.stringify(extension._app.config.Extensions));

  // add new res API example
  extension.RESTApi.get.jsonResponse(['/hw'], null, async () => {
    const conn = await extension.db.getSQLConnection();
    conn.getRepository(TestLoggerEntity).save({text: 'logger entry' + Date.now()});


    return await conn.getRepository(TestLoggerEntity).find();
  });

// set own tables
  await extension.db.setExtensionTables([TestLoggerEntity]);
  
  
  // add messenger example
  extension.messengers.addMessenger('loggerMsg',
    [{
      id: 'text',
      type: 'string',
      name: 'just a text',
      description: 'nothing to mention here',
      defaultValue: 'I hand picked these photos just for you:',
    }],
    {
      sendMedia: async (c, m) => {
        console.log(m);
      }
    });
};

export const cleanUp = async (extension: IExtensionObject<TestConfig>): Promise<void> => {
  extension.Logger.debug('Cleaning up');
};

See https://github.com/bpatrik/pigallery2/blob/master/src/backend/model/extension/IExtension.ts for full extension API

[bpatrik]

a more in depth documentation and easier usage comes later

[lukx]

Kudos @bpatrik for this extensive new feature, I am loving it. Thank you for staying innovative!