Multi-Tenant Multi-Database Modular Software As A Service

A modular approach to create a multi-tenant API as a service with multiple databases. Based on DBStalker and miniRouter.

Setup

To run the app, you need to install Docker and Docker Compose. After you are done you can start the app using docker-compose up

Configure Databases

• Create a new secrets/db_root_password.secret file with only the default db root password in it.
• Create a new secrets/config.json.secret with the same structure as secrets/config.json.secret.example.
• Use DBStalker configuration options to store the connection options under tenants in config.json.secret.
• Optional: add a salt for databases passwords: "dbSalt": "yourUniqueSalt" in config.json.secret.

See secrets/config.json.secret.example for example.

Register tenants in database

Add tenants' information as main seeds in tenants.seed.php, each tenant with unique:

• id: Will be used to access the tenants API and modules
• name: Will be used as a database name
• user: Will be used as a database username to access the tenant database
• password: Will be hashed and salted to be used as a password for the database user created
• per_day_backups The maximum number of backups that can be taken per day, default to 1
• max_backups The maximum number of backups that can be taken, default to 10

The tenant's APIs will be added under the unique tenant URL: /{tenant}

Reusable modules

Modules are pieces or reusable code that defines your business logic You can use modules to define:

• Database tables and views
• API endpoints
• Controllers, interfaces, business logic, etc.

To create a new module with name unique_module_name:

• Create a new directory called unique_module_name under modules directory
• Create a new directory for each version of the module. Add directory v1 in your newly created module directory. More on module versionning later
• Create a root module file called unique_module_name.module.php inside each version directory. This file will be included for each tenant subscribed to this version of the module

Modules versionning

Modules work ONLY in versions. Versions are used to introduce breaking changes to a module without breaking old versions. Each module version is treated as a separate moduleby the system.

Recommendations while using modules (unless you know what you are doing):

• A new version should ideally be introduced if it's uncompatible with the old version.
• Tenants should ideally subscribe to 1 version of each module to avoid conflict in database.

Create a new version

To create a new version, create a directory called v{base_version_number} under the module directory, where {base_version_number} is the major version number obtained by flooring the version number to a single integer.

Right version directories:

• v1
• v4
• v64

Wrong version directories:

• 1
• V1
• v1.1
• v1z

Sub-versions

Although only major version numbers can be created, sub-versions can be accessed in the underlying major version. The full version number can be access through $version_number variable.

For example: versions 1, 1.4, 1.0.1 all access v1 module and the full version can be accessed through $version_number variable.

The module version's APIs will be added under the unique URL: /{tenant_id}/{unique_module_name}/v{version_number}

Register modules in database

Add modules' information as main seeds in modules.seed.php, each module with unique:

• id: Will be used to connect tenants to modules
• name: The same name used for the module's directory name in modules

Modules dependancies

Some modules might depend on other modules. For each module dependancy, add a main seeds in modules_dependencies.seed.php with:

• id: Just a unique identifier required for main seeds, not used
• module_id The id of the original module that you need to define a dependancy to
• dependency_module_id: The id of the module it depends on
• version The major version number of the module module it depends on, default to 1

BE CAREFUL!! Since modules won't load until all of their dependancies are loaded, If 2 (or more) modules shouldn't depend on each other as this might cause an infinite loop!

e.g. if module 1 depends on module 2 and module 2 depends on module 1. If you try to load module 1 this loading sequence would happed: 1 -> 2 -> 1 -> 2 -> 1 -> 2 -> 1 -> ... causing an infinite loop!

Also this kind of 'dependancy loop' might happen for more than 2 modules.

e.g. if module 1 depends on module 2, module 2 depends on module 3 and module 3 depends on module 1. If you try to load module 1 this loading sequence would happed: 1 -> 2 -> 3 -> 1 -> 2 -> 3 -> 1 -> 2 -> ... causing an infinite loop!

Subscribe tenants to modules

For each module a tenant should subscribe to, add a main seeds in subscriptions.seed.php with:

• id: Just a unique identifier required for main seeds, not used
• tenant_id: The id of the tenant that will subscribe to the module
• module_id The id of the module to be subscribed to
• version The major version number of the module that should be used, default to 1

Database migration

In order to migrate changes in tables' structure and seeds. You can either:

• Set AUTO_MIGRATION in settings/constants.php to true
• Set AUTO_MIGRATION in settings/constants.php to false and migrate manually using the following routes:
  • To migrate the master tenants database, send a get request to: /migrate
  • To migrate a tenant database, send a get request to: /{tenant_id}/migrate
  • To force seed main seeds, send the request to /migrate/force
  • To seed temporary seeds, send the request to /migrate/seed
  • To remove temporary seeds, send the request to /migrate/deseed

Note: renaming columns or tables might result DROPPING the column or table entirely. so be careful!

Cronjobs

M3saas comes with dynamic cronjobs creation out of the box that modules can use to run functions daily, weekly, monthly or yearly.

Cronjobs setup

If you are using docker, You don't need to do anything to use dynamically created cronjobs. However, if you are not using docker, you should add a single cronjob manually to your server that runs app/bin/m3saas_cron.sh daily. An example of such cronjob can be found in app/m3saas_cronjob.

Register a cronjob

To add a cronjob a module:

• Create a crobjob entry file for the module at <module-name>/v<module-version>/<module-name>.cron.php
• Register a cron job using register_crobjob function with a uniqie name and closure

Here is the register_crobjob function definition:

register_crobjob(
    string $job_name,
    closure $job_fn,
    string $repeat = 'day',
    int $repeat_interval = 1,
    int $day = null,
    int $month = null
);

Cron jobs can be repeated every day, week, month or year. Here are some examples:

// Fire daily
register_crobjob('daily_cronjob', function() {}, 'day');
// Fire every 2 days
register_crobjob('every_3_days_cronjob', function() {}, 'day', 2);

// Fire weekly
register_crobjob('every_3_days_cronjob', function() {}, 'week');
// Fire every 6 weeks
register_crobjob('every_3_days_cronjob', function() {}, 'week', 6);

// Fire monthly
register_crobjob('every_3_days_cronjob', function() {}, 'month');
// Fire every 3 months
register_crobjob('every_3_days_cronjob', function() {}, 'month', 3);
// Fire every 3 months on day 1
register_crobjob('every_3_days_cronjob', function() {}, 'month', 3, 1);
// Fire monthly on day 15
register_crobjob('every_3_days_cronjob', function() {}, 'month', 1, 15);

// Fire yearly
register_crobjob('every_3_days_cronjob', function() {}, 'year');
// Fire every 5 years
register_crobjob('every_3_days_cronjob', function() {}, 'year', 5);
// Fire every 3 years on day 1 month 1
register_crobjob('every_3_days_cronjob', function() {}, 'year', 3, 1, 1);
// Fire yearly on day 1 month 7
register_crobjob('every_3_days_cronjob', function() {}, 'year', 1, 1, 7);

Cronjobs intervals are calculated from the last time the job was fired. Cronjobs get fired once if they were never fired before unless you specify an exact day or month for it to be fired on. You can't specify a day for jobs repeated every day or week.

MQTT Connection

The app comes bundled with HiveMQ MQTT Broker so the server can push messages to clients and the clients can communicate directly outside the server. If you are new to MQTT, we recomment reading MQTT Essentials first.

Define broker credentials

You can create unique users for your different tanents or applications using HiveMQ file RBAC extensions. To add a new user, add the username and password to hive-mq/extensions/hivemq-file-rbac-extension/credentials.xml, here is an example:

<users>
    <user>
        <name>app_user</name>
        <!-- Hash for: passs -->
        <password>R2l5d0tqTldZM3NtejdET3hkWHRBT0dxcFF6U1owRFk=:100:1MMjPJ2uOzC4cXx2SHNNMFwN2wo95TQVqcEAK9a3sC+QoblK+6UEqR/TA9W3ZVYQpzcaGiE7FCse7RFumxFdBA==</password>
        <roles>
            <id>scoped-role</id>
        </roles>
    </user>
</users>

You can find the roles available defined under <roles> in the same file. You can create your own roles if needed. To know more about adding users and roles refer to the extension's Credentials configuration docs.

Hashing passwords

All passwords for broker users are hashed, you can switch to plain text passwords by editing extension-config.xml but it's not recommended. To get a password hash:

• Start the hive-mq docker container, and get the container name
• In the project root, run sh ./hive-mq/bin/hash_pass.sh <container-name> <password-to-hash>

e.g. For a container with name m3saas_hive-mq_1 and a password passs:

$ sh ./hive-mq/bin/hash_pass.sh m3saas_hive-mq_1 passs

Add the following string as password to your credentials configuration file:
----------------------------------------------------------------------------
SXhuVzlpQ2tVUm5YVGZMVUJFaXZkbzZCV25UMnVKclQ=:100:8EaVMexR+jt45qicY35j7IDMpLeAkWQfS6uvXg2SsnX3+W09TMJaKJZy97wkvSaIkJZjCeAkCHb7G3qUxBZhWA==

Connect the server to the broker

To connect the server with the broker, you need to define your MQTT credentials inside secrets/config.json.secret:

{
    "mqtt": {
        "host": "hive-mq",
        "port": "1883",
        "username": "xxx",
        "password": "xxx"
    }
}

You should change the host and port according to your setup. If you don't want to use MQTT at all you can remove the mqtt property all together.

Publish messages from server

The server should only publish messages to clients, If a client wants to publish a message to the server, it should use the exposed HTTP API by the server. To publish a message from the server to clients:

• Get an instance from the MQTT client: $client = MQTT_Client::instance();
• Publish your message using $client->publish(string $topic, string $payload, int $qos = 0, bool $retain = false)

e.g.

// Get an instance of the MQTT client
$mqtt = MQTT_Client::instance();
// Publish an unretained message to topic "hello/world" message "Hello from the server!" with QoS 0
$mqtt->publish("hello/world", "Hello from the server!");

Config file fallback

The app gets its config file path ferom environment varialbe CONFIG_FILE defined in docker-compose, which points to secrets/config.json.secret. If you are using the app outside docker or wants to add a fallback for the config file, Add your configuration to config.json inside the 'app' directory.

Future Features

Bellow is a list of features or tasks to do in the future:

• Manage modules dependancies
• Disallow cronjobs to access all tanents data

License