Automatically enable openid listeners when ma1sd enabled

ma1sd requires the openid endpoints for certain functionality.
Example: https://github.com/ma1uta/ma1sd/blob/90b2b5301c34168346fdc5e7eccc09d6958e999f/src/main/java/io/kamax/mxisd/auth/AccountManager.java#L67-L99

If federation is disabled, we still need to expose these openid APIs on the
federation port.

Previously, we were doing similar magic for Dimension.
As per its documentation, when running unfederated, one is to enable
the openid listener as well. As per their recommendation, people
are advised to do enable it on the Client-Server API port
and use the `federationUrl` variable to override where the federation
port is (making federation requests go to the Client-Server API).

Because ma1sd always uses the federation port (unless you do some
DNS overwriting magic using its configuration -- which we'd rather not
do), it's better if we just default to putting the `openid` listener
where it belongs - on the federation port.

With this commit, we retain the "automatically enable openid APIs" thing
we've been doing for Dimension, but move it to the federation port instead.
We also now do the same thing when ma1sd is enabled.

Author: spantaleev

CHANGELOG.md

@@ -1,3 +1,15 @@
+# 2020-12-08
+
+## openid APIs exposed by default on the federation port when federation disabled
+
+We've changed some defaults. People running with our default configuration (federation enabled), are not affected at all.
+
+If you are running an unfederated server (`matrix_synapse_federation_enabled: false`), this may be of interest to you.
+
+When federation is disabled, but ma1sd or Dimension are enabled, we'll now expose the `openid` APIs on the federation port.
+These APIs are necessary for some ma1sd features to work. If you'd like to prevent this, you can: `matrix_synapse_federation_port_openid_resource_required: false`.
+
+
 # 2020-11-27
 
 ## Recent Jitsi updates may require configuration changes

docs/configuring-playbook-dimension.md

@@ -3,6 +3,9 @@
 **[Dimension](https://dimension.t2bot.io) can only be installed after Matrix services are installed and running.**
 If you're just installing Matrix services for the first time, please continue with the [Configuration](configuring-playbook.md) / [Installation](installing.md) flow and come back here later.
 
+**Note**: enabling Dimension, means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible).
+
+
 ## Prerequisites
 
 This playbook now supports running [Dimension](https://dimension.t2bot.io) in both a federated and an [unfederated](https://github.com/turt2live/matrix-dimension/blob/master/docs/unfederated.md) environment. This is handled automatically based on the value of `matrix_synapse_federation_enabled`.
@@ -48,7 +51,7 @@ To get an access token for the Dimension user, you can follow one of two options
 3. Copy the highlighted text to your configuration.
 4. Close the private browsing session. **Do not log out**. Logging out will invalidate the token, making it not work.
 
-*With CURL* 
+*With CURL*
 
 ```
 curl -X POST --header 'Content-Type: application/json' -d '{

docs/configuring-playbook-federation.md

@@ -37,3 +37,13 @@ matrix_synapse_federation_enabled: false
 ```
 
 With that, your server's users will only be able to talk among themselves, but not to anyone who is on another server.
+
+**Disabling federation does not necessarily disable the federation port** (`8448`). Services like [Dimension](configuring-playbook-dimension.md) and [ma1sd](configuring-playbook-ma1sd.md) normally rely on `openid` APIs exposed on that port. Even if you disable federation and only if necessary, we may still be exposing the federation port and serving the `openid` APIs there. To override this and completely disable Synapse's federation port use:
+
+```yaml
+# This stops the federation port on the Synapse side (normally `matrix-synapse:8048` on the container network).
+matrix_synapse_federation_port_enabled: false
+
+# This removes the `8448` virtual host from the matrix-nginx-proxy reverse-proxy server.
+matrix_nginx_proxy_proxy_matrix_federation_api_enabled: false
+```

docs/configuring-playbook-ma1sd.md

@@ -4,7 +4,9 @@ By default, this playbook configures an [ma1sd](https://github.com/ma1uta/ma1sd)
 
 This server is private by default, potentially at the expense of user discoverability.
 
-ma1sd is a fork of [mxisd](https://github.com/kamax-io/mxisd) which was pronounced end of life 2019-06-21.
+*ma1sd is a fork of [mxisd](https://github.com/kamax-io/mxisd) which was pronounced end of life 2019-06-21.*
+
+**Note**: enabling ma1sd (which is also the default), means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible).
 
 
 ## Disabling ma1sd
@@ -50,6 +52,9 @@ To use the [Registration](https://github.com/ma1uta/ma1sd/blob/master/docs/featu
 
 - `matrix_ma1sd_configuration_extension_yaml` - to configure ma1sd as required. See the [Registration feature's docs](https://github.com/ma1uta/ma1sd/blob/master/docs/features/registration.md) for inspiration. Also see the [Additional features](#additional-features) section below to learn more about how to use `matrix_ma1sd_configuration_extension_yaml`.
 
+**Note**: For this to work, either the homeserver needs to [federate](configuring-playbook-federation.md) or the `openid` APIs need to exposed on the federation port. When federation is disabled and ma1sd is enabled, we automatically expose the `openid` APIs (only!) on the federation port. Make sure the federation port (usually `https://matrix.DOMAIN:8448`) is whitelisted in your firewall (even if you don't actually use/need federation).
+
+
 ## Authentication
 
 [Authentication](https://github.com/ma1uta/ma1sd/blob/master/docs/features/authentication.md) provides the possibility to use your own [Identity Stores](https://github.com/ma1uta/ma1sd/blob/master/docs/stores/README.md) (for example LDAP) to authenticate users on your Homeserver. The following configuration can be used to authenticate against an LDAP server:

docs/prerequisites.md

@@ -22,6 +22,17 @@ This playbook doesn't support running on ARM (see [this issue](https://github.co
 
 - Properly configured DNS records for `<your-domain>` (details in [Configuring DNS](configuring-dns.md)).
 
-- Some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: `80/tcp` (HTTP webserver), `443/tcp` (HTTPS webserver), `3478/tcp` (TURN over TCP), `3478/udp` (TURN over UDP), `5349/tcp` (TURN over TCP), `5349/udp` (TURN over UDP), `8448/tcp` (Matrix Federation API HTTPS webserver), the range `49152-49172/udp` (TURN over UDP), `4443/tcp` (Jitsi Harvester fallback), `10000/udp` (Jitsi video RTP). Depending on your firewall/NAT setup, incoming RTP packets on port 10000 may have the external IP of your firewall as destination address, due to the usage of STUN in JVB (see [`matrix_jitsi_jvb_stun_servers`](../roles/matrix-jitsi/defaults/main.yml)).
+- Some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports:
+
+  - `80/tcp`: HTTP webserver
+  - `443/tcp`: HTTPS webserver
+  - `3478/tcp`: TURN over TCP (used by Coturn)
+  - `3478/udp`: TURN over UDP (used by Coturn)
+  - `5349/tcp`: TURN over TCP (used by Coturn)
+  - `5349/udp`: TURN over UDP (used by Coturn)
+  - `8448/tcp`: Matrix Federation API HTTPS webserver. In some cases, this **may necessary even with federation disabled**. Integration Servers (like Dimension) and Identity Servers (like ma1sd) may need to access `openid` APIs on the federation port.
+  - the range `49152-49172/udp`: TURN over UDP
+  - `4443/tcp`: Jitsi Harvester fallback
+  - `10000/udp`: Jitsi video RTP. Depending on your firewall/NAT setup, incoming RTP packets on port `10000` may have the external IP of your firewall as destination address, due to the usage of STUN in JVB (see [`matrix_jitsi_jvb_stun_servers`](../roles/matrix-jitsi/defaults/main.yml)).
 
 When ready to proceed, continue with [Configuring DNS](configuring-dns.md).

group_vars/matrix_servers

@@ -620,8 +620,6 @@ matrix_dimension_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_ena
 matrix_integration_manager_rest_url: "{{ matrix_dimension_integrations_rest_url if matrix_dimension_enabled else None }}"
 matrix_integration_manager_ui_url: "{{ matrix_dimension_integrations_ui_url if matrix_dimension_enabled else None }}"
 
-matrix_dimension_homeserver_federationUrl: "http://matrix-synapse:{{ 8048 if matrix_synapse_federation_enabled|bool else 8008 }}"
-
 ######################################################################
 #
 # /matrix-dimension
@@ -816,7 +814,8 @@ matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container: "matrix-ma1sd:
 matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container: "127.0.0.1:8090"
 
 # By default, we do TLS termination for the Matrix Federation API (port 8448) at matrix-nginx-proxy.
-matrix_nginx_proxy_proxy_matrix_federation_api_enabled: true
+# Unless this is handled there OR Synapse's federation listener port is disabled, we'll reverse-proxy.
+matrix_nginx_proxy_proxy_matrix_federation_api_enabled: "{{ matrix_synapse_federation_port_enabled and not matrix_synapse_tls_federation_listener_enabled }}"
 matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-synapse:8048"
 matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "127.0.0.1:8048"
 
@@ -990,12 +989,7 @@ matrix_synapse_tls_federation_listener_enabled: false
 matrix_synapse_tls_certificate_path: ~
 matrix_synapse_tls_private_key_path: ~
 
-matrix_synapse_http_listener_resource_names: |
-  {{
-    ["client"]
-    +
-    ( ["openid"] if matrix_dimension_enabled and not matrix_synapse_federation_enabled else [] )
-  }}
+matrix_synapse_federation_port_openid_resource_required: "{{ not matrix_synapse_federation_enabled and (matrix_dimension_enabled or matrix_ma1sd_enabled) }}"
 
 matrix_synapse_email_enabled: "{{ matrix_mailer_enabled }}"
 matrix_synapse_email_smtp_host: "matrix-mailer"

roles/matrix-synapse/defaults/main.yml

@@ -136,6 +136,11 @@ matrix_synapse_tls_private_key_path: "/data/{{ matrix_server_fqn_matrix }}.tls.k
 # names.
 matrix_synapse_http_listener_resource_names: ["client"]
 
+# Resources served on Synapse's federation port.
+# When disabling federation, we may wish to serve the `openid` resource here,
+# so that services like Dimension and ma1sd can work.
+matrix_synapse_federation_listener_resource_names: "{{ ['federation'] if matrix_synapse_federation_enabled else (['openid'] if matrix_synapse_federation_port_openid_resource_required else []) }}"
+
 # Enable this to allow Synapse to report utilization statistics about your server to matrix.org
 # (things like number of users, number of messages sent, uptime, load, etc.)
 matrix_synapse_report_stats: false
@@ -211,10 +216,25 @@ matrix_synapse_caches_global_factor: 0.5
 
 # Controls whether Synapse will federate at all.
 # Disable this to completely isolate your server from the rest of the Matrix network.
-# Also see: `matrix_synapse_tls_federation_listener_enabled` if you wish to keep federation enabled,
+#
+# Disabling this still keeps the federation port exposed, because it may be used for other services (`openid`).
+#
+# Also see:
+# - `matrix_synapse_tls_federation_listener_enabled` if you wish to keep federation enabled,
 # but want to stop the TLS listener (port 8448).
+# - `matrix_synapse_federation_port_enabled` to avoid exposing the federation ports
 matrix_synapse_federation_enabled: true
 
+# Controls whether the federation ports are used at all.
+# One may wish to disable federation (`matrix_synapse_federation_enabled: true`),
+# but still run other resources (like `openid`) on the federation port
+# by enabling them in `matrix_synapse_federation_listener_resource_names`.
+matrix_synapse_federation_port_enabled: "{{ matrix_synapse_federation_enabled or matrix_synapse_federation_port_openid_resource_required }}"
+
+# Controls whether an `openid` listener is to be enabled. Useful when disabling federation,
+# but needing the `openid` APIs for Dimension or an identity server like ma1sd.
+matrix_synapse_federation_port_openid_resource_required: false
+
 # A list of domain names that are allowed to federate with the given Synapse server.
 # An empty list value (`[]`) will also effectively stop federation, but if that's the desired
 # result, it's better to accomplish it by changing `matrix_synapse_federation_enabled`.

roles/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -188,7 +188,7 @@ listeners:
       - '0.0.0.0'
 {% endif %}
 
-{% if matrix_synapse_federation_enabled and matrix_synapse_tls_federation_listener_enabled %}
+{% if matrix_synapse_federation_port_enabled and matrix_synapse_tls_federation_listener_enabled %}
   # TLS-enabled listener: for when matrix traffic is sent directly to synapse.
   - port: 8448
     tls: true
@@ -197,7 +197,7 @@ listeners:
     x_forwarded: false
 
     resources:
-      - names: [federation]
+      - names: {{ matrix_synapse_federation_listener_resource_names|to_json }}
         compress: false
 {% endif %}
 
@@ -213,7 +213,7 @@ listeners:
       - names: {{ matrix_synapse_http_listener_resource_names|to_json }}
         compress: false
 
-{% if matrix_synapse_federation_enabled %}
+{% if matrix_synapse_federation_port_enabled %}
   # Unsecure HTTP listener (Federation API): for when matrix traffic passes through a reverse proxy
   # that unwraps TLS.
   - port: 8048
@@ -223,7 +223,7 @@ listeners:
     x_forwarded: true
 
     resources:
-      - names: [federation]
+      - names: {{ matrix_synapse_federation_listener_resource_names|to_json }}
         compress: false
 {% endif %}