Overrides not working correctly for webserver_config.py

[xeniape]

Affected Stackable version

Tested on 0.0.0-dev

Affected Apache Airflow version

No response

Current and expected behavior

Values in the webserver_config.py file / airflow-webserver ConfigMap are not being overridden properly using configOverrides.

Example Config setting configOverrides on role level and resulting webserver_config.py:

webservers:
  roleGroups:
    default:
      replicas: 1
  configOverrides:
    webserver_config.py:
      WTF_CSRF_ENABLED: "False"                                                                                                   
      AUTH_ROLES_SYNC_AT_LOGIN: "True"
      AUTH_TYPE: "AUTH_DB_TEST"
      AUTH_USER_REGISTRATION: "False"
      AUTH_USER_REGISTRATION_ROLE: "Admin"
      OAUTH_PROVIDERS: |
      [
        { 'name': 'azure',
          'icon': 'fa-windows',
          'token_key': 'access_token',
          'remote_app': {
            'client_id': os.environ.get('OIDC_XXX_CLIENT_ID'),
            'client_secret': os.environ.get('OIDC_XXX_CLIENT_SECRET'),
            'client_kwargs': {
              'scope': 'openid profile'
            },
            'api_base_url': 'https://keycloak/realms/sdp/protocol/test-url',
            'server_metadata_url': 'https://keycloak/realms/sdp/.well-known/openid-configuration-test',
          },
        }
       ]

webserver_config.py:                                                                                                                     
----                                                                                                                                     
import os                                                                                                                                
from flask_appbuilder.const import (AUTH_DB, AUTH_LDAP, AUTH_OAUTH, AUTH_OID, AUTH_REMOTE_USER)                                          
basedir = os.path.abspath(os.path.dirname(__file__))                                                                                     
WTF_CSRF_ENABLED = True                                                                                                                  
                                                                                                                                         
AUTH_ROLES_SYNC_AT_LOGIN = False                                                                                                         
AUTH_TYPE = AUTH_OAUTH                                                                                                                   
AUTH_USER_REGISTRATION = True                                                                                                            
AUTH_USER_REGISTRATION_ROLE = "User"                                                                                                     
OAUTH_PROVIDERS = [                                                                                                                      
{ 'name': 'keycloak',                                                                                                                    
  'icon': 'fa-key',                                                                                                                      
  'token_key': 'access_token',                                                                                                           
  'remote_app': {                                                                                                                        
    'client_id': os.environ.get('OIDC_BCFDE64522F1D73E_CLIENT_ID'),                                                                      
    'client_secret': os.environ.get('OIDC_BCFDE64522F1D73E_CLIENT_SECRET'),                                                              
    'client_kwargs': {                                                                                                                   
      'scope': 'openid'                                                                                                                  
    },                                                                                                                                   
    'api_base_url': 'https://keycloak/realms/sdp/protocol/',                                                                             
    'server_metadata_url': 'https://keycloak/realms/sdp/.well-known/openid-configuration',                                               
  },                                                                                                                                     
}                                                                                                                                        
]                                                                                                                                        
                                                                                                                                         
WTF_CSRF_ENABLED = False

(Maybe also noteable that WTF_CSRF_ENABLED appears twice now)

Example Config setting configOverrides on role group level and resulting webserver_config.py:

webservers:
  roleGroups:
    default:
      replicas: 1
      configOverrides:
        webserver_config.py:
          WTF_CSRF_ENABLED: "False"
          AUTH_ROLES_SYNC_AT_LOGIN: "True"
          AUTH_TYPE: "AUTH_DB_TEST"
          AUTH_USER_REGISTRATION: "False"
          AUTH_USER_REGISTRATION_ROLE: "Admin"
          OAUTH_PROVIDERS: |
          [
            { 'name': 'azure',
              'icon': 'fa-windows',
              'token_key': 'access_token',
              'remote_app': {
                'client_id': os.environ.get('OIDC_XXX_CLIENT_ID'),
                'client_secret': os.environ.get('OIDC_XXX_CLIENT_SECRET'),
                'client_kwargs': {
                  'scope': 'openid profile'
                },
                'api_base_url': 'https://keycloak/realms/sdp/protocol/test-url',
                'server_metadata_url': 'https://keycloak/realms/sdp/.well-known/openid-configuration-test',
              },
            }
          ]

webserver_config.py:                                                                                                                     
----                                                                                                                                     
import os                                                                                                                                
from flask_appbuilder.const import (AUTH_DB, AUTH_LDAP, AUTH_OAUTH, AUTH_OID, AUTH_REMOTE_USER)                                          
basedir = os.path.abspath(os.path.dirname(__file__))                                                                                     
WTF_CSRF_ENABLED = True                                                                                                                  
                                                                                                                                         
AUTH_ROLES_SYNC_AT_LOGIN = False                                                                                                         
AUTH_TYPE = AUTH_OAUTH                                                                                                                   
AUTH_USER_REGISTRATION = True                                                                                                            
AUTH_USER_REGISTRATION_ROLE = "User"                                                                                                     
OAUTH_PROVIDERS = [                                                                                                                      
{ 'name': 'keycloak',                                                                                                                    
  'icon': 'fa-key',                                                                                                                      
  'token_key': 'access_token',                                                                                                           
  'remote_app': {                                                                                                                        
    'client_id': os.environ.get('OIDC_BCFDE64522F1D73E_CLIENT_ID'),                                                                      
    'client_secret': os.environ.get('OIDC_BCFDE64522F1D73E_CLIENT_SECRET'),                                                              
    'client_kwargs': {                                                                                                                   
      'scope': 'openid'                                                                                                                  
    },                                                                                                                                   
    'api_base_url': 'https://keycloak/realms/sdp/protocol/',                                                                             
    'server_metadata_url': 'https://keycloak/realms/sdp/.well-known/openid-configuration',                                               
  },                                                                                                                                     
}                                                                                                                                        
]                                                                                                                                        
                                                                                                                                         
WTF_CSRF_ENABLED = False

Possible solution

No response

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

None

[adwk67]

Snippet for release notes:

Previously some default values for the webserver config were applied after any entries specified under configOverrides. This has now been reversed so that defaults are applied first.