Merge pull request #184 from dulajdilshan/deploy-consolidated-packages

Add configuration best practices, consolidated packages usages, and improvements for deployment guides

Author: anupama-pathirage

en/docs/assets/img/deploy/build-executable.png

@@ -18,9 +18,14 @@ Follow the steps below to execute the Docker image.
 1. Execute the docker images command to verify if the Docker image is generated.
 
 ```bash
-$ docker images
-REPOSITORY                                          TAG                    IMAGE ID       CREATED          SIZE
-fake_store_manager                                  latest                 e971e4336f71   58 minutes ago   237MB
+docker images
+```
+
+The output will be similar to the following:
+
+```bash
+REPOSITORY                                          TAG                    IMAGE ID             CREATED                SIZE
+fake_store_manager                          latest                 e971e4336f71   58 minutes ago   237MB
 ```
 
 2. Execute the `docker run -d -v <path/to/config>/Config.toml:/home/ballerina/Config.toml -p 8090:8090 fake_store_manager:latest` command to run the generated Docker image.

en/docs/deploy/containerized-deployment/deploy-as-docker-image.md

@@ -14,7 +14,7 @@ This guide explains how to deploy an integration in a Kubernetes cluster.
 
     - Specify the container image details by creating a `Cloud.toml` file.
 
-    ```
+    ```toml
     [container.image]
     repository="wso2inc" # Docker hub repository name.
     name="greeter" # container name
@@ -55,8 +55,8 @@ This guide explains how to deploy an integration in a Kubernetes cluster.
 ## Step 3:  Push the Docker image
 
     Execute the command below to push the created Docker image into Docker Hub for the cluster to get access to the previously built container.
-    ```
-    $ docker push wso2inc/greeter:latest
+    ```bash
+    docker push wso2inc/greeter:latest
     ```
 
     ???+ Note
@@ -73,8 +73,8 @@ This guide explains how to deploy an integration in a Kubernetes cluster.
 
     Execute the command below to deploy the application into the Kubernetes cluster.
 
-    ```
-    $ kubectl apply -f /home/example/greeter/target/kubernetes/greeter
+    ```bash
+    kubectl apply -f /home/example/greeter/target/kubernetes/greeter
     ```
     You view the output below.
 

en/docs/deploy/containerized-deployment/deploy-on-kubernetes.md

@@ -0,0 +1,234 @@
+# Configuration Best Practices
+
+This guide covers configuration management strategies across deployment models (VM-based, containerized, Kubernetes) and CI/CD pipelines. Core principle: configuration is injected at runtime, never embedded in artifacts.
+
+## VM-based deployments
+
+### Centralized approach
+
+All services share a single configuration management point.
+
+#### Structure
+- Base configuration: `/opt/integrations/shared/Config.toml`
+- Environment overrides: `/opt/integrations/environments/{env}/Config.{env}.toml`
+- Endpoints: `/opt/integrations/environments/{env}/endpoints.{env}.toml`
+- Secrets: `/opt/integrations/environments/{env}/secrets.{env}.toml`
+
+#### Loading
+Create a loader script that combines files in precedence order: secrets → environment-specific → base.
+
+#### Management
+All services use the same configuration loader mechanism. Update configurations in one place.
+
+### Decentralized approach
+
+Each service maintains its own configuration independently.
+
+#### Structure
+- Per service: `/opt/integrations/{service-name}/conf/base.toml`, `production.toml`, `secrets.production.toml`
+
+#### Loading
+Each service's systemd file references its own configuration via `BAL_CONFIG_FILES`.
+
+#### Management
+Services are deployed and configured independently. Good for teams owning individual services.
+
+## Containerized deployments
+
+???+ Warning
+    Do NOT include `Config.toml` in the container image.
+
+### Configuration injection methods
+
+1. Volume mounts: `-v /path/to/Config.toml:/home/ballerina/conf/Config.toml`
+2. Environment variables: `-e BAL_CONFIG_VAR_PORT=9090`
+3. Docker Compose: Reference external config files via `.env` files
+
+### Docker Compose pattern
+
+Use environment files (`.env.development`, `.env.production`) to define environment-specific values. Mount configuration files as read-only volumes. Each environment has a separate config directory.
+
+### Docker Swarm
+
+Use Docker Secrets for sensitive data, Docker Configs for non-sensitive configuration. Reference them in service definitions.
+
+## Kubernetes deployments
+
+### Configuration resources
+
+#### ConfigMap
+Non-sensitive configuration
+
+- Base application config
+- Endpoint definitions  
+- Feature flags
+
+#### Secrets
+Sensitive data
+
+- Database passwords
+- API keys
+- JWT secrets
+
+### Configuration delivery
+
+1. ConfigMap volumes mount as read-only files at `/etc/config/`
+2. Secret volumes mount with restricted permissions at `/etc/secrets/`
+3. Individual values are injected as environment variables
+
+### Environment management with kustomize
+
+Use kustomize overlays for environment progression:
+
+- `base/`: Common manifests
+- `overlays/development/`: Dev-specific ConfigMaps and patches
+- `overlays/staging/`: Staging overrides
+- `overlays/production/`: Production overrides
+
+Deploy with: `kubectl apply -k overlays/production/`
+
+## CI/CD configuration handling
+
+### Build process
+Build WITHOUT the configuration files. Create deployment artifacts independent of environment.
+
+### Deployment process  
+Each stage applies environment-specific configuration before updating the application:
+
+1. Dev deployment: Apply dev ConfigMaps/Secrets → Update image
+2. Staging deployment: Apply staging ConfigMaps/Secrets → Update image
+3. Production deployment: Apply production ConfigMaps/Secrets → Update image (with manual approval)
+
+### Configuration storage
+- Keep config files in repository (`config/` or `k8s/` directories)
+- Store secrets in platform secret storage (never in repo)
+- Each environment has separate secrets
+
+### Pattern across platforms
+GitHub Actions, GitLab CI, and Jenkins follow the same: build once, configure per environment, deploy many times.
+
+## Configuration promotion
+
+Configuration flows through environments: development → staging → production.
+
+### Promotion workflow
+
+1. Validate source environment configuration
+2. Backup existing target configuration
+3. Copy with environment-specific transformations
+4. Validate transformed configuration
+5. Apply to the target environment
+6. Audit log all changes
+
+### Transformations
+
+#### Dev to staging
+
+- Change: `dev.example.com` → `staging.example.com`
+- Keep: Authentication details unchanged
+- Adjust: Resource limits if needed
+
+#### Staging to production
+
+- Change: `staging.example.com` → `prod.example.com`
+- Increase: Log level requirements
+- Enable: Additional monitoring/security
+
+### Automation tools
+
+Automate promotion scripts that:
+
+- Validate before proceeding
+- Log all changes
+- Enable rollback
+- Handle transformation rules
+
+## Endpoint promotion
+
+External and internal service endpoints change across environments.
+
+### Endpoint configuration structure
+
+```toml
+[external_services.development]
+auth_service = "https://auth-dev.example.com"
+
+[external_services.staging]
+auth_service = "https://auth-staging.example.com"
+
+[external_services.production]
+auth_service = "https://auth.example.com"
+```
+
+### Endpoint resolution
+
+During application startup:
+
+- Read `endpoints.toml`
+- Select the correct endpoint block based on the `ENVIRONMENT` variable
+- Use the selected endpoint for all outbound connections
+
+### Endpoint promotion
+
+Promote independently from the general configuration:
+
+1. Extract source environment endpoint block
+2. Transform domain/URLs to target pattern
+3. Apply to the target environment
+4. Validate connectivity to new endpoints
+5. Document changes in the audit log
+
+## Common best practices
+
+### Separation
+
+Configuration should be separate from code and artifacts.
+
+### Precedence management
+
+Leverage configuration precedence order correctly:
+
+1. Environment variables (highest priority)
+2. Command-line arguments
+3. TOML files
+4. Embedded defaults (lowest priority)
+
+### Secrets security
+
+- Never commit secrets to version control
+- Use platform secret management (Vault, K8s Secrets, CI/CD secrets)
+- Rotate secrets regularly
+- Audit all secret access
+
+### Validation
+
+Validate the configuration at startup.
+
+- Check required values present
+- Validate value ranges and formats
+- Fail fast on invalid configuration
+
+### Documentation
+
+Maintain the endpoint and the configuration documentation.
+
+- Example configuration files (Config.toml.example)
+- Endpoint registry showing all external/internal endpoints
+- Document transformation rules for promotions
+
+### Auditability
+
+Log all configuration changes.
+
+- Who changed what
+- When changes occurred
+- Source and target environments
+- Rollback actions
+
+## References
+
+- [Managing Configurations](/deploy/managing-configurations)
+- [Kubernetes ConfigMaps & Secrets](https://kubernetes.io/docs/tasks/configure-pod-container/)
+- [Docker Compose Documentation](https://docs.docker.com/compose/)
+- [Kustomize](https://kustomize.io/)
+- [HashiCorp Vault](https://www.vaultproject.io/)

en/docs/deploy/deployment-best-practices/configuration-best-practices.md

@@ -6,17 +6,17 @@ Implement high availability (HA) and failover configurations to ensure continuou
 
 * **Cloud-native deployments:** Achieve high availability through the container orchestration platform (e.g., Kubernetes).
 
-* **VM-based deployments:** Use a minimum of two nodes configured for failover to maintain service continuity.
+* **VM-based deployments:** Deploy a minimum of two nodes configured for active-active or active-passive failover to maintain service continuity. For critical production environments with strict availability SLAs, consider three or more nodes.
 
 Continuously monitor the health and performance of all nodes within the cluster. Track key metrics such as resource utilization, response time anomalies, and the volume of incoming network connections. Effective monitoring helps you determine when to add failover instances or adjust network routing to prevent service disruptions.
 
 ## Maintain network-level logging
 
-Enable and retain logs for all network components, including proxy servers, load balancers, and other critical infrastructure devices. Regularly review these logs to detect abnormal behavior, unauthorized access attempts, or configuration changes.
+Enable and retain logs for all network components, including proxy servers, load balancers, and other critical infrastructure devices. Review these logs regularly to detect abnormal behavior, unauthorized access attempts, or configuration changes.
 
 ## Audit open ports and services
 
-Conduct periodic network scans to identify open ports and active services. Ensure that only the ports necessary for your WSO2 products are accessible on both internal and external networks. Disable or monitor any additional open ports that are not explicitly required.
+Conduct periodic network scans to identify open ports and active services. Use tools such as nmap, netstat, or ss for port scanning. Ensure that only the ports necessary for your WSO2 products are accessible on both internal and external networks. Disable or monitor any additional open ports that are not explicitly required.
 
 ## Enforce device-level security
 

en/docs/deploy/deployment-best-practices/network-level-security.md

@@ -4,17 +4,27 @@
 
 Use a dedicated OS-level user account to run WSO2 products. Assign only the minimum permissions necessary for running the product. Avoid using the root or administrator account, as these have full privileges by default and increase the risk of security breaches.
 
+Recommended permissions for the dedicated user:
+
+* Read and execute access to the application directory
+* Read and write access to the log directory
+* Read access to configuration files
+* No sudo or administrative privileges
+
 ## Minimize installed software
 Install only the software and packages required for your WSO2 product deployment. Unnecessary software can introduce vulnerabilities. Regularly review and monitor installed packages.
 
 Refer to the [system requirements](/references/system-requirements) for details on the minimum required software.
 
 ## Enable the firewall
-Enable and configure a host-level firewall (e.g., iptables) to protect inbound and outbound connections. Only open the ports that are required for product functionality.
+Enable and configure a host-level firewall (e.g., iptables, UFW, or firewalld) to protect inbound and outbound connections. Only open the ports that are required for product functionality.
 
 ## Restrict access to clustering ports
 
-Apply firewall rules to restrict access to TCP ports used for clustering (e.g., ports 4000, 4001, etc.) so that they are accessible only to other nodes within the WSO2 product cluster. Prevent access from unrecognized or external hosts.
+Apply firewall rules to restrict access to TCP ports used for clustering so that they are accessible only to other nodes within the WSO2 product cluster. Prevent access from unrecognized or external hosts.
+
+!!! note
+    The actual clustering ports depend on your product configuration. Verify the ports used in your deployment before configuring firewall rules.
 
 ## Use secure shell (SSH)
 
@@ -34,4 +44,18 @@ Enable OS-level logging and review logs periodically to monitor user actions. Co
 
 ## Perform regular backups
 
-Back up all the critical files and data regularly, and store them securely.
+Back up all critical files and data regularly, and store them securely.
+
+Critical files to include in backups:
+
+* Configuration files (e.g., `Config.toml`, environment-specific configurations)
+* Keystores and truststores
+* Database exports
+* Custom scripts and deployment artifacts
+
+Backup recommendations:
+
+* Perform daily backups for production environments
+* Store backups in encrypted, geographically-distributed storage
+* Test backup restoration procedures periodically
+* Implement access controls for backup storage

en/docs/deploy/deployment-best-practices/os-level-security.md

@@ -20,6 +20,12 @@ Network security covers the infrastructure and network-level protections for you
 
 [Learn more about network level security →](/deploy/deployment-best-practices/network-level-security)
 
+## Configuration best practices
+
+Adhering to secure configuration practices is essential for minimizing vulnerabilities in your BI deployment. This includes managing sensitive data, applying security patches, and following secure coding standards.
+
+[Learn more about configuration best practices →](/deploy/deployment-best-practices/configuration-best-practices)
+
 ## Getting started
 
 Review each category above to understand the security considerations for your deployment. Implement these practices based on your specific environment, compliance requirements, and security policies. These guidelines are applicable to both cloud-native (containerized) and traditional VM-based deployments.