generated from oracle-devrel/repo-template
-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
External dbs observability and management v2 #856
Open
mitoeth
wants to merge
6
commits into
main
Choose a base branch
from
external_dbs_observability_and_management
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+877
−0
Open
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
e4d5363
Observability & Management Service Enablement of External dbs v1
mitoeth cf4fee3
Line breaks in readme.md (line 87-89)
mitoeth def7498
Password placeholders in db_credentials.json and db_credentials_examp…
mitoeth 1203619
README updates
mitoeth 3e6cfab
Connection and credential variables validated for empty strings
mitoeth c8b0810
Default OCID value placeholders removed
mitoeth File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
123 changes: 123 additions & 0 deletions
123
...erations/observability-and-manageability/external-database-enablement/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| # Enable Observability & Management for Multiple External Database Systems with Terraform | ||
|
|
||
| This Terraform asset enables Database Management, Operations Insights, and/or Stack Monitoring for multiple external container and pluggable databases based on input from two JSON-files in **root_module**: | ||
| - **db_systems.json** | ||
| - **db_credentials.json** | ||
|
|
||
| Reviewed: 21.02.2024 | ||
|
|
||
| ## How does it work? | ||
|
|
||
| OCI's [External Database Service](https://docs.oracle.com/en-us/iaas/external-database/index.html) handles management agent connections to on-prem database systems. Using these connections, certain services in Observability & Management can then be enabled for the external databases: [Database Management](https://docs.oracle.com/en-us/iaas/database-management/home.htm), [Operations Insights](https://docs.oracle.com/en-us/iaas/operations-insights/home.htm), and [Stack Monitoring](https://docs.oracle.com/en-us/iaas/stack-monitoring/index.html). After [installing management agents](https://docs.oracle.com/en-us/iaas/management-agents/doc/install-management-agent-chapter.html), the Terraform configuration files in this asset can perform the database connection setup for agents and enable Observability & Management services for all external databases | ||
|
|
||
| ## When to use this asset? | ||
|
|
||
| This asset is for anyone managing multiple on-prem database systems who needs to enable services in Observability & Management. Instead of spending valuable time doing this manually for each container and pluggable database in the OCI Console, just use these Terraform configuration files to complete the setup for you | ||
|
|
||
| ## How to use this asset? | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| 1. Prepare required policies for Management Agent Service, Database Management, Operations Insights, and/or Stack Monitoring: | ||
| - Click [here](https://docs.oracle.com/en-us/iaas/management-agents/doc/perform-prerequisites-deploying-management-agents.html) for more about Management Agent policies | ||
| - Click [here](https://docs.oracle.com/en-us/iaas/database-management/doc/permissions-required-enable-database-management-external-databases.html#DBMGM-GUID-3DDC9D5F-99B8-4DD5-A0C4-194D29FC883F) for more about Database Management policies | ||
| - Click [here](https://docs.oracle.com/en-us/iaas/operations-insights/doc/set-groups-users-and-policies.html) for more about Operations Insights policies | ||
| - Click [here](https://docs.oracle.com/en-us/iaas/stack-monitoring/doc/service-requirements.html) for more about Stack Monitoring policies | ||
| 2. [Install management agent(s)](https://docs.oracle.com/en-us/iaas/management-agents/doc/install-management-agent-chapter.html) connecting to database systems | ||
|
|
||
| **NOTE**: The management agent can either be installed locally on a database instance host or connect to databases remotely on a separate machine. The agent can connect to and monitor one or more databases | ||
|
|
||
| 4. [Install Terraform and create RSA keys for API signing](https://docs.oracle.com/en-us/iaas/developer-tutorials/tutorials/tf-provider/01-summary.htm) | ||
|
|
||
| **NOTE**: Remember to save the values in the **configuration file preview** when adding the public API key to an OCI user | ||
|
|
||
| ### Prepare and apply Terraform configurations | ||
|
|
||
| 1. Download **root_module** and its configuration files to the machine with the private RSA key and Terraform installation | ||
| 2. Set configuration options for the OCI provider in **root_module/provider.tf** | ||
|
|
||
| **NOTE**: Most options can be set with the configuration file preview values shown when adding a public API key to an OCI user | ||
| 3. Set **compartment_ocid** variable in **root_module/variables.tf** | ||
|
|
||
| **NOTE**: This is the compartment where everything will be managed in the External Database Service: database connections and enablement of services in Observability & Management. This will also be the compartment where all monitoring data will be located across the enabled services | ||
| 4. Define your database systems in **root_module/db_systems.json**. See **root_module/db_systems_example.json** for an example: | ||
| - Create a database system object for every database system. You can copy-paste the **system1** object as a template in **root_module/db_systems.json** | ||
|
|
||
| **NOTE**: You can name the keys for the system objects however you want as long as they are unique e.g. system1, system2, systemA, systemB, Alpha, Beta, etc. | ||
| - For each database system object, copy-paste the template within the **pdbs** array to match the number of pluggable databases per system | ||
| - Now fill out the details for each container and pluggable database in the database system objects as described below: | ||
|
|
||
| | ***Key*** | ***Description*** | ***Mandatory*** | | ||
| |--------------|-----------|------------| | ||
| | **host** | Host name used by management agent for connections with container and pluggable databases in system<br>**NOTE**: It is recommended to use the SCAN hostname for RAC systems | Yes | | ||
| | **port** | Port used by management agent for connections with container and pluggable databases in system<br>**NOTE**: Set to 1521 by default if no value is provided | Yes | | ||
| | **protocol** | Protocol used by management agent for connections with container and pluggable databases in system<br>**NOTE**: Must be **TCP** or **TCPS** | Yes | | ||
| | **managementAgentId** | OCID of the management agent connecting to container and pluggable databases in system | Yes | | ||
| | **databaseCredentials** | Key for credential object in **root_module/db_credentials.json** used by management agent for database connections<br>**NOTE**: If **protocol** is set to **TCPS**, the credential object must include **sslSecretId** | Yes | | ||
| | **containerName** | Desired name for container database | Yes | | ||
| | **containerServiceName** | Service name used by management agent for container database connection | Yes | | ||
| | **containerDBManagement** | Add **enable** or **disable** to manage Database Management enablement for container database | Yes | | ||
| | **dbManagementLicense** | Add **BRING_YOUR_OWN_LICENSE** or **LICENSE_INCLUDED** to select the license type used for Database Management. Click [here](https://docs.oracle.com/en-us/iaas/database-management/doc/enable-database-management-external-databases.html) for more | Yes, if **containerDBManagement** is set to **enable** | | ||
| | **containerStackMonitoring** | Add **enable** or **disable** to manage Stack Monitoring enablement for container database | Yes | | ||
| | **asmStackMonitoring** | Add **enable** or **disable** to manage Stack Monitoring enablement for ASM<br>**NOTE**: Enablement requires **containerStackMonitoring** to be **enable** as well | Yes | | ||
| | **asmHost** | Host name used by management agent for ASM connection | Yes, if **asmStackMonitoring** is set to **enable** | | ||
| | **asmPort** | Port used by management agent for ASM connection | Yes, if **asmStackMonitoring** is set to **enable** | | ||
| | **asmServiceName** | Service name used by management agent for ASM connection | Yes, if **asmStackMonitoring** is set to **enable** | | ||
| | **asmCredentials** | Key for credential object in **root_module/db_credentials.json** used by management agent for ASM connection<br>**NOTE**: Credential object must include **userPasswordSecretId** for the ASM password | Yes, if **asmStackMonitoring** is set to **enable** | | ||
| | **pdbName** | Desired name for pluggable database | Yes | | ||
| | **databaseCredentials** | Key for credential object in **root_module/db_credentials.json** used by management agent for database connections<br>**NOTE**: If **protocol** is set to **TCPS**, the credential object must include **sslSecretId** | Yes | | ||
| | **pdbServiceName** | Service name used by management agent for pluggable database connection | Yes | | ||
| | **pdbDBManagement** | Add **enable** or **disable** to manage Database Management enablement for pluggable database<br>**NOTE**: Enablement requires **containerDBManagement** to be **enable** as well | Yes | | ||
| | **pdbStackMonitoring** | Add **enable** or **disable** to manage Database Management enablement for pluggable database<br>**NOTE**: Enablement DOES NOT REQUIRE **containerStackMonitoring** to be **enable** as well | Yes | | ||
| | **pdbOPSI** | Add **enable** or **disable** to manage Operations Insights enablement for pluggable database | Yes | | ||
|
|
||
| 5. Define your database credentials in **root_module/db_credentials.json**. See **root_module/db_credentials_example.json** for an example: | ||
| - Create a credential object for every credential set used for management agent connections in **root_module/db_systems.json**. You can copy-paste the **cred1** and/or **cred2** objects as templates in **root_module/db_credentials.json** | ||
| - Now fill out the details for each credential object as described below: | ||
| | ***Key*** | ***Description*** | ***Mandatory*** | | ||
| |--------------|-----------|------------| | ||
| | **userName** | Database user name for management agent connections e.g. DBSNMP | Yes | | ||
| | **userPassword** | Database user password in plain text for management agent connections | No, if using **userPasswordSecretId** instead | | ||
| | **userPasswordSecretId** | OCID for encrypted Secret with database user password in OCI Vault. Click [here](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) for more<br>**NOTE**: Required to enable Stack Monitoring for ASM | No, if using **userPassword** instead | | ||
| | **userRole** | Database user role for management agent connections<br>**NOTE**: For database connections, **userRole** can be **NORMAL** or **SYSDBA**. For ASM connections, **userRole** can be **SYSASM**, **SYSDBA**, or **SYSOPER** | Yes | | ||
| | **sslSecretId** | OCID for encrypted Secret with JSON containing SSL-settings for database connections via TCPS. Click [here](https://docs.oracle.com/en-us/iaas/external-database/doc/create-connection-external-database.html#EXTUG-GUID-59ECD72C-EAC2-426D-B865-D8DDB1297F0E) for more | Yes, if **protocol** is set to **TCPS** for database system object in **root_module/db_systems.json**| | ||
|
|
||
| **NOTE**: **CREDENTIAL VALUES ABOVE ARE SAVED AS PLAIN TEXT** IN BOTH **root_module/db_credentials.json** AS WELL AS IN **root_module/terraform.tfstate** AFTER APPLYING THE TERRAFORM CONFIGURATION. **ENSURE THAT THESE FILES ARE STORED SECURELY** | ||
|
|
||
| 6. Run the following commands from **root_module** to initialize the Terraform configuration, see its execution plan, and finally apply that plan: | ||
|
|
||
| **terraform init**<br> | ||
| **terraform plan**<br> | ||
| **terraform apply**<br> | ||
|
|
||
| ### Update applied Terraform configurations | ||
|
|
||
| To apply new configurations, update **root_module/db_systems.json** and/or **root_module/db_credentials.json** and run **terraform apply** from **root_module** again | ||
|
|
||
| ### Destroy applied Terraform configurations | ||
| To remove everything previously applied by Terraform configurations, run **terraform destroy** from **root_module** | ||
|
|
||
| ## Useful Links | ||
|
|
||
| - [Oracle](https://www.oracle.com) | ||
| - Oracle's Main Website | ||
| - [Terraform Provider](https://registry.terraform.io/providers/oracle/oci/latest/docs) | ||
| - General documentaion for Terraform's Oracle Cloud Infrastructure Provider | ||
| - [Management Agent Service](https://docs.oracle.com/en-us/iaas/management-agents/index.html) | ||
| - General documentation for Management Agent Service | ||
| - [External Database Service](https://docs.oracle.com/en-us/iaas/external-database/index.html) | ||
| - General documentation for External Database Service | ||
| - [Database Management](https://docs.oracle.com/en-us/iaas/database-management/home.htm) | ||
| - General documentation for Database Management | ||
| - [Operations Insights](https://docs.oracle.com/en-us/iaas/operations-insights/home.htm) | ||
| - General documentation for Operations Insights | ||
| - [Stack Monitoring](https://docs.oracle.com/en-us/iaas/stack-monitoring/index.html) | ||
| - General documentation for Stack Monitoring | ||
|
|
||
| ## License | ||
|
|
||
| Copyright (c) 2024 Oracle and/or its affiliates. | ||
|
|
||
| Licensed under the Universal Permissive License (UPL), Version 1.0. | ||
|
|
||
| See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. |
18 changes: 18 additions & 0 deletions
18
.../observability-and-manageability/external-database-enablement/db_credentials_example.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| { | ||
| "cred1": { | ||
| "userName":"DBSNMP", | ||
| "userPassword":"<INSERT PASSWORD HERE>", | ||
| "userRole":"NORMAL" | ||
| }, | ||
| "cred2": { | ||
| "userName":"ASMSNMP", | ||
| "userPasswordSecretId":"ocid1.vaultsecret.oc1.XXXXXX", | ||
| "userRole":"SYSDBA" | ||
| }, | ||
| "cred3": { | ||
| "userName":"DBSNMP", | ||
| "userPasswordSecretId":"ocid1.vaultsecret.oc1.XXXXXX", | ||
| "userRole":"NORMAL", | ||
| "sslSecretId":"ocid1.vaultsecret.oc1.XXXXXX" | ||
| } | ||
| } |
64 changes: 64 additions & 0 deletions
64
...ions/observability-and-manageability/external-database-enablement/db_systems_example.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| { | ||
| "system1": { | ||
| "host":"dba.com", | ||
| "port":"1521", | ||
| "protocol":"TCP", | ||
| "managementAgentId":"ocid1.managementagent.oc1.XXXXXX", | ||
| "databaseCredentials":"cred1", | ||
| "containerName":"DBA_container", | ||
| "containerServiceName":"dba_container_service", | ||
| "containerDBManagement":"disable", | ||
| "dbManagementLicense":"", | ||
| "containerStackMonitoring":"enable", | ||
| "asmStackMonitoring":"enable", | ||
| "asmHost":"dba_asm.com", | ||
| "asmPort":"1525", | ||
| "asmServiceName":"+ASM", | ||
| "asmCredentials":"cred2", | ||
| "pdbs":[ | ||
| { | ||
| "pdbName":"DBA_pdb1", | ||
| "databaseCredentials":"cred1", | ||
| "pdbServiceName":"dba_pdb1_service", | ||
| "pdbDBManagement":"disable", | ||
| "pdbStackMonitoring":"enable", | ||
| "pdbOPSI":"enable" | ||
| } | ||
| ] | ||
| }, | ||
| "system2": { | ||
| "host":"dbb-scan.com", | ||
| "port":"2484", | ||
| "protocol":"TCPS", | ||
| "managementAgentId":"ocid1.managementagent.oc1.XXXXXX", | ||
| "databaseCredentials":"cred3", | ||
| "containerName":"DBB_container", | ||
| "containerServiceName":"dbb_container_service", | ||
| "containerDBManagement":"enable", | ||
| "dbManagementLicense":"BRING_YOUR_OWN_LICENSE", | ||
| "containerStackMonitoring":"enable", | ||
| "asmStackMonitoring":"disable", | ||
| "asmHost":"", | ||
| "asmPort":"", | ||
| "asmServiceName":"", | ||
| "asmCredentials":"", | ||
| "pdbs":[ | ||
| { | ||
| "pdbName":"DBB_pdb1", | ||
| "databaseCredentials":"cred3", | ||
| "pdbServiceName":"dbb_pdb1_service", | ||
| "pdbDBManagement":"enable", | ||
| "pdbStackMonitoring":"disable", | ||
| "pdbOPSI":"enable" | ||
| }, | ||
| { | ||
| "pdbName":"DBB_pdb2", | ||
| "databaseCredentials":"cred3", | ||
| "pdbServiceName":"dbb_pdb2_service", | ||
| "pdbDBManagement":"enable", | ||
| "pdbStackMonitoring":"enable", | ||
| "pdbOPSI":"disable" | ||
| } | ||
| ] | ||
| } | ||
| } |
13 changes: 13 additions & 0 deletions
13
...ervability-and-manageability/external-database-enablement/root_module/db_credentials.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| { | ||
| "cred1": { | ||
| "userName":"DBSNMP", | ||
| "userPassword":"<INSERT PASSWORD HERE>", | ||
| "userRole":"NORMAL" | ||
| }, | ||
| "cred2": { | ||
| "userName":"DBSNMP", | ||
| "userPasswordSecretId":"ocid1.vaultsecret.oc1.XXXXXX", | ||
| "userRole":"NORMAL", | ||
| "sslSecretId":"ocid1.vaultsecret.oc1.XXXXXX" | ||
| } | ||
| } |
29 changes: 29 additions & 0 deletions
29
.../observability-and-manageability/external-database-enablement/root_module/db_systems.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| { | ||
| "system1": { | ||
| "host":"IP ADDRESS OR HOSTNAME TO ACCESS DATABASE SYSTEM", | ||
| "port":"1521", | ||
| "protocol":"TCP OR TCPS", | ||
| "managementAgentId":"OCID FOR MANAGEMENT AGENT CONNECTING TO CONTAINER AND PLUGGABLE DATABASES", | ||
| "databaseCredentials":"KEY FOR THE CORRECT CREDENTIAL OBJECT IN db_credentials.json. IF protocol IS TCPS, CREDENTIAL OBJECT MUST INCLUDE sslSecretId", | ||
| "containerName":"container1", | ||
| "containerServiceName":"CONTAINER SERVICE NAME", | ||
| "containerDBManagement":"INSERT enable OR disable TO MANAGE DATABASE MANAGEMENT FOR CONTAINER DATABASE", | ||
| "dbManagementLicense":"BRING_YOUR_OWN_LICENSE OR LICENSE_INCLUDED", | ||
| "containerStackMonitoring":"INSERT enable OR disable TO MANAGE STACK MONITORING FOR CONTAINER DATABASE", | ||
| "asmStackMonitoring":"INSERT enable OR disable TO MANAGE STACK MONITORING FOR ASM. IF disable, OTHER ASM DETAILS ARE IGNORED", | ||
| "asmHost":"IP ADDRESS OR HOSTNAME TO ACCESS ASM INSTANCE", | ||
| "asmPort":"PORT TO ACCESS ASM INSTANCE", | ||
| "asmServiceName":"ASM SERVICE NAME", | ||
| "asmCredentials":"KEY FOR THE CORRECT CREDENTIAL OBJECT IN db_credentials.json. CREDENTIAL OBJECT MUST INCLUDE userPasswordSecretId", | ||
| "pdbs":[ | ||
| { | ||
| "pdbName":"pdb1", | ||
| "databaseCredentials":"KEY FOR THE CORRECT CREDENTIAL OBJECT IN db_credentials.json", | ||
| "pdbServiceName":"PDB SERVICE NAME", | ||
| "pdbDBManagement":"INSERT enable OR disable TO MANAGE DATABASE MANAGEMENT FOR PLUGGABLE DATABASE", | ||
| "pdbStackMonitoring":"INSERT enable OR disable TO MANAGE STACK MONITORING FOR PLUGGABLE DATABASE", | ||
| "pdbOPSI":"INSERT enable OR disable TO MANAGE OPERATIONS INSIGHTS FOR PLUGGABLE DATABASE" | ||
| } | ||
| ] | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a warning that the password values may be present in clear text in the tfstate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. See commit: 1203619