feat: create new connections to each database in case of azure sql database

[sairaj18]

Changes in this PR -

When instrumenting an Azure SQL Database type service.

• Populate database metrics using queries that are specific to Azure SQL Database.
• Create new connections to each database for retrieving database metrics.
• Add unit tests for the same.

Below are the list of database metrics that are retrieved using Azure SQL Database specific queries.

1. log.transactionGrowth
2. io.stallInMilliseconds
3. bufferpool.sizePerDatabaseInBytes
4. pageFileTotal
5. pageFileAvailable

Testing Details :

root@sairaj-mssql-test-2:/home/sairaj/nri-mssql/bin# ./nri-mssql -username="newrelic"  -password="*****" -port=1433 -hostname="test.database.windows.net" -enable_buffer_metrics=true -enable_database_reserve_metrics=true -enable_disk_metrics_in_bytes -verbose=true
07:44:33.778077 [DEBUG] store file (/tmp/nr-integrations/com.newrelic.mssql-620af03b7337b72def7d45b9e21132e3.json) is older than 6m0s, skipping loading from disk.
[DEBUG] Running query: select COALESCE( @@SERVERNAME, SERVERPROPERTY('ServerName'), SERVERPROPERTY('MachineName')) as instance_name
[DEBUG] Running query: SELECT SERVERPROPERTY('EngineEdition') AS EngineEdition;
[DEBUG] Skipping query 'EXEC sp_configure' for unsupported engine edition 5
[DEBUG] Running query: select name, value from sys.configurations
[DEBUG] Running query: select name as db_name from sys.databases where name not in ('master', 'tempdb', 'msdb', 'model', 'rdsadmin', 'distribution', 'model_msdb', 'model_replicatedmaster')
[DEBUG] Running query: 
                        SELECT 
                            sd.name AS db_name,
                            spc.cntr_value AS log_growth
                        FROM sys.dm_os_performance_counters spc
                        INNER JOIN sys.databases sd 
                            ON sd.physical_database_name = spc.instance_name
                        WHERE spc.counter_name = 'Log Growths'
                            AND spc.object_name LIKE '%:Databases%'
                            AND sd.database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                            DB_NAME() AS db_name,
                            SUM(io_stall) AS io_stalls
                        FROM sys.dm_io_virtual_file_stats(NULL, NULL)
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT 
                            DB_NAME() AS db_name, 
                            COUNT_BIG(*) * (8 * 1024) AS buffer_pool_size
                        FROM sys.dm_os_buffer_descriptors WITH (NOLOCK) 
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                                DB_NAME() AS db_name,
                                sum(a.total_pages) * 8.0 * 1024 AS reserved_space,
                                (sum(a.total_pages)*8.0 - sum(a.used_pages)*8.0) * 1024 AS reserved_space_not_used
                        FROM sys.partitions p with (nolock)
                        INNER JOIN sys.allocation_units a WITH (NOLOCK) ON p.partition_id = a.container_id
                        LEFT JOIN sys.internal_tables it WITH (NOLOCK) ON p.object_id = it.object_id

[DEBUG] Running query: 
                        SELECT 
                            sd.name AS db_name,
                            spc.cntr_value AS log_growth
                        FROM sys.dm_os_performance_counters spc
                        INNER JOIN sys.databases sd 
                            ON sd.physical_database_name = spc.instance_name
                        WHERE spc.counter_name = 'Log Growths'
                            AND spc.object_name LIKE '%:Databases%'
                            AND sd.database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                            DB_NAME() AS db_name,
                            SUM(io_stall) AS io_stalls
                        FROM sys.dm_io_virtual_file_stats(NULL, NULL)
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT 
                            DB_NAME() AS db_name, 
                            COUNT_BIG(*) * (8 * 1024) AS buffer_pool_size
                        FROM sys.dm_os_buffer_descriptors WITH (NOLOCK) 
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                                DB_NAME() AS db_name,
                                sum(a.total_pages) * 8.0 * 1024 AS reserved_space,
                                (sum(a.total_pages)*8.0 - sum(a.used_pages)*8.0) * 1024 AS reserved_space_not_used
                        FROM sys.partitions p with (nolock)
                        INNER JOIN sys.allocation_units a WITH (NOLOCK) ON p.partition_id = a.container_id
                        LEFT JOIN sys.internal_tables it WITH (NOLOCK) ON p.object_id = it.object_id

[DEBUG] Running query: 
                        SELECT 
                            sd.name AS db_name,
                            spc.cntr_value AS log_growth
                        FROM sys.dm_os_performance_counters spc
                        INNER JOIN sys.databases sd 
                            ON sd.physical_database_name = spc.instance_name
                        WHERE spc.counter_name = 'Log Growths'
                            AND spc.object_name LIKE '%:Databases%'
                            AND sd.database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                            DB_NAME() AS db_name,
                            SUM(io_stall) AS io_stalls
                        FROM sys.dm_io_virtual_file_stats(NULL, NULL)
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT 
                            DB_NAME() AS db_name, 
                            COUNT_BIG(*) * (8 * 1024) AS buffer_pool_size
                        FROM sys.dm_os_buffer_descriptors WITH (NOLOCK) 
                        WHERE database_id = DB_ID()

[DEBUG] Running query: 
                        SELECT
                                DB_NAME() AS db_name,
                                sum(a.total_pages) * 8.0 * 1024 AS reserved_space,
                                (sum(a.total_pages)*8.0 - sum(a.used_pages)*8.0) * 1024 AS reserved_space_not_used
                        FROM sys.partitions p with (nolock)
                        INNER JOIN sys.allocation_units a WITH (NOLOCK) ON p.partition_id = a.container_id
                        LEFT JOIN sys.internal_tables it WITH (NOLOCK) ON p.object_id = it.object_id

[DEBUG] Running query: SELECT
                t1.cntr_value AS sql_compilations,
                t2.cntr_value AS sql_recompilations,
                t3.cntr_value AS user_connections,
                t4.cntr_value AS lock_wait_time_ms,
                t5.cntr_value AS page_splits_sec,
                t6.cntr_value AS checkpoint_pages_sec,
                t7.cntr_value AS deadlocks_sec,
                t8.cntr_value AS user_errors,
                t9.cntr_value AS kill_connection_errors,
                t10.cntr_value AS batch_request_sec,
                (t11.cntr_value * 1000.0) AS page_life_expectancy_ms,
                t12.cntr_value AS transactions_sec,
                t13.cntr_value AS forced_parameterizations_sec
                FROM 
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'SQL Compilations/sec') t1,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'SQL Re-Compilations/sec') t2,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'User Connections') t3,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Lock Wait Time (ms)' AND instance_name = '_Total') t4,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Page Splits/sec') t5,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Checkpoint pages/sec') t6,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Number of Deadlocks/sec' AND instance_name = '_Total') t7,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE object_name LIKE '%SQL Errors%' AND instance_name = 'User Errors') t8,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE object_name LIKE '%SQL Errors%' AND instance_name LIKE 'Kill Connection Errors%') t9,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Batch Requests/sec') t10,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Page life expectancy' AND object_name LIKE '%Manager%') t11,
                (SELECT Sum(cntr_value) AS cntr_value FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Transactions/sec') t12,
                (SELECT * FROM sys.dm_os_performance_counters WITH (nolock) WHERE counter_name = 'Forced Parameterizations/sec') t13
[DEBUG] Running query: SELECT (a.cntr_value * 1.0 / b.cntr_value) * 100.0 AS buffer_pool_hit_percent
                FROM sys.dm_os_performance_counters 
                a JOIN (SELECT cntr_value, OBJECT_NAME FROM sys.dm_os_performance_counters WHERE counter_name = 'Buffer cache hit ratio base') 
                b ON  a.OBJECT_NAME = b.OBJECT_NAME 
                WHERE a.counter_name = 'Buffer cache hit ratio'
[DEBUG] Running query: SELECT
                Sum(wait_time_ms) AS wait_time
                FROM sys.dm_os_wait_stats
                WHERE [wait_type] NOT IN (
                N'CLR_SEMAPHORE',    N'LAZYWRITER_SLEEP',
                N'RESOURCE_QUEUE',   N'SQLTRACE_BUFFER_FLUSH',
                N'SLEEP_TASK',       N'SLEEP_SYSTEMTASK',
                N'WAITFOR',          N'HADR_FILESTREAM_IOMGR_IOCOMPLETION',
                N'CHECKPOINT_QUEUE', N'REQUEST_FOR_DEADLOCK_SEARCH',
                N'XE_TIMER_EVENT',   N'XE_DISPATCHER_JOIN',
                N'LOGMGR_QUEUE',     N'FT_IFTS_SCHEDULER_IDLE_WAIT',
                N'BROKER_TASK_STOP', N'CLR_MANUAL_EVENT',
                N'CLR_AUTO_EVENT',   N'DISPATCHER_QUEUE_SEMAPHORE',
                N'TRACEWRITE',       N'XE_DISPATCHER_WAIT',
                N'BROKER_TO_FLUSH',  N'BROKER_EVENTHANDLER',
                N'FT_IFTSHC_MUTEX',  N'SQLTRACE_INCREMENTAL_FLUSH_SLEEP',
                N'DIRTY_PAGE_POLL',  N'SP_SERVER_DIAGNOSTICS_SLEEP')
[DEBUG] Running query: SELECT
                Max(CASE WHEN sessions.status = 'preconnect' THEN counts ELSE 0 END) AS preconnect,
                Max(CASE WHEN sessions.status = 'background' THEN counts ELSE 0 END) AS background,
                Max(CASE WHEN sessions.status = 'dormant' THEN counts ELSE 0 END) AS dormant,
                Max(CASE WHEN sessions.status = 'runnable' THEN counts ELSE 0 END) AS runnable,
                Max(CASE WHEN sessions.status = 'suspended' THEN counts ELSE 0 END) AS suspended,
                Max(CASE WHEN sessions.status = 'running' THEN counts ELSE 0 END) AS running,
                Max(CASE WHEN sessions.status = 'blocked' THEN counts ELSE 0 END) AS blocked,
                Max(CASE WHEN sessions.status = 'sleeping' THEN counts ELSE 0 END) AS sleeping
                FROM (SELECT status, Count(*) counts FROM (
                        SELECT CASE WHEN req.status IS NOT NULL THEN
                                CASE WHEN req.blocking_session_id <> 0 THEN 'blocked' ELSE req.status END
                          ELSE sess.status END status, req.blocking_session_id
                        FROM sys.dm_exec_sessions sess
                        LEFT JOIN sys.dm_exec_requests req
                        ON sess.session_id = req.session_id
                        WHERE sess.session_id > 50 ) statuses
                  GROUP BY status) sessions
[DEBUG] Running query: SELECT Sum(runnable_tasks_count) AS runnable_tasks_count
                FROM sys.dm_os_schedulers
                WHERE   scheduler_id < 255 AND [status] = 'VISIBLE ONLINE'
[DEBUG] Running query: SELECT Count(dbid) AS instance_active_connections FROM sys.sysprocesses WITH (nolock) WHERE dbid > 0
[DEBUG] Skipping query 'SELECT
                Max(sys_mem.total_physical_memory_kb * 1024.0) AS total_physical_memory,
                Max(sys_mem.available_physical_memory_kb * 1024.0) AS available_physical_memory,
                (Max(proc_mem.physical_memory_in_use_kb) / (Max(sys_mem.total_physical_memory_kb) * 1.0)) * 100 AS memory_utilization
                FROM sys.dm_os_process_memory proc_mem,
                  sys.dm_os_sys_memory sys_mem,
                  sys.dm_os_performance_counters perf_count WHERE object_name = 'SQLServer:Memory Manager'' for unsupported engine edition 5
[DEBUG] Running query:  SELECT
      Count_big(*) * (8*1024) AS instance_buffer_pool_size
      FROM sys.dm_os_buffer_descriptors WITH (nolock)
      WHERE database_id <> 32767 -- ResourceDB 
[DEBUG] Skipping query 'SELECT Sum(total_bytes) AS total_disk_space FROM (
                        SELECT DISTINCT
                        dovs.volume_mount_point,
                        dovs.available_bytes available_bytes,
                        dovs.total_bytes total_bytes
                        FROM sys.master_files mf WITH (nolock)
                        CROSS apply sys.Dm_os_volume_stats(mf.database_id, mf.file_id) dovs
                        ) drives' for unsupported engine edition 5
[DEBUG] Running query: SELECT wait_type, wait_time_ms AS wait_time, waiting_tasks_count
FROM sys.dm_os_wait_stats wait_stats
WHERE wait_time_ms != 0
output.json was printed here.
root@sairaj-mssql-test-2:/home/sairaj/nri-mssql/bin# 

output.json

[rahulreddy15]

Few minor nits.
Other than that PR looks great.

[rahulreddy15]

LGTM 🚀

[rajrohanyadav]

Added a few comments.
As you have added async code, we need to confirm if we are running unit tests with race condition in our CI. If not, we need to add that asap.
It's always a good idea to be extra sure with async code :D

[sairaj18]

Added a few comments.
As you have added async code, we need to confirm if we are running unit tests with race condition in our CI. If not, we need to add that asap.
It's always a good idea to be extra sure with async code :D

We are already running tests with race. Refer here for the same.