Standardization of VM diagnostics info API.

Before this patch, VM diagnostics response was just a
'blob' of data returned by each hypervisor. New API
version makes diagnostics response standardized.
New response has a set of fields which each hypervisor
will try to fill. If hypervisor unable to provide a
specific field then this field will be reported as 'None'.

Tempest tests: I7757c5beeea3d3b0bc15a51cafc5ea2ada65e76c

DocImpact: admin guide docs should be updated to mention
standardized version of the diagnostics response

blueprint: restore-vm-diagnostics

Change-Id: If0b1493cc5c1c7f0d9896dd68342ad4dea4f7da2

api-ref/source/diagnostics.inc

@@ -31,12 +31,39 @@ Request
 Response
 --------
 
-The response format for diagnostics is backend hypervisor specific,
-and not well defined. This should be considered a debug interface
-only, and not relied upon by programmatic tools.
+Starting from **microversion 2.48** diagnostics response is standardized
+across all virt drivers. The response should be considered a debug interface
+only and not relied upon by programmatic tools. All response fields are listed
+below. If the virt driver is unable to provide a specific field then this field
+will be reported as ``None`` in the response.
+
+.. rest_parameters:: parameters.yaml
+
+  - config_drive: config_drive_diagnostics
+  - state: vm_state_diagnostics
+  - driver: driver_diagnostics
+  - hypervisor: hypervisor_diagnostics
+  - hypervisor_os: hypervisor_os_diagnostics
+  - uptime: uptime_diagnostics
+  - num_cpus: num_cpus_diagnostics
+  - num_disks: num_disks_diagnostics
+  - num_nics: num_nics_diagnostics
+  - memory_details: memory_details_diagnostics
+  - cpu_details: cpu_details_diagnostics
+  - disk_details: disk_details_diagnostics
+  - nic_details: nic_details_diagnostics
 
 **Example Server diagnostics**
 
+.. literalinclude:: ../../doc/api_samples/os-server-diagnostics/v2.48/server-diagnostics-get-resp.json
+   :language: javascript
+
+.. warning::
+   Before **microversion 2.48** the response format for diagnostics was not
+   well defined. Each hypervisor had its own format.
+
+**Example Old Server diagnostics**
+
 Below is an example of diagnostics for a libvirt based instance. The unit of the return
 value is hypervisor specific, but in this case the unit of vnet1_rx* and
 vnet1_tx* is octets.

api-ref/source/parameters.yaml

@@ -1516,6 +1516,13 @@ config_drive:
   in: body
   required: false
   type: boolean
+config_drive_diagnostics:
+  description: |
+    Indicates whether or not a config drive was used for this server.
+  in: body
+  required: true
+  type: boolean
+  min_version: 2.48
 config_drive_resp:
   description: |
     Indicates whether or not a config drive was used for this server.
@@ -1602,6 +1609,20 @@ cores_quota_optional:
   in: body
   required: false
   type: integer
+cpu_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM CPUs.
+    Following fields are presented in each dictionary:
+
+    - ``id`` - the ID of CPU (Integer)
+
+    - ``time`` - CPU Time in nano seconds (Integer)
+
+    - ``utilisation`` - CPU utilisation in percents (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 cpu_info:
   description: |
     A dictionary that contains cpu information like ``arch``, ``model``,
@@ -1800,6 +1821,24 @@ disk_config:
   in: body
   required: true
   type: string
+disk_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM disks.
+    Following fields are presented in each dictionary:
+
+    - ``read_bytes`` - Disk reads in bytes (Integer)
+
+    - ``read_requests`` - Read requests (Integer)
+
+    - ``write_bytes`` - Disk writes in bytes (Integer)
+
+    - ``write_requests`` - Write requests (Integer)
+
+    - ``errors_count`` - Disk errors (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 disk_over_commit:
   description: |
     Set to ``True`` to enable over commit when the destination host is checked for
@@ -1833,6 +1872,19 @@ display_name_optional:
   in: body
   required: false
   type: string
+driver_diagnostics:
+  description: |
+    The driver on which the VM is running. Possible values are:
+
+    - ``libvirt``
+    - ``xenapi``
+    - ``hyperv``
+    - ``vmwareapi``
+    - ``ironic``
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 ended_at:
   description: |
     The date and time when the server was deleted.
@@ -2701,6 +2753,14 @@ hypervisor_count:
   in: body
   required: true
   type: integer
+hypervisor_diagnostics:
+  description: |
+    The hypervisor on which the VM is running. Examples for libvirt driver
+    may be: ``qemu``, ``kvm`` or ``xen``.
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 hypervisor_free_disk_gb:
   description: |
     The free disk remaining on this hypervisor(in GB).
@@ -2730,6 +2790,13 @@ hypervisor_links:
   type: array
   min_version: 2.33
   required: false
+hypervisor_os_diagnostics:
+  description: |
+    The hypervisor OS.
+  in: body
+  type: string
+  required: true
+  min_version: 2.48
 hypervisor_service:
   description: |
     The hypervisor service object.
@@ -3341,6 +3408,19 @@ members:
   in: body
   required: true
   type: array
+memory_details_diagnostics:
+  description: |
+    The dictionary with information about VM memory usage.
+    Following fields are presented in the dictionary:
+
+    - ``maximum`` - Amount of memory provisioned for the VM in MB (Integer)
+
+    - ``used`` - Amount of memory that is currently used by the guest operating
+      system and its applications in MB (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
 memory_mb:
   description: |
     The memory of this hypervisor(in MB).
@@ -3624,6 +3704,57 @@ new_file:
   in: body
   required: true
   type: string
+nic_details_diagnostics:
+  description: |
+    The list of dictionaries with detailed information about VM NICs.
+    Following fields are presented in each dictionary:
+
+    - ``mac_address`` - Mac address of the interface (String)
+
+    - ``rx_octets`` - Received octets (Integer)
+
+    - ``rx_errors`` - Received errors (Integer)
+
+    - ``rx_drop`` - Received packets dropped (Integer)
+
+    - ``rx_packets`` - Received packets (Integer)
+
+    - ``rx_rate`` - Receive rate in bytes (Integer)
+
+    - ``tx_octets`` - Transmitted Octets (Integer)
+
+    - ``tx_errors`` - Transmit errors (Integer)
+
+    - ``tx_drop`` - Transmit dropped packets (Integer)
+
+    - ``tx_packets`` - Transmit packets (Integer)
+
+    - ``tx_rate`` - Transmit rate in bytes (Integer)
+  in: body
+  required: true
+  type: array
+  min_version: 2.48
+num_cpus_diagnostics:
+  description: |
+    The number of vCPUs.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
+num_disks_diagnostics:
+  description: |
+    The number of disks.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
+num_nics_diagnostics:
+  description: |
+    The number of vNICs.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
 on_shared_storage:
   description: |
     Server on shared storage.
@@ -5009,6 +5140,13 @@ uptime:
   in: body
   required: true
   type: string
+uptime_diagnostics:
+  description: |
+    The amount of time in seconds that the VM has been running.
+  in: body
+  required: true
+  type: integer
+  min_version: 2.48
 uptime_simple_tenant_usage:
   description: |
     The uptime of the server.
@@ -5148,6 +5286,20 @@ virtual_interfaces:
   in: body
   required: true
   type: array
+vm_state_diagnostics:
+  description: |
+    A string enum denoting the current state of the VM. Possible values are:
+
+    - ``pending``
+    - ``running``
+    - ``paused``
+    - ``shutdown``
+    - ``crashed``
+    - ``suspended``
+  in: body
+  required: true
+  type: string
+  min_version: 2.48
 vm_state_optional:
   description: |
     The VM state.

doc/api_samples/os-server-diagnostics/v2.48/server-diagnostics-get-resp.json

@@ -0,0 +1,46 @@
+{
+    "config_drive": true,
+    "cpu_details": [
+        {
+            "id": 0,
+            "time": 17300000000,
+            "utilisation": 15
+        }
+    ],
+    "disk_details": [
+        {
+            "errors_count": 1,
+            "read_bytes": 262144,
+            "read_requests": 112,
+            "write_bytes": 5778432,
+            "write_requests": 488
+        }
+    ],
+    "driver": "libvirt",
+    "hypervisor": "kvm",
+    "hypervisor_os": "ubuntu",
+    "memory_details": {
+        "maximum": 524288,
+        "used": 0
+    },
+    "nic_details": [
+        {
+            "mac_address": "01:23:45:67:89:ab",
+            "rx_drop": 200,
+            "rx_errors": 100,
+            "rx_octets": 2070139,
+            "rx_packets": 26701,
+            "rx_rate": 300,
+            "tx_drop": 500,
+            "tx_errors": 400,
+            "tx_octets": 140208,
+            "tx_packets": 662,
+            "tx_rate": 600
+        }
+    ],
+    "num_cpus": 1,
+    "num_disks": 1,
+    "num_nics": 1,
+    "state": "running",
+    "uptime": 46664
+}

doc/api_samples/versions/v21-version-get-resp.json

@@ -19,7 +19,7 @@
             }
         ],
         "status": "CURRENT",
-        "version": "2.47",
+        "version": "2.48",
         "min_version": "2.1",
         "updated": "2013-07-23T11:33:21Z"
     }

doc/api_samples/versions/versions-get-resp.json

@@ -22,7 +22,7 @@
                 }
             ],
             "status": "CURRENT",
-            "version": "2.47",
+            "version": "2.48",
             "min_version": "2.1",
             "updated": "2013-07-23T11:33:21Z"
         }

nova/api/openstack/api_version_request.py

@@ -114,6 +114,7 @@
              rather than a link.  If the user is prevented from retrieving
              the flavor extra-specs by policy, simply omit the field from
              the output.
+    * 2.48 - Standardize VM diagnostics info.
 """
 
 # The minimum and maximum versions of the API supported
@@ -122,7 +123,7 @@
 # Note(cyeoh): This only applies for the v2.1 API once microversions
 # support is fully merged. It does not affect the V2 API.
 _MIN_API_VERSION = "2.1"
-_MAX_API_VERSION = "2.47"
+_MAX_API_VERSION = "2.48"
 DEFAULT_API_VERSION = _MIN_API_VERSION
 
 # Almost all proxy APIs which related to network, images and baremetal

nova/api/openstack/compute/rest_api_version_history.rst

@@ -556,3 +556,11 @@ user documentation.
   indexing extra-specs, then the ``extra_specs`` field will not be included in the
   flavor information.
 
+2.48
+----
+
+  Before version 2.48, VM diagnostics response was just a 'blob' of data
+  returned by each hypervisor. From this version VM diagnostics response is
+  standardized. It has a set of fields which each hypervisor will try to fill.
+  If a hypervisor driver unable to provide a specific field then this field
+  will be reported as 'None'.