Allow unshelve to a specific host (REST API part)

This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.

Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.

Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7

api-ref/source/parameters.yaml

@@ -1858,8 +1858,11 @@ availability_zone_state:
 availability_zone_unshelve:
   description: |
     The availability zone name. Specifying an availability zone is only
-    allowed when the server status is ``SHELVED_OFFLOADED`` otherwise a
-    409 HTTPConflict response is returned.
+    allowed when the server status is ``SHELVED_OFFLOADED`` otherwise
+    HTTP 409 conflict response is returned.
+
+    Since microversion 2.91 ``"availability_zone":null`` allows unpinning the
+    instance from any availability_zone it is pinned to.
   in: body
   required: false
   type: string
@@ -3690,6 +3693,15 @@ host_status_update_rebuild:
   required: false
   type: string
   min_version: 2.75
+host_unshelve:
+  description: |
+    The destination host name. Specifying a destination host is by default only
+    allowed to project_admin, if it not the case HTTP 403 forbidden response
+    is returned.
+  in: body
+  required: false
+  type: string
+  min_version: 2.91
 host_zone:
   description: |
     The available zone of the host.

api-ref/source/servers-action-shelve.inc

@@ -121,9 +121,65 @@ Policy defaults enable only users with the administrative role or the owner of t
 
 **Preconditions**
 
-The server status must be ``SHELVED`` or ``SHELVED_OFFLOADED``.
+Unshelving a server without parameters requires its status to be ``SHELVED`` or ``SHELVED_OFFLOADED``.
+
+Unshelving a server with availability_zone and/or host parameters requires its status to be only ``SHELVED_OFFLOADED`` otherwise HTTP 409 conflict response is returned.
+
+If a server is locked, you must have administrator privileges to unshelve the server.
+
+As of ``microversion 2.91``, you can unshelve to a specific compute node if you have PROJECT_ADMIN privileges.
+This microversion also gives the ability to pin a server to an availability_zone and to unpin a server
+from any availability_zone.
+
+When a server is pinned to an availability_zone, the server move operations will keep the server in that
+availability_zone. However, when the server is not pinned to any availability_zone, the move operations can
+move the server to nodes in different availability_zones.
+
+The behavior according to unshelve parameters will follow the below table.
+
++----------+---------------------------+----------+--------------------------------+
+| Boot     | AZ (1)                    | Host (1) | Result                         |
++==========+===========================+==========+================================+
+|  No AZ   | No AZ or AZ=null          | No       | Free scheduling (2)            |
++----------+---------------------------+----------+--------------------------------+
+|  No AZ   | No AZ or AZ=null          | Host1    | Schedule to Host1.             |
+|          |                           |          | Server remains unpinned.       |
++----------+---------------------------+----------+--------------------------------+
+|  No AZ   | AZ="AZ1"                  | No       | Schedule to any host in "AZ1". |
+|          |                           |          | Server is pined to "AZ1".      |
++----------+---------------------------+----------+--------------------------------+
+|  No AZ   | AZ="AZ1"                  | Host1    | Verify Host1 is in "AZ1",      |
+|          |                           |          | then schedule to Host1,        |
+|          |                           |          | otherwise reject the request.  |
+|          |                           |          | Server is pined to "AZ1".      |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | No AZ                     | No       | Schedule to any host in "AZ1". |
+|          |                           |          | Server remains pined to "AZ1". |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | AZ=null                   | No       | Free scheduling (2).           |
+|          |                           |          | Server is unpinned.            |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | No AZ                     | Host1    | Verify Host1 is in "AZ1",      |
+|          |                           |          | then schedule to Host1,        |
+|          |                           |          | otherwise reject the request.  |
+|          |                           |          | Server remains pined to "AZ1". |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | AZ=null                   | Host1    | Schedule to Host1.             |
+|          |                           |          | Server is unpinned.            |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | AZ="AZ2"                  | No       | Schedule to any host in "AZ2". |
+|          |                           |          | Server is pined to "AZ2".      |
++----------+---------------------------+----------+--------------------------------+
+|  AZ1     | AZ="AZ2"                  | Host1    | Verify Host1 is in "AZ2" then  |
+|          |                           |          | schedule to Host1,             |
+|          |                           |          | otherwise reject the request.  |
+|          |                           |          | Server is pined to "AZ2".      |
++----------+---------------------------+----------+--------------------------------+
+
+(1) Unshelve body parameters
+(2) Schedule to any host available.
+
 
-If the server is locked, you must have administrator privileges to unshelve the server.
 
 **Asynchronous Postconditions**
 
@@ -147,11 +203,30 @@ Request
           {"unshelve": null} or {"unshelve": {"availability_zone": <string>}}.
           A request body of {"unshelve": {}} is not allowed.
 
+.. note:: Since microversion 2.91, allowed request body schema are
+
+  - {"unshelve": null}   (Keep compatibility with previous microversions)
+
+  or
+
+  - {"unshelve": {"availability_zone": <string>}} (Unshelve and pin server to availability_zone)
+  - {"unshelve": {"availability_zone": null}}   (Unshelve and unpin server from any availability zone)
+  - {"unshelve": {"host": <fqdn>}}
+  - {"unshelve": {"availability_zone": <string>, "host": <fqdn>}}
+  - {"unshelve": {"availability_zone": null, "host": <fqdn>}}
+
+  Everything else is not allowed, examples:
+
+  - {"unshelve": {}}
+  - {"unshelve": {"host": <fqdn>, "host": <fqdn>}}
+  - {"unshelve": {"foo": <string>}}
+
 .. rest_parameters:: parameters.yaml
 
   - server_id: server_id_path
   - unshelve: unshelve
   - availability_zone: availability_zone_unshelve
+  - host: host_unshelve
 
 |
 
@@ -162,9 +237,22 @@ Request
 
 **Example Unshelve server (unshelve Action) (v2.77)**
 
-.. literalinclude:: ../../doc/api_samples/os-shelve/v2.77/os-unshelve.json
+.. literalinclude:: ../../doc/api_samples/os-shelve/v2.77/os-unshelve-az.json
    :language: javascript
 
+**Examples Unshelve server (unshelve Action) (v2.91)**
+
+.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-host.json
+   :language: javascript
+
+.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-az-host.json
+   :language: javascript
+
+.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-host-and-unpin-az.json
+   :language: javascript
+
+.. literalinclude:: ../../doc/api_samples/os-shelve/v2.91/os-unshelve-unpin-az.json
+   :language: javascript
 
 Response
 --------

doc/api_samples/os-shelve/v2.91/os-unshelve-az-host.json

@@ -0,0 +1,6 @@
+{
+    "unshelve": {
+        "availability_zone": "nova",
+        "host": "host01"
+    }
+}

doc/api_samples/os-shelve/v2.91/os-unshelve-host-and-unpin-az.json

@@ -0,0 +1,6 @@
+{
+    "unshelve": {
+        "availability_zone": null,
+        "host": "host01"
+    }
+}

doc/api_samples/os-shelve/v2.91/os-unshelve-host.json

@@ -0,0 +1,5 @@
+{
+    "unshelve": {
+        "host": "host01"
+    }
+}

doc/api_samples/os-shelve/v2.91/os-unshelve-unpin-az.json

@@ -0,0 +1,5 @@
+{
+    "unshelve": {
+        "availability_zone": null
+    }
+}

doc/api_samples/versions/v21-version-get-resp.json

@@ -19,7 +19,7 @@
             }
         ],
         "status": "CURRENT",
-        "version": "2.90",
+        "version": "2.91",
         "min_version": "2.1",
         "updated": "2013-07-23T11:33:21Z"
     }

doc/api_samples/versions/versions-get-resp.json

@@ -22,7 +22,7 @@
                 }
             ],
             "status": "CURRENT",
-            "version": "2.90",
+            "version": "2.91",
             "min_version": "2.1",
             "updated": "2013-07-23T11:33:21Z"
         }

nova/api/openstack/api_version_request.py

@@ -247,6 +247,8 @@
              updating or rebuilding an instance. The
              ``OS-EXT-SRV-ATTR:hostname`` attribute is now returned in various
              server responses regardless of policy configuration.
+    * 2.91 - Add support to unshelve instance to a specific host and
+             to pin/unpin AZ.
 """
 
 # The minimum and maximum versions of the API supported
@@ -255,7 +257,7 @@
 # Note(cyeoh): This only applies for the v2.1 API once microversions
 # support is fully merged. It does not affect the V2 API.
 _MIN_API_VERSION = '2.1'
-_MAX_API_VERSION = '2.90'
+_MAX_API_VERSION = '2.91'
 DEFAULT_API_VERSION = _MIN_API_VERSION
 
 # Almost all proxy APIs which are related to network, images and baremetal

nova/api/openstack/compute/rest_api_version_history.rst

@@ -1202,3 +1202,12 @@ hostname based on the display name.
 
 In addition, the ``OS-EXT-SRV-ATTR:hostname`` field for all server
 responses is now visible to all users. Previously this was an admin-only field.
+
+.. _microversion 2.91:
+
+2.91
+----
+
+Add support to unshelve instance to a specific host.
+
+Add support to pin a server to an availability zone or unpin a server from any availability zone.

nova/api/openstack/compute/schemas/shelve.py

@@ -15,7 +15,7 @@
 from nova.api.validation import parameter_types
 
 # NOTE(brinzhang): For older microversion there will be no change as
-# schema is applied only for >2.77 with unshelve a server API.
+# schema is applied only for version < 2.91 with unshelve a server API.
 # Anything working in old version keep working as it is.
 unshelve_v277 = {
     'type': 'object',
@@ -35,3 +35,55 @@
     'required': ['unshelve'],
     'additionalProperties': False,
 }
+
+# NOTE(rribaud):
+# schema is applied only for version >= 2.91 with unshelve a server API.
+# Add host parameter to specify to unshelve to this specific host.
+#
+# Schema has been redefined for better clarity instead of extend 2.77.
+#
+# API can be called with the following body:
+#
+# - {"unshelve": null}   (Keep compatibility with previous microversions)
+#
+# or
+#
+# - {"unshelve": {"availability_zone": <string>}}
+# - {"unshelve": {"availability_zone": null}}   (Unpin availability zone)
+# - {"unshelve": {"host": <fqdn>}}
+# - {"unshelve": {"availability_zone": <string>, "host": <fqdn>}}
+# - {"unshelve": {"availability_zone": null, "host": <fqdn>}}
+#
+#
+# Everything else is not allowed, examples:
+#
+# - {"unshelve": {}}
+# - {"unshelve": {"host": <fqdn>, "host": <fqdn>}}
+# - {"unshelve": {"foo": <string>}}
+
+unshelve_v291 = {
+    "type": "object",
+    "properties": {
+        "unshelve": {
+            "oneOf": [
+                {
+                    "type": ["object"],
+                    "properties": {
+                        "availability_zone": {
+                            "oneOf": [
+                                {"type": ["null"]},
+                                {"type": "string"}]
+                        },
+                        "host": {
+                            "type": "string"
+                        }
+                    },
+                    "additionalProperties": False
+                },
+                {"type": ["null"]}
+            ]
+        }
+    },
+    "required": ["unshelve"],
+    "additionalProperties": False
+}

nova/api/openstack/compute/shelve.py

@@ -68,7 +68,6 @@ def _shelve_offload(self, req, id, body):
         context.can(shelve_policies.POLICY_ROOT % 'shelve_offload',
                     target={'user_id': instance.user_id,
                             'project_id': instance.project_id})
-
         try:
             self.compute_api.shelve_offload(context, instance)
         except exception.InstanceIsLocked as e:
@@ -87,33 +86,59 @@ def _shelve_offload(self, req, id, body):
     # In microversion 2.77 we support specifying 'availability_zone' to
     # unshelve a server. But before 2.77 there is no request body
     # schema validation (because of body=null).
-    @validation.schema(shelve_schemas.unshelve_v277, min_version='2.77')
+    @validation.schema(
+        shelve_schemas.unshelve_v277,
+        min_version='2.77',
+        max_version='2.90'
+    )
+    # In microversion 2.91 we support specifying 'host' to
+    # unshelve an instance to a specific hostself.
+    # 'availability_zone' = None is supported as well to unpin the
+    # availability zone of an instance bonded to this availability_zone
+    @validation.schema(shelve_schemas.unshelve_v291, min_version='2.91')
     def _unshelve(self, req, id, body):
         """Restore an instance from shelved mode."""
         context = req.environ["nova.context"]
         instance = common.get_instance(self.compute_api, context, id)
-        context.can(shelve_policies.POLICY_ROOT % 'unshelve',
-                    target={'project_id': instance.project_id})
+        context.can(
+            shelve_policies.POLICY_ROOT % 'unshelve',
+            target={'project_id': instance.project_id}
+        )
 
         unshelve_args = {}
 
-        unshelve_dict = body['unshelve']
-        support_az = api_version_request.is_supported(req, '2.77')
-        if support_az and unshelve_dict:
-            unshelve_args['new_az'] = unshelve_dict['availability_zone']
+        unshelve_dict = body.get('unshelve')
+        support_az = api_version_request.is_supported(
+            req, '2.77')
+        support_host = api_version_request.is_supported(
+            req, '2.91')
+        if unshelve_dict:
+            if support_az and 'availability_zone' in unshelve_dict:
+                unshelve_args['new_az'] = (
+                    unshelve_dict['availability_zone']
+                )
+            if support_host:
+                unshelve_args['host'] = unshelve_dict.get('host')
 
         try:
-            self.compute_api.unshelve(context, instance, **unshelve_args)
-        except (exception.InstanceIsLocked,
-                exception.UnshelveInstanceInvalidState,
-                exception.MismatchVolumeAZException) as e:
+            self.compute_api.unshelve(
+                context,
+                instance,
+                **unshelve_args,
+            )
+        except (
+            exception.InstanceIsLocked,
+            exception.UnshelveInstanceInvalidState,
+            exception.UnshelveHostNotInAZ,
+            exception.MismatchVolumeAZException,
+        ) as e:
             raise exc.HTTPConflict(explanation=e.format_message())
         except exception.InstanceInvalidState as state_error:
-            common.raise_http_conflict_for_instance_invalid_state(state_error,
-                                                                  'unshelve',
-                                                                  id)
+            common.raise_http_conflict_for_instance_invalid_state(
+                state_error, 'unshelve', id)
         except (
             exception.InvalidRequest,
             exception.ExtendedResourceRequestOldCompute,
+            exception.ComputeHostNotFound,
         ) as e:
             raise exc.HTTPBadRequest(explanation=e.format_message())

nova/compute/api.py

@@ -75,6 +75,7 @@
 from nova.objects import service as service_obj
 from nova.pci import request as pci_request
 from nova.policies import servers as servers_policies
+from nova.policies import shelve as shelve_policies
 import nova.policy
 from nova import profiler
 from nova import rpc
@@ -4504,6 +4505,14 @@ def unshelve(
         # host is requested, so we have to see if it exists and does not
         # contradict with the AZ of the instance
         if host:
+            # Make sure only admin can unshelve to a specific host.
+            context.can(
+                shelve_policies.POLICY_ROOT % 'unshelve_to_host',
+                target={
+                    'user_id': instance.user_id,
+                    'project_id': instance.project_id
+                }
+            )
             # Ensure that the requested host exists otherwise raise
             # a ComputeHostNotFound exception
             objects.ComputeNode.get_first_node_by_host_for_old_compat(