Giving some love to Spot instances

.spacelift/config.yml

@@ -1,5 +1,5 @@
 version: 2
-module_version: 5.1.0
+module_version: 5.2.0
 
 tests:
   - name: AMD64-based workerpool
@@ -14,6 +14,9 @@ tests:
   - name: Custom IAM Role
     project_root: examples/custom-iam-role
 
+  - name: Spot instances
+    project_root: examples/spot-instances
+
   - name: With autoscaling
     project_root: examples/autoscaler
     environment:

README.md

@@ -11,6 +11,7 @@ This module supports both SaaS and self-hosted Spacelift deployments, and can op
 - Optional autoscaling based on worker pool queue length
 - Secure storage of credentials using AWS Secrets Manager
 - Support for ARM64 instances for cost optimization
+- Spot instance support for significant cost savings (up to 90%)
 - Instance lifecycle management with worker draining
 - Configurable instance types, volume sizes, and more
 - "Bring Your Own" (BYO) options for SSM parameters and Secrets Manager
@@ -26,7 +27,7 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = "~> 5.0"
+      version = "~> 6.0"
     }
   }
 }
@@ -36,7 +37,7 @@ provider "aws" {
 }
 
 module "spacelift_workerpool" {
-  source = "github.com/spacelift-io/terraform-aws-spacelift-workerpool-on-ec2?ref=v4.4.3"
+  source = "github.com/spacelift-io/terraform-aws-spacelift-workerpool-on-ec2?ref=v5.2.0"
 
   secure_env_vars = {
     SPACELIFT_TOKEN            = var.worker_pool_config
@@ -59,6 +60,7 @@ For more examples covering specific use cases, please see the [examples director
 
 - [AMD64 deployment](./examples/amd64/)
 - [ARM64 deployment](./examples/arm64/)
+- [Spot instances for cost optimization](./examples/spot-instances/)
 - [Autoscaler configuration](./examples/autoscaler/)
 - [Custom S3 package for autoscaler](./examples/autoscaler-custom-s3-package/)
 - [BYO SSM and Secrets Manager](./examples/byo-ssm-secretsmanager-with-autoscaling-and-lifecycle/)
@@ -184,6 +186,60 @@ This module includes a lifecycle management feature that ensures graceful termin
 
 To enable lifecycle management, provide the `spacelift_api_credentials` variable and configure the `instance_refresh` variable. See the [BYO SSM and Secrets Manager with Autoscaling and Lifecycle example](./examples/byo-ssm-secretsmanager-with-autoscaling-and-lifecycle/) for a complete configuration.
 
+## 💰 Spot Instances for Cost Optimization
+
+This module supports AWS Spot Instances, which can provide significant cost savings (up to 90%) compared to On-Demand instances. Spot instances use spare AWS capacity and are ideal for fault-tolerant and flexible workloads.
+
+> ⚠️ Spot instances are **NOT recommended for critical workloads** as they can be interrupted with only 2 minutes notice, potentially causing:
+> - Incomplete or corrupted Terraform state
+> - Failed deployments leaving infrastructure in inconsistent state
+> - Loss of work-in-progress for long-running operations
+
+### How to Enable Spot Instances
+
+Configure spot instances using the `instance_market_options` variable:
+
+```hcl
+module "spacelift_workerpool" {
+  source = "github.com/spacelift-io/terraform-aws-spacelift-workerpool-on-ec2"
+
+  # ... other configuration ...
+
+  # Enable spot instances
+  instance_market_options = {
+    market_type = "spot"
+    # spot_options = {
+    #  spot_instance_type             = "one-time"
+    #  instance_interruption_behavior = "terminate"
+    # }
+  }
+}
+```
+
+### Configuration Options
+
+- **max_price** (Optional): Maximum hourly price you're willing to pay. AWS recommends omitting this to use current Spot pricing, as setting a lower price can increase interruption frequency.
+- **spot_instance_type**: Use `"one-time"` for Auto Scaling Groups (recommended) or `"persistent"` for individual instances.
+- **instance_interruption_behavior**: How instances behave when interrupted - `"terminate"` (default), `"stop"`, or `"hibernate"`. For AutoScaling Groups, it's recommended to use `"terminate"`, as the ASG handles replacements automatically.
+
+These options use sensible defaults when omitted, so explicit configuration is typically unnecessary.
+
+Use the [AWS EC2 Spot Instance Advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) to select cost-effective instance types and understand interruption rates.
+
+### Graceful Spot Instance Interruption Handling
+
+The Spacelift worker includes built-in spot instance interruption detection to minimize job disruption:
+
+- **Automatic Monitoring**: The worker polls the EC2 Instance Metadata Service for spot interruption notices
+- **Graceful Shutdown**: When an interruption is detected, the worker:
+  - Exits immediately if idle (no active runs)
+  - If processing a run, allows the current run to finish without cancellation, then shuts down gracefully
+  - **Important**: If the run doesn't complete within the 2-minute interruption grace period, the run will be abruptly terminated and crash
+
+**Reference**: [AWS Spot Instance Termination Notices](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-instance-termination-notices.html)
+
+For a complete example, see the [spot instances example](./examples/spot-instances/).
+
 ## Default AMI
 
 The default AMI used by this module comes from the [spacelift-worker-image](https://github.com/spacelift-io/spacelift-worker-image) repository. You can find the full list of AMIs on the [releases](https://github.com/spacelift-io/spacelift-worker-image/releases) page.

examples/spot-instances/README.md

@@ -0,0 +1,31 @@
+# Spot Instances Example
+
+This example demonstrates how to configure the Spacelift worker pool to use EC2 spot instances for cost optimization.
+
+> ⚠️ Spot instances are **NOT recommended for critical workloads** as they can be interrupted with only 2 minutes notice, potentially causing:
+> - Incomplete or corrupted Terraform state
+> - Failed deployments leaving infrastructure in inconsistent state
+> - Loss of work-in-progress for long-running operations
+
+## Spot Instance Configuration
+
+The key configuration for spot instances is:
+
+```hcl
+instance_market_options = {
+  market_type = "spot"
+  spot_options = {
+    max_price                      = "0.05"  # Maximum price per hour in USD
+    spot_instance_type             = "one-time"
+    instance_interruption_behavior = "terminate"
+  }
+}
+```
+
+### Configuration Options
+
+- **max_price** (Optional): Maximum hourly price you're willing to pay. AWS recommends omitting this to use current Spot pricing, as setting a lower price can increase interruption frequency.
+- **spot_instance_type**: Use `"one-time"` for Auto Scaling Groups (recommended) or `"persistent"` for individual instances.
+- **instance_interruption_behavior**: How instances behave when interrupted - `"terminate"` (default), `"stop"`, or `"hibernate"`. For AutoScaling Groups, it's recommended to use `"terminate"`, as the ASG handles replacements automatically.
+
+These options use sensible defaults when omitted, so **explicit configuration is typically unnecessary**.

examples/spot-instances/main.tf

@@ -0,0 +1,67 @@
+terraform {
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 6.0.0"
+    }
+
+    random = { source = "hashicorp/random" }
+  }
+}
+
+provider "aws" {
+  region = "eu-west-1"
+
+  default_tags {
+    tags = {
+      TfModule = "terraform-aws-spacelift-workerpool-on-ec2"
+      TestCase = "spot-instances"
+    }
+  }
+}
+
+data "aws_vpc" "this" {
+  default = true
+}
+
+data "aws_security_group" "this" {
+  name   = "default"
+  vpc_id = data.aws_vpc.this.id
+}
+
+data "aws_subnets" "this" {
+  filter {
+    name   = "vpc-id"
+    values = [data.aws_vpc.this.id]
+  }
+}
+
+resource "random_string" "worker_pool_id" {
+  length  = 26
+  numeric = true
+  # Use special and override special to allow only uppercase letters and numbers
+  # but exclude I, L, O, and U as it does not conform to the regex used by Spacelift
+  special          = true
+  override_special = "ABCDEFGHJKMNPQRSTVWXYZ"
+  lower            = false
+  upper            = false
+}
+
+#### Spacelift worker pool with spot instances ####
+
+module "this" {
+  source = "../../"
+
+  secure_env_vars = {
+    SPACELIFT_TOKEN            = "<token-here>"
+    SPACELIFT_POOL_PRIVATE_KEY = "<private-key-here>"
+  }
+  security_groups = [data.aws_security_group.this.id]
+  vpc_subnets     = data.aws_subnets.this.ids
+  worker_pool_id  = random_string.worker_pool_id.id
+
+  # Enable spot instances
+  instance_market_options = {
+    market_type = "spot"
+  }
+}

variables.tf

@@ -237,7 +237,45 @@ variable "instance_refresh" {
 }
 
 variable "instance_market_options" {
-  description = "The market (purchasing) option for the instance"
+  description = <<EOF
+  The market (purchasing) option for the instance. Configuration block for setting 
+  up an instance on Spot vs On-Demand markets in the launch template.
+
+  Configuration:
+  - market_type: (Required) Type of market for the instance. Valid values: "spot"
+  - spot_options: (Optional) Block to configure the Spot request
+    - max_price: (Optional) The maximum hourly price you're willing to pay for the instance. 
+      If not specified, you will pay the current Spot price (recommended). AWS does not 
+      recommend using this parameter as it can lead to increased interruptions. 
+      The price will never exceed the On-Demand price. Format: "0.05" (string)
+    - spot_instance_type: (Optional) The Spot instance request type. For Auto Scaling Groups, 
+      use "one-time" as ASG handles requesting new instances. Valid values: 
+      "one-time" (recommended for ASG) or "persistent"
+    - instance_interruption_behavior: (Optional) Indicates Spot instance behavior when 
+      interrupted. Valid values: "terminate" (default), "stop", or "hibernate". 
+      For persistent requests, "stop" and "hibernate" are valid.
+    - block_duration_minutes: (Optional, Deprecated) The required duration for Spot block 
+      instances in minutes. Note: Spot blocks are deprecated by AWS.
+
+  Example configuration for spot instances:
+  {
+    market_type = "spot"
+    spot_options = {
+      spot_instance_type             = "one-time"
+      instance_interruption_behavior = "terminate"
+      # max_price omitted to use current Spot price (recommended)
+    }
+  }
+
+  ⚠️ WARNING: Spot instances are NOT recommended for critical workloads as they can be 
+  interrupted with only 2 minutes notice, potentially causing:
+  - Incomplete or corrupted Terraform state
+  - Failed deployments leaving infrastructure in inconsistent state
+  - Loss of work-in-progress for long-running operations
+  Use only for non-critical development, testing, or ephemeral workloads.
+
+  Note: Auto Scaling Groups only support 'one-time' Spot instance requests with no duration.
+  EOF
   type        = any
   default     = null
 }