Skip to content

Commit 7781147

Browse files
authored
feat!: Upgrade AWS provider and min required Terraform version to 6.0 and 1.5.7 respectively (#38)
* feat: Bump MSV of Terraform and AWS provider, update attribute lookup syntax, change policies from list to map * feat: Bump versions, add missing arguments and variable type definitions * feat: Add support for region arguments and upgrade guide * chore: Lower min supported Terraform version
1 parent c478369 commit 7781147

File tree

14 files changed

+927
-301
lines changed

14 files changed

+927
-301
lines changed

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
repos:
22
- repo: https://github.com/antonbabenko/pre-commit-terraform
3-
rev: v1.96.1
3+
rev: v1.99.4
44
hooks:
55
- id: terraform_fmt
66
- id: terraform_docs

README.md

Lines changed: 24 additions & 17 deletions
Large diffs are not rendered by default.

docs/UPGRADE-3.0.md

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Upgrade from v2.x to v3.x
2+
3+
If you have any questions regarding this upgrade process, please consult the `examples` directory:
4+
5+
- [EC2](https://github.com/terraform-aws-modules/terraform-aws-batch/tree/master/examples/ec2)
6+
- [Fargate](https://github.com/terraform-aws-modules/terraform-aws-batch/tree/master/examples/fargate)
7+
8+
If you find a bug, please open an issue with supporting configuration to reproduce.
9+
10+
## List of backwards incompatible changes
11+
12+
- Terraform v1.5.7 is now minimum supported version
13+
- AWS provider v6.0.0 is now minimum supported version
14+
- `instance_iam_role_additional_policies`, `service_iam_role_additional_policies`, `spot_fleet_iam_role_additional_policies` types are now `map(string)` instead of `list(string)`
15+
- IAM assume role policy SIDs have been modified to match their use (previously all were `ECSAssumeRole` which is inaccurate)
16+
- `compute_environment_order` is now a required argument for `aws_batch_job_queue` per the Batch API and replaces `compute_environments`
17+
18+
## Additional changes
19+
20+
### Added
21+
22+
- Support for `region` parameter to specify the AWS region for the resources created if different from the provider region.
23+
- Support for `compute_environment_order`, `job_state_time_limit_action`, `timeouts` arguments for job queues
24+
- All (currently) supported arguments for `eks_properties` argument have been added to the job definition resource
25+
- Support for `scheduling_priority` and `node_properties` arguments for job definitions
26+
27+
### Modified
28+
29+
- Variable definitions now contain detailed `object` types in place of the previously used any type.
30+
- `compute_environment_name` argument has been changed to `name` per provider `v6.x` API; no-op for users
31+
- `compute_environment_name_prefix` argument has been changed to `name_prefix` per provider `v6.x` API; no-op for users
32+
33+
### Removed
34+
35+
- None
36+
37+
### Variable and output changes
38+
39+
1. Removed variables:
40+
41+
- None
42+
43+
2. Renamed variables:
44+
45+
- None
46+
47+
3. Added variables:
48+
49+
- None
50+
51+
4. Removed outputs:
52+
53+
- None
54+
55+
5. Renamed outputs:
56+
57+
- None
58+
59+
6. Added outputs:
60+
61+
- None
62+
63+
## Upgrade State Migrations
64+
65+
### Before 2.x Example
66+
67+
```hcl
68+
module "batch" {
69+
source = "terraform-aws-modules/batch/aws"
70+
version = "2.1.0"
71+
72+
# Truncated for brevity, only relevant module API changes are shown ...
73+
74+
instance_iam_role_additional_policies = [
75+
"arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
76+
]
77+
78+
compute_environments = {
79+
a_ec2 = {
80+
... # Other properties for a_ec2 compute environment
81+
}
82+
83+
b_ec2_spot = {
84+
... # Other properties for b_ec2_spot compute environment
85+
}
86+
}
87+
88+
tags = local.tags
89+
}
90+
```
91+
92+
### After 3.x Example
93+
94+
```hcl
95+
module "batch" {
96+
source = "terraform-aws-modules/batch/aws"
97+
version = "3.0.0"
98+
99+
# Truncated for brevity, only relevant module API changes are shown ...
100+
101+
instance_iam_role_additional_policies = {
102+
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
103+
}
104+
105+
compute_environments = {
106+
a_ec2 = {
107+
... # Other properties for a_ec2 compute environment
108+
}
109+
110+
b_ec2_spot = {
111+
... # Other properties for b_ec2_spot compute environment
112+
}
113+
}
114+
# Now required
115+
compute_environment_order = {
116+
0 = {
117+
compute_environment_key = "a_ec2"
118+
}
119+
1 = {
120+
compute_environment_key = "b_ec2_spot"
121+
}
122+
}
123+
124+
tags = local.tags
125+
}
126+
```
127+
128+
To migrate from the `v2.x` version to `v3.x` version example shown above, the following state move commands can be performed to maintain the current resources without modification:
129+
130+
```bash
131+
# For each additional policy in instance_iam_role_additional_policies, simply move the prior value to the new key you have defined in your configuration
132+
# This can be done similarly for aws_iam_role_policy_attachment.service and aws_iam_role_policy_attachment.spot_fleet
133+
terraform state mv \
134+
'module.batch.aws_iam_role_policy_attachment.instance["arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"]' \
135+
'module.batch.aws_iam_role_policy_attachment.instance["AmazonSSMManagedInstanceCore"]'
136+
```

examples/ec2/README.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,31 +24,30 @@ Note that this example may create resources which will incur monetary charges on
2424

2525
| Name | Version |
2626
|------|---------|
27-
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
28-
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0 |
27+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.5.7 |
28+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 6.0 |
2929

3030
## Providers
3131

3232
| Name | Version |
3333
|------|---------|
34-
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0 |
34+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 6.0 |
3535

3636
## Modules
3737

3838
| Name | Source | Version |
3939
|------|--------|---------|
4040
| <a name="module_batch"></a> [batch](#module\_batch) | ../.. | n/a |
4141
| <a name="module_batch_disabled"></a> [batch\_disabled](#module\_batch\_disabled) | ../.. | n/a |
42-
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 4.0 |
43-
| <a name="module_vpc_endpoint_security_group"></a> [vpc\_endpoint\_security\_group](#module\_vpc\_endpoint\_security\_group) | terraform-aws-modules/security-group/aws | ~> 4.0 |
44-
| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 4.0 |
42+
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 6.0 |
43+
| <a name="module_vpc_endpoints"></a> [vpc\_endpoints](#module\_vpc\_endpoints) | terraform-aws-modules/vpc/aws//modules/vpc-endpoints | ~> 6.0 |
4544

4645
## Resources
4746

4847
| Name | Type |
4948
|------|------|
5049
| [aws_cloudwatch_log_group.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
51-
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
50+
| [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
5251

5352
## Inputs
5453

examples/ec2/main.tf

Lines changed: 62 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,14 @@ provider "aws" {
22
region = local.region
33
}
44

5+
data "aws_availability_zones" "available" {}
6+
57
locals {
68
region = "us-east-1"
7-
name = "batch-ex-${replace(basename(path.cwd), "_", "-")}"
9+
name = "batch-ex-${basename(path.cwd)}"
10+
11+
vpc_cidr = "10.0.0.0/16"
12+
azs = slice(data.aws_availability_zones.available.names, 0, 3)
813

914
tags = {
1015
Name = local.name
@@ -13,8 +18,6 @@ locals {
1318
}
1419
}
1520

16-
data "aws_region" "current" {}
17-
1821
################################################################################
1922
# Batch Module
2023
################################################################################
@@ -31,9 +34,9 @@ module "batch" {
3134
instance_iam_role_name = "${local.name}-ecs-instance"
3235
instance_iam_role_path = "/batch/"
3336
instance_iam_role_description = "IAM instance role/profile for AWS Batch ECS instance(s)"
34-
instance_iam_role_additional_policies = [
35-
"arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
36-
]
37+
instance_iam_role_additional_policies = {
38+
AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
39+
}
3740
instance_iam_role_tags = {
3841
ModuleCreatedRole = "Yes"
3942
}
@@ -64,7 +67,7 @@ module "batch" {
6467
desired_vcpus = 4
6568
instance_types = ["m5.large", "r5.large"]
6669

67-
security_group_ids = [module.vpc_endpoint_security_group.security_group_id]
70+
security_group_ids = [module.vpc_endpoints.security_group_id]
6871
subnets = module.vpc.private_subnets
6972

7073
# Note - any tag changes here will force compute environment replacement
@@ -91,7 +94,7 @@ module "batch" {
9194
desired_vcpus = 4
9295
instance_types = ["m4.large", "m3.large", "r4.large", "r3.large"]
9396

94-
security_group_ids = [module.vpc_endpoint_security_group.security_group_id]
97+
security_group_ids = [module.vpc_endpoints.security_group_id]
9598
subnets = module.vpc.private_subnets
9699

97100
# Note - any tag changes here will force compute environment replacement
@@ -118,7 +121,14 @@ module "batch" {
118121
state = "ENABLED"
119122
priority = 1
120123

121-
compute_environments = ["b_ec2_spot"]
124+
compute_environment_order = {
125+
0 = {
126+
compute_environment_key = "b_ec2_spot"
127+
}
128+
1 = {
129+
compute_environment_key = "a_ec2"
130+
}
131+
}
122132

123133
tags = {
124134
JobQueue = "Low priority job queue"
@@ -130,6 +140,12 @@ module "batch" {
130140
state = "ENABLED"
131141
priority = 99
132142

143+
compute_environment_order = {
144+
0 = {
145+
compute_environment_key = "a_ec2"
146+
}
147+
}
148+
133149
fair_share_policy = {
134150
compute_reservation = 1
135151
share_decay_seconds = 3600
@@ -201,88 +217,60 @@ module "batch" {
201217

202218
module "vpc" {
203219
source = "terraform-aws-modules/vpc/aws"
204-
version = "~> 4.0"
220+
version = "~> 6.0"
205221

206222
name = local.name
207-
cidr = "10.99.0.0/18"
223+
cidr = local.vpc_cidr
208224

209-
azs = ["${local.region}a", "${local.region}b", "${local.region}c"]
210-
public_subnets = ["10.99.0.0/24", "10.99.1.0/24", "10.99.2.0/24"]
211-
private_subnets = ["10.99.3.0/24", "10.99.4.0/24", "10.99.5.0/24"]
225+
azs = local.azs
226+
private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
227+
public_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
212228

213229
enable_nat_gateway = true
214230
single_nat_gateway = true
215231

216-
public_route_table_tags = { Name = "${local.name}-public" }
217-
public_subnet_tags = { Name = "${local.name}-public" }
218-
private_route_table_tags = { Name = "${local.name}-private" }
219-
private_subnet_tags = { Name = "${local.name}-private" }
220-
221-
enable_dhcp_options = true
222-
enable_dns_hostnames = true
223-
dhcp_options_domain_name = data.aws_region.current.name == "us-east-1" ? "ec2.internal" : "${data.aws_region.current.name}.compute.internal"
224-
225232
tags = local.tags
226233
}
227234

228235
module "vpc_endpoints" {
229236
source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
230-
version = "~> 4.0"
231-
232-
vpc_id = module.vpc.vpc_id
233-
security_group_ids = [module.vpc_endpoint_security_group.security_group_id]
234-
235-
endpoints = {
236-
ecr_api = {
237-
service = "ecr.api"
238-
private_dns_enabled = true
239-
subnet_ids = module.vpc.private_subnets
240-
}
241-
ecr_dkr = {
242-
service = "ecr.dkr"
243-
private_dns_enabled = true
244-
subnet_ids = module.vpc.private_subnets
245-
}
246-
ecs = {
247-
service = "ecs"
248-
private_dns_enabled = true
249-
subnet_ids = module.vpc.private_subnets
250-
}
251-
ssm = {
252-
service = "ssm"
253-
private_dns_enabled = true
254-
subnet_ids = module.vpc.private_subnets
255-
}
256-
s3 = {
257-
service = "s3"
258-
service_type = "Gateway"
259-
route_table_ids = module.vpc.private_route_table_ids
237+
version = "~> 6.0"
238+
239+
vpc_id = module.vpc.vpc_id
240+
241+
# Security group
242+
create_security_group = true
243+
security_group_name_prefix = "${local.name}-vpc-endpoints-"
244+
security_group_description = "VPC endpoint security group"
245+
security_group_rules = {
246+
ingress_https = {
247+
description = "HTTPS from VPC"
248+
cidr_blocks = [module.vpc.vpc_cidr_block]
260249
}
261250
}
262251

263-
tags = local.tags
264-
}
265-
266-
module "vpc_endpoint_security_group" {
267-
source = "terraform-aws-modules/security-group/aws"
268-
version = "~> 4.0"
269-
270-
name = "${local.name}-vpc-endpoint"
271-
description = "Security group for VPC endpoints"
272-
vpc_id = module.vpc.vpc_id
273-
274-
ingress_with_self = [
252+
endpoints = merge(
275253
{
276-
from_port = 443
277-
to_port = 443
278-
protocol = "tcp"
279-
description = "Container to VPC endpoint service"
280-
self = true
254+
s3 = {
255+
service = "s3"
256+
service_type = "Gateway"
257+
route_table_ids = module.vpc.private_route_table_ids
258+
tags = {
259+
Name = "${local.name}-s3"
260+
}
261+
}
281262
},
282-
]
283-
284-
egress_cidr_blocks = ["0.0.0.0/0"]
285-
egress_rules = ["https-443-tcp"]
263+
{
264+
for service in toset(["ecr.api", "ecr.dkr", "ecs", "ssm"]) :
265+
replace(service, ".", "_") =>
266+
{
267+
service = service
268+
subnet_ids = module.vpc.private_subnets
269+
private_dns_enabled = true
270+
tags = { Name = "${local.name}-${service}" }
271+
}
272+
}
273+
)
286274

287275
tags = local.tags
288276
}

0 commit comments

Comments
 (0)