Skip to content

Latest commit

 

History

History
1350 lines (1211 loc) · 57.2 KB

README.md

File metadata and controls

1350 lines (1211 loc) · 57.2 KB
Raw

Project Module

This module implements the creation and management of one GCP project including IAM, organization policies, Shared VPC host or service attachment, service API activation, and tag attachment. It also offers a convenient way to refer to managed service identities (aka robot service accounts) for APIs.

TOC

Basic Project Creation

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
}
# tftest modules=1 resources=3 inventory=basic.yaml e2e

IAM

IAM is managed via several variables that implement different features and levels of control:

  • iam and iam_by_principals configure authoritative bindings that manage individual roles exclusively, and are internally merged
  • iam_bindings configure authoritative bindings with optional support for conditions, and are not internally merged with the previous two variables
  • iam_bindings_additive configure additive bindings via individual role/member pairs with optional support conditions

The authoritative and additive approaches can be used together, provided different roles are managed by each. Some care must also be taken with the iam_by_principals variable to ensure that variable keys are static values, so that Terraform is able to compute the dependency graph.

Be mindful about service identity roles when using authoritative IAM, as you might inadvertently remove a role from a service identity or default service account. For example, using roles/editor with iam or iam_principals will remove the default permissions for the Cloud Services identity. A simple workaround for these scenarios is described below.

Authoritative IAM

The iam variable is based on role keys and is typically used for service accounts, or where member values can be dynamic and would create potential problems in the underlying for_each cycle.

locals {
  gke_service_account = "my_gke_service_account"
}

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
  iam = {
    "roles/container.hostServiceAgentUser" = [
      "serviceAccount:${local.gke_service_account}"
    ]
  }
}
# tftest modules=1 resources=4 inventory=iam-authoritative.yaml

The iam_by_principals variable uses principals as keys and is a convenient way to assign roles to humans following Google's best practices. The end result is readable code that also serves as documentation.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  iam_by_principals = {
    "group:${var.group_email}" = [
      "roles/cloudasset.owner",
      "roles/cloudsupport.techSupportEditor",
      "roles/iam.securityReviewer",
      "roles/logging.admin",
    ]
  }
}
# tftest modules=1 resources=5 inventory=iam-group.yaml e2e

The iam_bindings variable behaves like a more verbose version of iam, and allows setting binding-level IAM conditions.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
  iam_bindings = {
    iam_admin_conditional = {
      members = [
        "group:${var.group_email}"
      ]
      role = "roles/resourcemanager.projectIamAdmin"
      condition = {
        title      = "delegated_network_user_one"
        expression = <<-END
          api.getAttribute(
            'iam.googleapis.com/modifiedGrantsByRole', []
          ).hasOnly([
            'roles/compute.networkAdmin'
          ])
        END
      }
    }
  }
}
# tftest modules=1 resources=4 inventory=iam-bindings.yaml e2e

Additive IAM

Additive IAM is typically used where bindings for specific roles are controlled by different modules or in different Terraform stages. One common example is a host project managed by the networking team, and a project factory that manages service projects and needs to assign roles/networkUser on the host project.

The iam_bindings_additive variable allows setting individual role/principal binding pairs. Support for IAM conditions is implemented like for iam_bindings above.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "compute.googleapis.com"
  ]
  iam_bindings_additive = {
    group-owner = {
      member = "group:${var.group_email}"
      role   = "roles/owner"
    }
  }
}
# tftest modules=1 resources=3 inventory=iam-bindings-additive.yaml e2e

Service Identities and Authoritative IAM

As mentioned above, there are cases where authoritative management of specific IAM roles results in removal of default bindings from service identities. One example is outlined below, with a simple workaround leveraging the service_accounts output to identify the service identity. A full list of service identities and their roles can be found here.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  iam = {
    "roles/editor" = [
      "serviceAccount:${module.project.service_accounts.cloud_services}"
    ]
  }
}
# tftest modules=1 resources=2 e2e

Service Identities Requiring Manual IAM Grants

The module will create service identities at project creation instead of creating of them at the time of first use. This allows granting these service identities roles in other projects, something which is usually necessary in a Shared VPC context.

You can grant roles to service identities using the following construct:

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "apigee.googleapis.com",
  ]
  iam = {
    "roles/apigee.serviceAgent" = [
      "serviceAccount:${module.project.service_accounts.robots.apigee}"
    ]
  }
}
# tftest modules=1 resources=4 e2e

This table lists all affected services and roles that you need to grant to service identities

service service identity role
apigee.googleapis.com apigee roles/apigee.serviceAgent
artifactregistry.googleapis.com artifactregistry roles/artifactregistry.serviceAgent
cloudasset.googleapis.com cloudasset roles/cloudasset.serviceAgent
cloudbuild.googleapis.com cloudbuild roles/cloudbuild.builds.builder
connectors.googleapis.com connectors roles/connectors.serviceAgent
dataform.googleapis.com dataform roles/dataform.serviceAgent
dataplex.googleapis.com dataplex roles/dataplex.serviceAgent
dlp.googleapis.com dlp roles/dlp.serviceAgent
gkehub.googleapis.com fleet roles/gkehub.serviceAgent
meshconfig.googleapis.com servicemesh roles/anthosservicemesh.serviceAgent
multiclusteringress.googleapis.com multicluster-ingress roles/multiclusteringress.serviceAgent
pubsub.googleapis.com pubsub roles/pubsub.serviceAgent
sqladmin.googleapis.com sqladmin roles/cloudsql.serviceAgent

Shared VPC

The module allows managing Shared VPC status for both hosts and service projects, and control of IAM bindings for API service identities.

Project service association for VPC host projects can be

  • authoritatively managed in the host project by enabling Shared VPC and specifying the set of service projects, or
  • additively managed in service projects by enabling Shared VPC in the host project and then "attaching" each service project independently

IAM bindings in the host project for API service identities can be managed from service projects in two different ways:

  • via the service_identity_iam attribute, by specifying the set of roles and service agents
  • via the service_iam_grants attribute that leverages a fixed list of roles for each service, by specifying a list of services
  • via the service_identity_subnet_iam attribute, by providing a map of "<region>/<subnet_name>" -> [ "<service_identity>", (...)], to grant compute.networkUser role on subnet level to service identity

While the first method is more explicit and readable, the second method is simpler and less error prone as all appropriate roles are predefined for all required service agents (eg compute and cloud services). You can mix and match as the two sets of bindings are then internally combined.

This example shows a simple configuration with a host project, and a service project independently attached with granular IAM bindings for service identities.

module "host-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "host"
  parent          = var.folder_id
  prefix          = var.prefix
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "service"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "run.googleapis.com"
  ]
  shared_vpc_service_config = {
    host_project = module.host-project.project_id
    service_identity_iam = {
      "roles/compute.networkUser" = [
        "cloudservices", "container-engine"
      ]
      "roles/vpcaccess.user" = [
        "cloudrun"
      ]
      "roles/container.hostServiceAgentUser" = [
        "container-engine"
      ]
    }
  }
}
# tftest modules=2 resources=10 inventory=shared-vpc.yaml e2e

This example shows a similar configuration, with the simpler way of defining IAM bindings for service identities. The list of services passed to service_iam_grants uses the same module's outputs to establish a dependency, as service identities are only typically available after service (API) activation.

module "host-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "host"
  parent          = var.folder_id
  prefix          = var.prefix
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "service"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
  ]
  shared_vpc_service_config = {
    host_project = module.host-project.project_id
    # reuse the list of services from the module's outputs
    service_iam_grants = module.service-project.services
  }
}
# tftest modules=2 resources=9 inventory=shared-vpc-auto-grants.yaml e2e

The compute.networkUser role for identities other than API services (e.g. users, groups or service accounts) can be managed via the network_users attribute, by specifying the list of identities. Avoid using dynamically generated lists, as this attribute is involved in a for_each loop and may result in Terraform errors.

Note that this configuration grants the role at project level which results in the identities being able to configure resources on all the VPCs and subnets belonging to the host project. The most reliable way to restrict which subnets can be used on the newly created project is via the compute.restrictSharedVpcSubnetworks organization policy. For more information on the Org Policy configuration check the corresponding Organization Policy section. The following example details this configuration.

module "host-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "host"
  parent          = var.folder_id
  prefix          = var.prefix
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "service"
  parent          = var.folder_id
  prefix          = var.prefix
  org_policies = {
    "compute.restrictSharedVpcSubnetworks" = {
      rules = [{
        allow = {
          values = ["projects/host/regions/europe-west1/subnetworks/prod-default-ew1"]
        }
      }]
    }
  }
  services = [
    "container.googleapis.com",
  ]
  shared_vpc_service_config = {
    host_project  = module.host-project.project_id
    network_users = ["group:${var.group_email}"]
    # reuse the list of services from the module's outputs
    service_iam_grants = module.service-project.services
  }
}
# tftest modules=2 resources=11 inventory=shared-vpc-host-project-iam.yaml e2e

In specific cases it might make sense to selectively grant the compute.networkUser role for service identities at the subnet level, and while that is best done via org policies it's also supported by this module. In this example, Compute service identity and [email protected] Google Group will be granted compute.networkUser in the gce subnet defined in europe-west1 region in the host project (not included in the example) via the service_identity_subnet_iam and network_subnet_users attributes.

module "host-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "host"
  parent          = var.folder_id
  prefix          = var.prefix
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "service"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "compute.googleapis.com",
  ]
  shared_vpc_service_config = {
    host_project = module.host-project.project_id
    service_identity_subnet_iam = {
      "europe-west1/gce" = ["compute"]
    }
    network_subnet_users = {
      "europe-west1/gce" = ["group:[email protected]"]
    }
  }
}
# tftest modules=2 resources=7 inventory=shared-vpc-subnet-grants.yaml

Organization Policies

To manage organization policies, the orgpolicy.googleapis.com service should be enabled in the quota project.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  org_policies = {
    "compute.disableGuestAttributesAccess" = {
      rules = [{ enforce = true }]
    }
    "compute.skipDefaultNetworkCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyUpload" = {
      rules = [
        {
          condition = {
            expression  = "resource.matchTagId('tagKeys/1234', 'tagValues/1234')"
            title       = "condition"
            description = "test condition"
            location    = "somewhere"
          }
          enforce = true
        },
        {
          enforce = false
        }
      ]
    }
    "iam.allowedPolicyMemberDomains" = {
      rules = [{
        allow = {
          values = ["C0xxxxxxx", "C0yyyyyyy"]
        }
      }]
    }
    "compute.trustedImageProjects" = {
      rules = [{
        allow = {
          values = ["projects/my-project"]
        }
      }]
    }
    "compute.vmExternalIpAccess" = {
      rules = [{ deny = { all = true } }]
    }
  }
}
# tftest modules=1 resources=8 inventory=org-policies.yaml e2e

Organization Policy Factory

Organization policies can be loaded from a directory containing YAML files where each file defines one or more constraints. The structure of the YAML files is exactly the same as the org_policies variable.

Note that constraints defined via org_policies take precedence over those in org_policies_data_path. In other words, if you specify the same constraint in a YAML file and in the org_policies variable, the latter will take priority.

The example below deploys a few organization policies split between two YAML files.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  factories_config = {
    org_policies = "configs/org-policies/"
  }
}
# tftest modules=1 resources=8 files=boolean,list inventory=org-policies.yaml e2e
# tftest-file id=boolean path=configs/org-policies/boolean.yaml

---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.

compute.disableGuestAttributesAccess:
  rules:
  - enforce: true
compute.skipDefaultNetworkCreation:
  rules:
  - enforce: true
iam.disableServiceAccountKeyCreation:
  rules:
  - enforce: true
iam.disableServiceAccountKeyUpload:
  rules:
  - condition:
      description: test condition
      expression: resource.matchTagId('tagKeys/1234', 'tagValues/1234')
      location: somewhere
      title: condition
    enforce: true
  - enforce: false
# tftest-file id=list path=configs/org-policies/list.yaml

---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.

compute.trustedImageProjects:
  rules:
  - allow:
      values:
      - projects/my-project
compute.vmExternalIpAccess:
  rules:
  - deny:
      all: true
iam.allowedPolicyMemberDomains:
  rules:
  - allow:
      values:
      - C0xxxxxxx
      - C0yyyyyyy

Log Sinks

module "gcs" {
  source        = "./fabric/modules/gcs"
  project_id    = var.project_id
  name          = "gcs_sink"
  location      = "EU"
  prefix        = var.prefix
  force_destroy = true
}

module "dataset" {
  source     = "./fabric/modules/bigquery-dataset"
  project_id = var.project_id
  id         = "bq_sink"
  options    = { delete_contents_on_destroy = true }
}

module "pubsub" {
  source     = "./fabric/modules/pubsub"
  project_id = var.project_id
  name       = "pubsub_sink"
}

module "bucket" {
  source      = "./fabric/modules/logging-bucket"
  parent_type = "project"
  parent      = var.project_id
  id          = "${var.prefix}-bucket"
}

module "destination-project" {
  source          = "./fabric/modules/project"
  name            = "dest-prj"
  billing_account = var.billing_account_id
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "logging.googleapis.com"
  ]
}

module "project-host" {
  source          = "./fabric/modules/project"
  name            = "project"
  billing_account = var.billing_account_id
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "logging.googleapis.com"
  ]
  logging_sinks = {
    warnings = {
      destination = module.gcs.id
      filter      = "severity=WARNING"
      type        = "storage"
    }
    info = {
      destination = module.dataset.id
      filter      = "severity=INFO"
      type        = "bigquery"
    }
    notice = {
      destination = module.pubsub.id
      filter      = "severity=NOTICE"
      type        = "pubsub"
    }
    debug = {
      destination = module.bucket.id
      filter      = "severity=DEBUG"
      exclusions = {
        no-compute = "logName:compute"
      }
      type = "logging"
    }
    alert = {
      destination = module.destination-project.id
      filter      = "severity=ALERT"
      type        = "project"
    }
  }
  logging_exclusions = {
    no-gce-instances = "resource.type=gce_instance"
  }
}
# tftest modules=6 resources=19 inventory=logging.yaml e2e

Data Access Logs

Activation of data access logs can be controlled via the logging_data_access variable. If the iam_bindings_authoritative variable is used to set a resource-level IAM policy, the data access log configuration will also be authoritative as part of the policy.

This example shows how to set a non-authoritative access log configuration:

module "project" {
  source          = "./fabric/modules/project"
  name            = "project"
  billing_account = var.billing_account_id
  parent          = var.folder_id
  prefix          = var.prefix
  logging_data_access = {
    allServices = {
      # logs for principals listed here will be excluded
      ADMIN_READ = ["group:${var.group_email}"]
    }
    "storage.googleapis.com" = {
      DATA_READ  = []
      DATA_WRITE = []
    }
  }
}
# tftest modules=1 resources=3 inventory=logging-data-access.yaml e2e

Cloud KMS Encryption Keys

The module offers a simple, centralized way to assign roles/cloudkms.cryptoKeyEncrypterDecrypter to service identities.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  prefix          = var.prefix
  parent          = var.folder_id
  services = [
    "compute.googleapis.com",
    "storage.googleapis.com"
  ]
  service_encryption_key_ids = {
    compute = [
      var.kms_key.id
    ]
    storage = [
      var.kms_key.id
    ]
  }
}
# tftest modules=1 resources=6 e2e

Tags

Refer to the Creating and managing tags documentation for details on usage.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  prefix          = var.prefix
  parent          = var.folder_id
  services = [
    "compute.googleapis.com",
  ]
  tags = {
    environment = {
      description = "Environment specification."
      iam = {
        "roles/resourcemanager.tagAdmin" = ["group:${var.group_email}"]
      }
      iam_bindings = {
        viewer = {
          role    = "roles/resourcemanager.tagViewer"
          members = ["group:[email protected]"]
        }
      }
      iam_bindings_additive = {
        user_app1 = {
          role   = "roles/resourcemanager.tagUser"
          member = "group:[email protected]"
        }
      }
      values = {
        dev = {
          iam_bindings_additive = {
            user_app2 = {
              role   = "roles/resourcemanager.tagUser"
              member = "group:[email protected]"
            }
          }
        }
        prod = {
          description = "Environment: production."
          iam = {
            "roles/resourcemanager.tagViewer" = ["group:[email protected]"]
          }
          iam_bindings = {
            admin = {
              role    = "roles/resourcemanager.tagAdmin"
              members = ["group:[email protected]"]
              condition = {
                title      = "gcp_support"
                expression = <<-END
                  request.time.getHours("Europe/Berlin") <= 9 &&
                  request.time.getHours("Europe/Berlin") >= 17
                END
              }
            }
          }
        }
      }
    }
  }
  tag_bindings = {
    env-prod = module.project.tag_values["environment/prod"].id
  }
}
# tftest modules=1 resources=12 inventory=tags.yaml

You can also define network tags through the dedicated network_tags variable:

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  prefix          = var.prefix
  parent          = var.folder_id
  services = [
    "compute.googleapis.com"
  ]
  network_tags = {
    net-environment = {
      description = "This is a network tag."
      network     = "${var.project_id}/${var.vpc.name}"
      iam = {
        "roles/resourcemanager.tagAdmin" = ["group:${var.group_email}"]
      }
      values = {
        dev = {}
        prod = {
          description = "Environment: production."
          iam = {
            "roles/resourcemanager.tagUser" = ["group:${var.group_email}"]
          }
        }
      }
    }
  }
}
# tftest modules=1 resources=7 inventory=tags-network.yaml

Tag Bindings

You can bind secure tags to a project with the tag_bindings attribute

module "org" {
  source          = "./fabric/modules/organization"
  organization_id = var.organization_id
  tags = {
    environment = {
      description = "Environment specification."
      values = {
        dev  = {}
        prod = {}
      }
    }
  }
}

module "project" {
  source = "./fabric/modules/project"
  name   = "project"
  parent = var.folder_id
  tag_bindings = {
    env-prod = module.org.tag_values["environment/prod"].id
    foo      = "tagValues/12345678"
  }
}
# tftest modules=2 resources=6

Project-scoped Tags

To create project-scoped secure tags, use the tags and network_tags attributes.

module "project" {
  source = "./fabric/modules/project"
  name   = "project"
  parent = var.folder_id
  tags = {
    mytag1 = {}
    mytag2 = {
      iam = {
        "roles/resourcemanager.tagAdmin" = ["user:[email protected]"]
      }
      values = {
        myvalue1 = {}
        myvalue2 = {
          iam = {
            "roles/resourcemanager.tagUser" = ["user:[email protected]"]
          }
        }
      }
    }
  }
  network_tags = {
    my_net_tag = {
      network = "${var.project_id}/${var.vpc.name}"
    }
  }
}
# tftest modules=1 resources=8

Custom Roles

Custom roles can be defined via the custom_roles variable, and referenced via the custom_role_id output (this also provides explicit dependency on the custom role):

module "project" {
  source = "./fabric/modules/project"
  name   = "project"
  custom_roles = {
    "myRole" = [
      "compute.instances.list",
    ]
  }
  iam = {
    (module.project.custom_role_id.myRole) = ["group:${var.group_email}"]
  }
}
# tftest modules=1 resources=3

Custom Roles Factory

Custom roles can also be specified via a factory in a similar way to organization policies and policy constraints. Each file is mapped to a custom role, where

  • the role name defaults to the file name but can be overridden via a name attribute in the yaml
  • role permissions are defined in an includedPermissions map

Custom roles defined via the variable are merged with those coming from the factory, and override them in case of duplicate names.

module "project" {
  source = "./fabric/modules/project"
  name   = "project"
  factories_config = {
    custom_roles = "data/custom_roles"
  }
}
# tftest modules=1 resources=3 files=custom-role-1,custom-role-2
# tftest-file id=custom-role-1 path=data/custom_roles/test_1.yaml

includedPermissions:
 - compute.globalOperations.get
# tftest-file id=custom-role-2 path=data/custom_roles/test_2.yaml

name: projectViewer
includedPermissions:
  - resourcemanager.projects.get
  - resourcemanager.projects.getIamPolicy
  - resourcemanager.projects.list

Quotas

Project and regional quotas can be managed via the quotas variable. Keep in mind, that metrics returned by gcloud compute regions describe do not match quota_ids. To get a list of quotas in the project use the API call, for example to get quotas for compute.googleapis.com use:

curl -X GET \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "X-Goog-User-Project: ${PROJECT_ID}" \
  "https://cloudquotas.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/services/compute.googleapis.com/quotaInfos?pageSize=1000" \
  | grep quotaId
module "project" {
  source          = "./fabric/modules/project"
  name            = "project"
  billing_account = var.billing_account_id
  parent          = var.folder_id
  prefix          = var.prefix
  quotas = {
    cpus-ew8 = {
      service         = "compute.googleapis.com"
      quota_id        = "CPUS-per-project-region"
      contact_email   = "[email protected]"
      preferred_value = 321
      dimensions = {
        region = "europe-west8"
      }
    }
  }
  services = [
    "cloudquotas.googleapis.com",
    "compute.googleapis.com"
  ]
}
# tftest modules=1 resources=4 inventory=quotas.yaml e2e

Quotas factory

Quotas can be also specified via a factory in a similar way to organization policies, policy constraints and custom roles by pointing to a directory containing YAML files where each file defines one or more quotas. The structure of the YAML files is exactly the same as the quotas variable.

module "project" {
  source          = "./fabric/modules/project"
  name            = "project"
  billing_account = var.billing_account_id
  parent          = var.folder_id
  prefix          = var.prefix
  factories_config = {
    quotas = "data/quotas"
  }
  services = [
    "cloudquotas.googleapis.com",
    "compute.googleapis.com"
  ]
}
# tftest modules=1 resources=4 files=quota-cpus-ew8 inventory=quotas.yaml e2e
# tftest-file id=quota-cpus-ew8 path=data/quotas/cpus-ew8.yaml

---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.

cpus-ew8:
  service: compute.googleapis.com
  quota_id: CPUS-per-project-region
  contact_email: [email protected]
  preferred_value: 321
  dimensions:
    region: europe-west8

VPC Service Controls

This module also allows managing project membership in VPC Service Controls perimeters. When using this functionality care should be taken so that perimeter management (e.g. via the vpc-sc module) does not try reconciling resources, to avoid permadiffs and related violations.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
  vpc_sc = {
    perimeter_name = "accessPolicies/1234567890/servicePerimeters/default"
  }
}
# tftest modules=1 resources=4 inventory=vpc-sc.yaml

Perimeter bridges and dry run configuration are also supported.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
  vpc_sc = {
    perimeter_name = "accessPolicies/1234567890/servicePerimeters/default"
    perimeter_bridges = [
      "accessPolicies/1234567890/servicePerimeters/b1",
      "accessPolicies/1234567890/servicePerimeters/b2",
    ]
    is_dry_run = true
  }
}
# tftest modules=1 resources=6

Outputs

Most of this module's outputs depend on its resources, to allow Terraform to compute all dependencies required for the project to be correctly configured. This allows you to reference outputs like project_id in other modules or resources without having to worry about setting depends_on blocks manually.

One non-obvious output is service_accounts, which offers a simple way to discover service identities and default service accounts, and guarantees that service identities that require an API call to trigger creation (like GCS or BigQuery) exist before use.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  prefix          = var.prefix
  parent          = var.folder_id
  services = [
    "compute.googleapis.com"
  ]
}

output "compute_robot" {
  value = module.project.service_accounts.robots.compute
}
# tftest modules=1 resources=2 inventory=outputs.yaml e2e

Managing project related configuration without creating it

The module offers managing all related resources without ever touching the project itself by using project_create = false

module "create-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
}

module "project" {
  source          = "./fabric/modules/project"
  depends_on      = [module.create-project]
  billing_account = var.billing_account_id
  name            = "project"
  parent          = var.folder_id
  prefix          = var.prefix
  project_create  = false

  iam_by_principals = {
    "group:${var.group_email}" = [
      "roles/cloudasset.owner",
      "roles/cloudsupport.techSupportEditor",
      "roles/iam.securityReviewer",
      "roles/logging.admin",
    ]
  }
  iam_bindings = {
    iam_admin_conditional = {
      members = [
        "group:${var.group_email}"
      ]
      role = "roles/resourcemanager.projectIamAdmin"
      condition = {
        title      = "delegated_network_user_one"
        expression = <<-END
          api.getAttribute(
            'iam.googleapis.com/modifiedGrantsByRole', []
          ).hasOnly([
            'roles/compute.networkAdmin'
          ])
        END
      }
    }
  }
  iam_bindings_additive = {
    group-owner = {
      member = "group:${var.group_email}"
      role   = "roles/owner"
    }
  }
  iam = {
    "roles/editor" = [
      "serviceAccount:${module.project.service_accounts.cloud_services}"
    ]
    "roles/apigee.serviceAgent" = [
      "serviceAccount:${module.project.service_accounts.robots.apigee}"
    ]
  }
  logging_data_access = {
    allServices = {
      # logs for principals listed here will be excluded
      ADMIN_READ = ["group:${var.group_email}"]
    }
    "storage.googleapis.com" = {
      DATA_READ  = []
      DATA_WRITE = []
    }
  }
  logging_sinks = {
    warnings = {
      destination = module.gcs.id
      filter      = "severity=WARNING"
      type        = "storage"
    }
    info = {
      destination = module.dataset.id
      filter      = "severity=INFO"
      type        = "bigquery"
    }
    notice = {
      destination = module.pubsub.id
      filter      = "severity=NOTICE"
      type        = "pubsub"
    }
    debug = {
      destination = module.bucket.id
      filter      = "severity=DEBUG"
      exclusions = {
        no-compute = "logName:compute"
      }
      type = "logging"
    }
  }
  logging_exclusions = {
    no-gce-instances = "resource.type=gce_instance"
  }
  org_policies = {
    "compute.disableGuestAttributesAccess" = {
      rules = [{ enforce = true }]
    }
    "compute.skipDefaultNetworkCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyUpload" = {
      rules = [
        {
          condition = {
            expression  = "resource.matchTagId('tagKeys/1234', 'tagValues/1234')"
            title       = "condition"
            description = "test condition"
            location    = "somewhere"
          }
          enforce = true
        },
        {
          enforce = false
        }
      ]
    }
    "iam.allowedPolicyMemberDomains" = {
      rules = [{
        allow = {
          values = ["C0xxxxxxx", "C0yyyyyyy"]
        }
      }]
    }
    "compute.trustedImageProjects" = {
      rules = [{
        allow = {
          values = ["projects/my-project"]
        }
      }]
    }
    "compute.vmExternalIpAccess" = {
      rules = [{ deny = { all = true } }]
    }
  }
  shared_vpc_service_config = {
    host_project       = module.host-project.project_id
    service_iam_grants = module.project.services
    service_identity_iam = {
      "roles/cloudasset.owner" = [
        "cloudservices", "container-engine"
      ]
    }
  }
  services = [
    "apigee.googleapis.com",
    "bigquery.googleapis.com",
    "container.googleapis.com",
    "logging.googleapis.com",
    "run.googleapis.com",
    "storage.googleapis.com",
  ]
  service_encryption_key_ids = {
    compute = [
      var.kms_key.id
    ]
    storage = [
      var.kms_key.id
    ]
  }
}

module "host-project" {
  source          = "./fabric/modules/project"
  billing_account = var.billing_account_id
  name            = "host"
  parent          = var.folder_id
  prefix          = var.prefix
  shared_vpc_host_config = {
    enabled = true
  }
}

module "gcs" {
  source        = "./fabric/modules/gcs"
  project_id    = var.project_id
  name          = "gcs_sink"
  location      = "EU"
  prefix        = var.prefix
  force_destroy = true
}

module "dataset" {
  source     = "./fabric/modules/bigquery-dataset"
  project_id = var.project_id
  id         = "bq_sink"
  options    = { delete_contents_on_destroy = true }
}

module "pubsub" {
  source     = "./fabric/modules/pubsub"
  project_id = var.project_id
  name       = "pubsub_sink"
}

module "bucket" {
  source      = "./fabric/modules/logging-bucket"
  parent_type = "project"
  parent      = var.project_id
  id          = "${var.prefix}-bucket"
}
# tftest modules=7 resources=53 inventory=data.yaml e2e

Files

name description resources
iam.tf IAM bindings. google_project_iam_binding · google_project_iam_custom_role · google_project_iam_member
logging.tf Log sinks and supporting resources. google_bigquery_dataset_iam_member · google_logging_project_exclusion · google_logging_project_sink · google_project_iam_audit_config · google_project_iam_member · google_pubsub_topic_iam_member · google_storage_bucket_iam_member
main.tf Module-level locals and resources. google_compute_project_metadata_item · google_essential_contacts_contact · google_monitoring_monitored_project · google_project · google_project_service · google_resource_manager_lien
organization-policies.tf Project-level organization policies. google_org_policy_policy
outputs.tf Module outputs.
quotas.tf None google_cloud_quotas_quota_preference
service-accounts.tf Service identities and supporting resources. google_kms_crypto_key_iam_member · google_project_default_service_accounts · google_project_iam_member · google_project_service_identity
shared-vpc.tf Shared VPC project-level configuration. google_compute_shared_vpc_host_project · google_compute_shared_vpc_service_project · google_compute_subnetwork_iam_member · google_project_iam_member
tags.tf None google_tags_tag_binding · google_tags_tag_key · google_tags_tag_key_iam_binding · google_tags_tag_key_iam_member · google_tags_tag_value · google_tags_tag_value_iam_binding · google_tags_tag_value_iam_member
variables-iam.tf None
variables-quotas.tf None
variables-tags.tf None
variables.tf Module variables.
versions.tf Version pins.
vpc-sc.tf VPC-SC project-level perimeter configuration. google_access_context_manager_service_perimeter_dry_run_resource · google_access_context_manager_service_perimeter_resource

Variables

name description type required default
name Project name and id suffix. string ✓
auto_create_network Whether to create the default network for the project. bool false
billing_account Billing account id. string null
compute_metadata Optional compute metadata key/values. Only usable if compute API has been enabled. map(string) {}
contacts List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. map(list(string)) {}
custom_roles Map of role name => list of permissions to create in this project. map(list(string)) {}
default_service_account Project default service account setting: can be one of delete, deprivilege, disable, or keep. string "keep"
descriptive_name Name of the project name. Used for project name instead of name variable. string null
factories_config Paths to data files and folders that enable factory functionality. object({…}) {}
iam Authoritative IAM bindings in {ROLE => [MEMBERS]} format. map(list(string)) {}
iam_bindings Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. map(object({…})) {}
iam_bindings_additive Individual additive IAM bindings. Keys are arbitrary. map(object({…})) {}
iam_by_principals Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the iam variable. map(list(string)) {}
labels Resource labels. map(string) {}
lien_reason If non-empty, creates a project lien with this description. string null
logging_data_access Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services. map(map(list(string))) {}
logging_exclusions Logging exclusions for this project in the form {NAME -> FILTER}. map(string) {}
logging_sinks Logging sinks to create for this project. map(object({…})) {}
metric_scopes List of projects that will act as metric scopes for this project. list(string) []
network_tags Network tags by key name. If id is provided, key creation is skipped. The iam attribute behaves like the similarly named one at module level. map(object({…})) {}
org_policies Organization policies applied to this project keyed by policy name. map(object({…})) {}
parent Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format. string null
prefix Optional prefix used to generate project id and name. string null
project_create Create project. When set to false, uses a data source to reference existing project. bool true
quotas Service quota configuration. map(object({…})) {}
service_config Configure service API activation. object({…}) {…}
service_encryption_key_ids Cloud KMS encryption key in {SERVICE => [KEY_URL]} format. map(list(string)) {}
services Service APIs to enable. list(string) []
shared_vpc_host_config Configures this project as a Shared VPC host project (mutually exclusive with shared_vpc_service_project). object({…}) null
shared_vpc_service_config Configures this project as a Shared VPC service project (mutually exclusive with shared_vpc_host_config). object({…}) {…}
skip_delete Allows the underlying resources to be destroyed without destroying the project itself. bool false
tag_bindings Tag bindings for this project, in key => tag value id format. map(string) null
tags Tags by key name. If id is provided, key or value creation is skipped. The iam attribute behaves like the similarly named one at module level. map(object({…})) {}
vpc_sc VPC-SC configuration for the project, use when ignore_changes for resources is set in the VPC-SC module. object({…}) null

Outputs

name description sensitive
custom_role_id Map of custom role IDs created in the project.
custom_roles Map of custom roles resources created in the project.
id Project id.
name Project name.
network_tag_keys Tag key resources.
network_tag_values Tag value resources.
number Project number.
project_id Project id.
quota_configs Quota configurations.
quotas Quota resources.
service_accounts Product robot service accounts in project.
services Service APIs to enabled in the project.
sink_writer_identities Writer identities created for each sink.
tag_keys Tag key resources.
tag_values Tag value resources.