Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: setup Cost and Usuage Report module #317

Merged
merged 2 commits into from
Nov 6, 2024
Merged

feat: setup Cost and Usuage Report module #317

merged 2 commits into from
Nov 6, 2024

Conversation

patheard
Copy link
Member

@patheard patheard commented Nov 6, 2024

Summary

Rename the billing_extract_tags module to repurpose it so that it can hold all the Cost and Usage Report resources.

Related

@patheard
feat: setup Cost and Usuage Report module
17ba119 
Rename the billing extract tags module to repurpose it so that it can
hold all the Cost and Usage Report resources.
@patheard patheard requested a review from a team November 6, 2024 20:14
@patheard patheard self-assigned this Nov 6, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 6, 2024

Plan for org_account/cost_usage_report

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

⚠️   Warning: resources will be destroyed by this change!

Plan: 10 to import, 7 to add, 2 to change, 1 to destroy
Show summary
CHANGE NAME
add aws_s3_bucket_policy.billing_extract_tags
aws_s3_bucket_policy.cost_usage_report
module.billing_extract_tags.aws_s3_bucket.this
module.billing_extract_tags.aws_s3_bucket_public_access_block.this
module.cost_usage_report.aws_s3_bucket.this
module.cost_usage_report.aws_s3_bucket_public_access_block.this
update aws_iam_policy.billing_extract_tags
aws_lambda_function.billing_extract_tags
recreate aws_lambda_permission.billing_extract_tags
Show plan 
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # data.aws_iam_policy_document.billing_extract_tags will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "billing_extract_tags" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "logs:CreateLogGroup",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:logs:ca-central-1:659087519042:*",
            ]
        }
      + statement {
          + actions   = [
              + "logs:CreateLogStream",
              + "logs:PutLogEvents",
            ]
          + effect    = "Allow"
          + resources = [
              + "arn:aws:logs:ca-central-1:659087519042:log-group:/aws/lambda/billing_extract_tags:*",
            ]
        }
      + statement {
          + actions   = [
              + "s3:DeleteObject*",
              + "s3:GetBucketLocation",
              + "s3:GetObject*",
              + "s3:ListBucket",
              + "s3:PutObject*",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]
        }
    }

  # data.aws_iam_policy_document.billing_extract_tags_bucket will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "billing_extract_tags_bucket" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:DeleteObject*",
              + "s3:GetBucketLocation",
              + "s3:GetObject*",
              + "s3:ListBucket",
              + "s3:PutObject*",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "arn:aws:iam::659087519042:role/BillingExtractTags",
                ]
              + type        = "AWS"
            }
        }
      + statement {
          + actions   = [
              + "s3:AbortMultipartUpload",
              + "s3:GetBucketLocation",
              + "s3:GetObject",
              + "s3:ListBucket",
              + "s3:ListBucketMultipartUploads",
              + "s3:ListMultipartUploadParts",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "arn:aws:iam::066023111852:root",
                ]
              + type        = "AWS"
            }
        }
    }

  # data.aws_iam_policy_document.cost_usage_report will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "cost_usage_report" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "s3:GetBucketPolicy",
              + "s3:PutObject",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]
          + sid       = "EnableAWSDataExportsToWriteToS3AndCheckPolicy"

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "659087519042",
                ]
              + variable = "aws:SourceAccount"
            }
          + condition {
              + test     = "StringLike"
              + values   = [
                  + "arn:aws:cur:us-east-1:659087519042:definition/*",
                  + "arn:aws:bcm-data-exports:us-east-1:659087519042:export/*",
                ]
              + variable = "aws:SourceArn"
            }

          + principals {
              + identifiers = [
                  + "bcm-data-exports.amazonaws.com",
                  + "billingreports.amazonaws.com",
                ]
              + type        = "Service"
            }
        }
      + statement {
          + actions   = [
              + "s3:AbortMultipartUpload",
              + "s3:GetBucketLocation",
              + "s3:GetObject",
              + "s3:ListBucket",
              + "s3:ListBucketMultipartUploads",
              + "s3:ListMultipartUploadParts",
            ]
          + effect    = "Allow"
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + principals {
              + identifiers = [
                  + "arn:aws:iam::066023111852:root",
                ]
              + type        = "AWS"
            }
        }
    }

  # aws_cloudwatch_event_rule.billing_extract_tags will be imported
    resource "aws_cloudwatch_event_rule" "billing_extract_tags" {
        arn                 = "arn:aws:events:ca-central-1:659087519042:rule/billing_extract_tags_daily"
        event_bus_name      = "default"
        id                  = "default/billing_extract_tags_daily"
        is_enabled          = true
        name                = "billing_extract_tags_daily"
        schedule_expression = "cron(0 5 * * ? *)"
        tags                = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        tags_all            = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
    }

  # aws_cloudwatch_event_target.billing_extract_tags will be imported
    resource "aws_cloudwatch_event_target" "billing_extract_tags" {
        arn            = "arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags"
        event_bus_name = "default"
        id             = "billing_extract_tags_daily-terraform-20240305180415483500000002"
        rule           = "billing_extract_tags_daily"
        target_id      = "terraform-20240305180415483500000002"
    }

  # aws_cloudwatch_log_group.billing_extract_tags will be imported
    resource "aws_cloudwatch_log_group" "billing_extract_tags" {
        arn               = "arn:aws:logs:ca-central-1:659087519042:log-group:/aws/lambda/billing_extract_tags"
        id                = "/aws/lambda/billing_extract_tags"
        name              = "/aws/lambda/billing_extract_tags"
        retention_in_days = 14
        tags              = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        tags_all          = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
    }

  # aws_iam_policy.billing_extract_tags will be updated in-place
  # (imported from "arn:aws:iam::659087519042:policy/BillingExtractTags")
  ~ resource "aws_iam_policy" "billing_extract_tags" {
        arn       = "arn:aws:iam::659087519042:policy/BillingExtractTags"
        id        = "arn:aws:iam::659087519042:policy/BillingExtractTags"
        name      = "BillingExtractTags"
        path      = "/"
      ~ policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = "logs:CreateLogGroup"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:logs:ca-central-1:659087519042:*"
                    },
                  - {
                      - Action   = [
                          - "logs:PutLogEvents",
                          - "logs:CreateLogStream",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:logs:ca-central-1:659087519042:log-group:/aws/lambda/billing_extract_tags:*"
                    },
                  - {
                      - Action   = [
                          - "s3:PutObject*",
                          - "s3:ListBucket",
                          - "s3:GetObject*",
                          - "s3:GetBucketLocation",
                          - "s3:DeleteObject*",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:s3:::5bf89a78-1503-4e02-9621-3ac658f558fb/*",
                          - "arn:aws:s3:::5bf89a78-1503-4e02-9621-3ac658f558fb",
                        ]
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        policy_id = "ANPAZS5FNNVBMXEK54I7I"
        tags      = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        tags_all  = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
    }

  # aws_iam_role.billing_extract_tags will be imported
    resource "aws_iam_role" "billing_extract_tags" {
        arn                   = "arn:aws:iam::659087519042:role/BillingExtractTags"
        assume_role_policy    = jsonencode(
            {
                Statement = [
                    {
                        Action    = "sts:AssumeRole"
                        Effect    = "Allow"
                        Principal = {
                            Service = "lambda.amazonaws.com"
                        }
                    },
                ]
                Version   = "2012-10-17"
            }
        )
        create_date           = "2024-03-05T16:05:11Z"
        force_detach_policies = false
        id                    = "BillingExtractTags"
        managed_policy_arns   = [
            "arn:aws:iam::659087519042:policy/BillingExtractTags",
            "arn:aws:iam::aws:policy/AWSOrganizationsReadOnlyAccess",
            "arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy",
        ]
        max_session_duration  = 3600
        name                  = "BillingExtractTags"
        path                  = "/"
        tags                  = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        tags_all              = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        unique_id             = "AROAZS5FNNVBFKDLK3FY2"
    }

  # aws_iam_role_policy_attachment.billing_extract_tags will be imported
    resource "aws_iam_role_policy_attachment" "billing_extract_tags" {
        id         = "BillingExtractTags-arn:aws:iam::659087519042:policy/BillingExtractTags"
        policy_arn = "arn:aws:iam::659087519042:policy/BillingExtractTags"
        role       = "BillingExtractTags"
    }

  # aws_iam_role_policy_attachment.lambda_insights will be imported
    resource "aws_iam_role_policy_attachment" "lambda_insights" {
        id         = "BillingExtractTags-arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy"
        policy_arn = "arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy"
        role       = "BillingExtractTags"
    }

  # aws_iam_role_policy_attachment.org_read_only will be imported
    resource "aws_iam_role_policy_attachment" "org_read_only" {
        id         = "BillingExtractTags-arn:aws:iam::aws:policy/AWSOrganizationsReadOnlyAccess"
        policy_arn = "arn:aws:iam::aws:policy/AWSOrganizationsReadOnlyAccess"
        role       = "BillingExtractTags"
    }

  # aws_lambda_function.billing_extract_tags will be updated in-place
  # (imported from "billing_extract_tags")
  ~ resource "aws_lambda_function" "billing_extract_tags" {
        architectures                  = [
            "x86_64",
        ]
        arn                            = "arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags"
      + filename                       = "/tmp/main.py.zip"
        function_name                  = "billing_extract_tags"
        handler                        = "main.handler"
        id                             = "billing_extract_tags"
        invoke_arn                     = "arn:aws:apigateway:ca-central-1:lambda:path/2015-03-31/functions/arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags/invocations"
      ~ last_modified                  = "2024-03-05T19:17:30.000+0000" -> (known after apply)
        layers                         = []
        memory_size                    = 1024
        package_type                   = "Zip"
      + publish                        = false
        qualified_arn                  = "arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags:$LATEST"
        qualified_invoke_arn           = "arn:aws:apigateway:ca-central-1:lambda:path/2015-03-31/functions/arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags:$LATEST/invocations"
        reserved_concurrent_executions = -1
        role                           = "arn:aws:iam::659087519042:role/BillingExtractTags"
        runtime                        = "python3.11"
        skip_destroy                   = false
        source_code_hash               = "tuYDTu43qo+6Z5hsmZctAh5OeQ1XygVO+qRzDe65sIk="
        source_code_size               = 1057
        tags                           = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        tags_all                       = {
            "CostCentre" = "cds-aws-lz-production"
            "Terraform"  = "true"
        }
        timeout                        = 30
        version                        = "$LATEST"

      ~ environment {
          ~ variables = {
              - "TARGET_BUCKET" = "5bf89a78-1503-4e02-9621-3ac658f558fb"
            } -> (known after apply)
        }

        ephemeral_storage {
            size = 512
        }

        tracing_config {
            mode = "PassThrough"
        }
    }

  # aws_lambda_permission.billing_extract_tags must be replaced
  # (imported from "billing_extract_tags/AllowBillingExtractTagsDaily")
  # Warning: this will destroy the imported resource
-/+ resource "aws_lambda_permission" "billing_extract_tags" {
        action              = "lambda:InvokeFunction"
      ~ function_name       = "arn:aws:lambda:ca-central-1:659087519042:function:billing_extract_tags" -> "billing_extract_tags" # forces replacement
      ~ id                  = "AllowBillingExtractTagsDaily" -> (known after apply)
        principal           = "events.amazonaws.com"
        source_arn          = "arn:aws:events:ca-central-1:659087519042:rule/billing_extract_tags_daily"
        statement_id        = "AllowBillingExtractTagsDaily"
      + statement_id_prefix = (known after apply)
    }

  # aws_s3_bucket_policy.billing_extract_tags will be created
  + resource "aws_s3_bucket_policy" "billing_extract_tags" {
      + bucket = (known after apply)
      + id     = (known after apply)
      + policy = (known after apply)
    }

  # aws_s3_bucket_policy.cost_usage_report will be created
  + resource "aws_s3_bucket_policy" "cost_usage_report" {
      + bucket = (known after apply)
      + id     = (known after apply)
      + policy = (known after apply)
    }

  # module.billing_extract_tags.aws_s3_bucket.this will be created
  + resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "cds-account-billing-extract-tags"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags                        = {
          + "CostCentre" = "cds-aws-lz-production"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + tags_all                    = {
          + "CostCentre" = "cds-aws-lz-production"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = false

              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

      + versioning {
          + enabled    = true
          + mfa_delete = false
        }
    }

  # module.billing_extract_tags.aws_s3_bucket_public_access_block.this will be created
  + resource "aws_s3_bucket_public_access_block" "this" {
      + block_public_acls       = true
      + block_public_policy     = true
      + bucket                  = (known after apply)
      + id                      = (known after apply)
      + ignore_public_acls      = true
      + restrict_public_buckets = true
    }

  # module.cost_usage_report.aws_s3_bucket.this will be created
  + resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "cds-cost-usage-report"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags                        = {
          + "CostCentre" = "cds-aws-lz-production"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + tags_all                    = {
          + "CostCentre" = "cds-aws-lz-production"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = false

              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

      + versioning {
          + enabled    = true
          + mfa_delete = false
        }
    }

  # module.cost_usage_report.aws_s3_bucket_public_access_block.this will be created
  + resource "aws_s3_bucket_public_access_block" "this" {
      + block_public_acls       = true
      + block_public_policy     = true
      + bucket                  = (known after apply)
      + id                      = (known after apply)
      + ignore_public_acls      = true
      + restrict_public_buckets = true
    }

Plan: 10 to import, 7 to add, 2 to change, 1 to destroy.

Warning: Argument is deprecated

  with module.billing_extract_tags.aws_s3_bucket.this,
  on .terraform/modules/billing_extract_tags/S3/main.tf line 8, in resource "aws_s3_bucket" "this":
   8: resource "aws_s3_bucket" "this" {

Use the aws_s3_bucket_server_side_encryption_configuration resource instead

(and 4 more similar warnings elsewhere)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results 
20 tests, 20 passed, 0 warnings, 0 failures, 0 exceptions

@patheard patheard merged commit 8effb74 into main Nov 6, 2024
16 checks passed
@patheard patheard deleted the feat/cur branch November 6, 2024 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sylviamclaughlin sylviamclaughlin sylviamclaughlin approved these changes

@gcharest gcharest gcharest approved these changes

Assignees

@patheard patheard

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@patheard @gcharest @sylviamclaughlin