Provider omits /projects/ prefix for GCP-hosted Commercetools endpoints, causing 404 errors

[sooyp]

Version information

• terraform: 1.2.2
• terraform provider: labd/commercetools v1.21.1

Describe the bug

When applying Commercetools resources via Terraform, the provider returns 404 Not Found responses for all API calls.
This occurs because the provider constructs incorrect request URLs - it automatically prepends /projects/{projectKey} to the configured api_url, even when the URL already includes the project key.

For example, Terraform attempts:

POST https://api.us-central1.gcp.commercetools.com/projects/ct-sync-task/customer-groups

This responds with:

404 Not Found

However, the same request made manually via curl to the correct endpoint works:

POST https://api.us-central1.gcp.commercetools.com/ct-sync-task/customer-groups

This indicates the provider is injecting an unnecessary /projects/ prefix.

To Reproduce

1. Provider configuration:

terraform {
  required_version = ">=1.2.2"

  required_providers {
    commercetools = {
      source  = "labd/commercetools"
      version = "~> 1.21.1"
    }
  }

  backend "s3" {
    bucket               = "app-tf-state-nonprod"
    workspace_key_prefix = "app"
    key                  = "terraform.tfstate"
    region               = "us-east-1"
    encrypt              = "true"
  }
}

provider "commercetools" {
  client_id     = var.commerce_tools_client_id
  client_secret = var.commerce_tools_client_secret
  project_key   = "ct-sync-task"
  api_url       = "https://api.us-central1.gcp.commercetools.com/ct-sync-task"
  token_url     = "https://auth.us-central1.gcp.commercetools.com/oauth/token"
}

2. Create any Commercetools resource:

resource "commercetools_customer_group" "example" {
  name = "Test Group"
}

3. Run

terraform apply

4. Observe Terraform debug output (incorrect path):

POST https://api.us-central1.gcp.commercetools.com/projects/ct-sync-task/customer-groups
Response: 404 Not Found

Expected behavior

Terraform should call:

https://api.us-central1.gcp.commercetools.com/mxp-store-rave-ct-sync-task7/customer-groups

without injecting the /projects/ prefix. This matches Commercetools’ documented API structure and succeeds when tested manually using curl.

Screenshots

Incorrect URL generated by Terraform:

POST https://api.us-central1.gcp.commercetools.com/projects/ct-sync-task/customer-groups
Response: 404 Not Found

Working manual curl request:

curl -X POST https://api.us-central1.gcp.commercetools.com/ct-sync-task/customer-groups
# Response: 201 Created

Additional context

• Authentication succeeds - OAuth tokens are issued correctly with valid scopes.
• Issue occurs for multiple resource types (customer_group, type, product_type, etc.).
• Removing /projects/ from the URL resolves the issue.
• It appears the provider still assumes an older API path format.

Suggested fix

It appears the provider currently assumes the /projects/{projectKey} path should always be included. Adjusting this logic so that the api_url represents the root API domain (e.g., https://api.us-central1.gcp.commercetools.com) and /projects/{projectKey} is appended dynamically could resolve the issue and ensure compatibility with regional Commercetools endpoints.

Happy to provide debug logs or test a patch version if needed.

[demeyerthom]

Hi @sooyp your configurations are incorrect:

The api url and token_url need to be:

  api_url       = "https://api.us-central1.gcp.commercetools.com"
  token_url     = "https://auth.us-central1.gcp.commercetools.com"

So only the domain. If I make these changes it works

[sooyp]

Hi @demeyerthom,

Thanks for clarifying the correct domain structure — we updated the configuration exactly as suggested:

terraform {
  required_version = ">=1.2.2"
  required_providers {
    commercetools = {
      source  = "labd/commercetools"
      version = "1.21.1"
    }
  }
}

provider "commercetools" {
  client_id     = "<redacted>"
  client_secret = "<redacted>"
  project_key   = "mxp-store-rave-ct-sync-task7"
  api_url       = "https://api.us-central1.gcp.commercetools.com"
  token_url     = "https://auth.us-central1.gcp.commercetools.com"
}

After re-initialising and retesting with v1.21.1, the authentication step now works correctly when tested manually, a direct POST to the auth endpoint returns a valid bearer token.

However, when running through the provider, every API call still fails with:

POST https://api.us-central1.gcp.commercetools.com/mxp-store-rave-ct-sync-task/customer-groups
oauth2: cannot fetch token: 404 Not Found

This confirms the credentials and endpoints are fine, but the provider is still failing before or during token handling.
From the debug output, it looks like requests are being sent to the API host instead of using the obtained auth token, or the internal request URL is being mis-constructed after token retrieval.

Could you review whether the provider might still be caching or re-using an invalid token host reference internally?
We’ve eliminated every external cause, and the 404s only occur when using the provider.

[demeyerthom]

Hi @sooyp

No, the provider does not cache or re-use a token over multiple runs, so that error is only happening in the scope of your reply. I see mxp-store-rave-ct-sync-task in the error message, but mxp-store-rave-ct-sync-task7 in your code snippet above. Was this error the result of running that snippet?

I cannot reproduce the above error