Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terragrunt run-all plan failing with exec.ExitError exit status 126. #414

Open
nishilampally opened this issue Nov 10, 2023 · 1 comment

Comments

@nishilampally
Copy link

Recently we have implemented tfenv and tgenv to manage multiple versions of terraform and terragrunt . When I try to run terragrunt run-all plan I'm getting a 126 error exit code. I've enabled debug logging and it's not very helpful. The issue is intermittent where run--all plan get succeeds sometimes and fails intermittently. User(AzDevOps) has execute permissions on both terraform and terragrunt. I am not sure why I am getting this error. There is not much information available from the logs.

Setup:
Ubuntu machine 20_04-lts

Command:
terragrunt run-all plan --terragrunt-source-update -no-color --terragrunt-include-external-dependencies

Permissions:

drwxr-xr-x 3 root root 4096 Sep 20 16:48 aws-cli
drwxr-xr-x 3 root root 4096 Sep 20 16:48 bin
drwxr-xr-x 2 root root 4096 Aug 31 21:39 etc
drwxr-xr-x 2 root root 4096 Aug 31 21:39 games
drwxr-xr-x 2 root root 4096 Aug 31 21:39 include
drwxr-xr-x 3 root root 4096 Aug 31 21:39 lib
lrwxrwxrwx 1 root root 9 Aug 31 21:39 man -> share/man
drwxr-xr-x 2 root root 4096 Aug 31 21:39 sbin
drwxr-xr-x 5 root root 4096 Sep 20 16:44 share
drwxr-xr-x 2 root root 4096 Aug 31 21:39 src
drwxrwxrwx 10 root root 4096 Sep 20 16:46 tfenv
drwxrwxrwx 9 root root 4096 Sep 20 16:46 tgenv
/agent/_work/1/s
total 60

total 44
-rwxrwxrwx 1 root root 304 Sep 20 16:46 CHANGELOG.md
-rwxrwxrwx 1 root root 1086 Sep 20 16:46 LICENSE
-rwxrwxrwx 1 root root 5440 Sep 20 16:46 README.md
drwxrwxrwx 2 root root 4096 Sep 20 16:46 assets
drwxrwxrwx 2 root root 4096 Sep 20 16:46 bin
drwxrwxrwx 2 root root 4096 Sep 20 16:46 libexec
-rwxrwxrwx 1 root root 1165 Sep 20 16:46 list_all_versions_offline
drwxrwxrwx 2 root root 4096 Sep 20 16:46 test
-rwxrwxrwx 1 root root 7 Sep 20 16:46 version
drwxrwxrwx 4 root root 4096 Oct 19 19:41 versions
/agent/_work/1/s
total 8
drwxr-xr-x 2 AzDevOps AzDevOps 4096 Oct 19 19:42 1.0.1
drwxrwxrwx 2 root root 4096 Sep 20 16:46 1.5.5
total 8
drwxr-xr-x 2 AzDevOps AzDevOps 4096 Oct 19 19:41 0.37.1
drwxrwxrwx 2 root root 4096 Sep 20 16:46 0.49.1
/agent/_work/1/s
total 78812
-rwxr-xr-x 1 AzDevOps AzDevOps 80702103 Jun 24 2021 terraform
total 35700
-rwxr-xr-x 1 AzDevOps AzDevOps 36554016 Oct 19 19:41 terragrunt
/agent/_work/1/s
-rwxr-xr-x 1 AzDevOps AzDevOps 80702103 Jun 24 2021 /usr/local/tfenv/versions/1.0.1/terraform
-rwxr-xr-x 1 AzDevOps AzDevOps 36554016 Oct 19 19:41 /usr/local/tgenv/versions/0.37.1/terragrunt

Error:

time=2023-11-06T03:03:51Z level=error msg=169 errors occurred:
* Cannot process module Module /agent/_work/1/s/prd/apps/omnicc/key_vault_ado_access_policies/awslandingzone_ado (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/omnicc/key_vault_ado]) because one of its dependencies, Module /agent/_work/1/s/prd/apps/omnicc/key_vault_ado (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/omnicc/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/apps/omnicc/key_vault_ado (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/omnicc/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/prd/apps/omnicc/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/appdeploy_pool/key_vault_access_policies/disk_encryption_set (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/appdeploy_pool/key_vault, /agent/_work/1/s/prd/services/azure_devops/agent_pool/appdeploy_pool/disk_encryption_set]) because one of its dependencies, Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/appdeploy_pool/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/appdeploy_pool/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]), finished with an error: Cannot process module Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/resource_group_loganalytics (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* exit status 126
* exit status 126
* Cannot process module Module /agent/_work/1/s/shared_cac/services/private_dns_zones/privatelink_postgres_database_azure_com (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/services/private_dns_zones/resource_group]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/services/private_dns_zones/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/onprem_pool/key_vault_keys/disk_encryption_set (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/onprem_pool/key_vault]) because one of its dependencies, Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/onprem_pool/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/onprem_pool/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]), finished with an error: Cannot process module Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/resource_group_loganalytics (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/apps/dea/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/dea/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/prd/apps/dea/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/apps/omnicc/agent_pool/vmss_iac (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/omnicc/resource_group, /agent/_work/1/s/prd/network/subnet_adoagents, /agent/_work/1/s/prd/apps/omnicc/agent_pool/disk_encryption_set]) because one of its dependencies, Module /agent/_work/1/s/prd/network/subnet_adoagents (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/network/vnet]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/network/subnet_adoagents (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/network/vnet]) because one of its dependencies, Module /agent/_work/1/s/prd/network/vnet (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/network/resource_group_vnet]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/network/vnet (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/network/resource_group_vnet]) because one of its dependencies, Module /agent/_work/1/s/prd/network/resource_group_vnet (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/disk_encryption_set (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault_keys/disk_encryption_set]) because one of its dependencies, Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault_keys/disk_encryption_set (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault_keys/disk_encryption_set (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault]) because one of its dependencies, Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/azure_devops/agent_pool/vmss_mfbuild/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/azure_devops/agent_pool/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]), finished with an error: Cannot process module Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/resource_group_loganalytics (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group, /agent/_work/1/s/prd/services/virtual_desktop/biz_users/app_group]) because one of its dependencies, Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/aks (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/services/container_registry_forestersfinancial, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace, /agent/_work/1/s/shared_cac/services/private_dns_zones/privatelink_canadacentral_azmk8s_io, /agent/_work/1/s/prd/network/subnet_kubernetes, /agent/_work/1/s/prd/network/route_table_kubernetes, /agent/_work/1/s/prd/network/resource_group_vnet]) because one of its dependencies, Module /agent/_work/1/s/prd/network/resource_group_vnet (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/apps/omnicc/key_vault_ado (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/apps/omnicc/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/prd/apps/omnicc/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/storage_share_profiles (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/storage_account_profiles]) because one of its dependencies, Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/storage_account_profiles (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group, /agent/_work/1/s/prd/services/virtual_desktop/biz_users/key_vault]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/storage_account_profiles (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group, /agent/_work/1/s/prd/services/virtual_desktop/biz_users/key_vault]) because one of its dependencies, Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]), finished with an error: Cannot process module Module /agent/_work/1/s/prd/services/virtual_desktop/biz_users/key_vault (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/prd/services/virtual_desktop/biz_users/resource_group, /agent/_work/1/s/shared_cac/operations/loganalytics_workspace]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]), finished with an error: Cannot process module Module /agent/_work/1/s/shared_cac/operations/loganalytics_workspace (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/operations/resource_group_loganalytics]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/operations/resource_group_loganalytics (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/shared_cac/services/private_dns_zones/privatelink_canadacentral_azmk8s_io (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/services/private_dns_zones/resource_group]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/services/private_dns_zones/resource_group (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126
* Cannot process module Module /agent/_work/1/s/shared_cac/network/subnet_azurefw (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/network/vnet]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/network/vnet (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/network/resource_group_vnet]), finished with an error: Cannot process module Module /agent/_work/1/s/shared_cac/network/vnet (excluded: false, assume applied: false, dependencies: [/agent/_work/1/s/shared_cac/network/resource_group_vnet]) because one of its dependencies, Module /agent/_work/1/s/shared_cac/network/resource_group_vnet (excluded: false, assume applied: false, dependencies: []), finished with an error: exit status 126

@iokiwi
Copy link

iokiwi commented Dec 30, 2023

Note that a similar issue has been raised for tgenv/tgenv here tgenv/tgenv#27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants