At least one account is missing the Workload Discovery IAM Role. Visit the Accounts page to see what accounts are affected and verify that the AWS CloudFormation StackSets stack instance in these accounts deployed correctly.

[jlong-inteprosfed]

If your issue relates to the Discovery Process, please first follow the steps described in the implementation guide Debugging the Discovery Component

---

Describe the bug
I've deployed Workflow Discovery a number of times, all with the same issue, that and IAM role is not deployed. During pre-reqs, I check for the AWSServiceRoleForAmazonOpenSearchService, and i do not have it. When deploying the template, it asks to CreateOpensearchServiceRole. I say yes to create it, however, the details state it will create AWSServiceRoleForAmazonElasticsearchService. That is created. I cannot import my account, because I can't get past the IAM alert. I've deployed the Global and Regional stacks, and all other config looks good.

To Reproduce
Deploy Workload Discovery, leave the IAM optional parameter blank

Expected behavior
A clear and concise description of what you expected to happen.
I would expect if I follow the instructions, and have all pre-req set, this would work
Please complete the following information about the solution:

• Version: [e.g. v2.0.0]
  version:v2.2.4

To get the version of the solution, you can look at the description of the created CloudFormation stack.
For example, "Workload Discovery on AWS Main Template (SO0075a) - Solution - Main Template (uksb-abcdef12) (version:v2.0.0)". If the description does not contain the version information,
you can look at the mappings section of the template:

Mappings:

  Solution:
    Constants:
      # ...
      SolutionVersion: v2.0.0
      # ...

• Region: [e.g. us-east-1] - US-East-1
• Was the solution modified from the version published on this repository? - No
• If the answer to the previous question was yes, are the changes available on GitHub?
• Have you checked your service quotas for the
  services this solution uses?
• Were there any errors in the CloudWatch Logs?

Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context
Add any other context about the problem here.

[svozza]

That IAM role in the message is not referring to the OpenSearch service role, we don't do any checks in the UI for that and the CloudFormation stacks would not deploy if there was an error creating it. The IAM role it is referring to is the role provisioned by the global template. Can you select the View account configuration button and post a screenshot of what the Account page looks like?

[jlong-inteprosfed]

[cid:4cce94dd-0a21-4952-9152-2f091a724a0a] Justin Long Director of Engineering - IntePros Federal 501 School Street, SW Suite 601 Washington, DC 20024 Email: ***@***.******@***.***> www.inteprosfed.com<http://www.inteprosfed.com/> [Image]

…

________________________________ From: Stefano Vozza ***@***.***> Sent: Wednesday, April 30, 2025 12:52 PM To: aws-solutions/workload-discovery-on-aws ***@***.***> Cc: Justin Long ***@***.***>; Author ***@***.***> Subject: Re: [aws-solutions/workload-discovery-on-aws] At least one account is missing the Workload Discovery IAM Role. Visit the Accounts page to see what accounts are affected and verify that the AWS CloudFormation StackSets stack instance in these accounts d... You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> [https://avatars.githubusercontent.com/u/8573472?s=20&v=4]svozza left a comment (aws-solutions/workload-discovery-on-aws#589)<#589 (comment)> That IAM role in the messgae is not referring to the OpenSearch service role, we don't do any checks in the UI for that and the CFN solution would not deploy if there was an error creating it.The IAM role it is referring to is the role provisioned by the global template. Can you select the View account configuration there and post a screenshot of what the Account page looks like? — Reply to this email directly, view it on GitHub<#589 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BFWGNDLI2TBVK46U5IAXTRD24D5VXAVCNFSM6AAAAAB4GA4HW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNBSGYZTENBXG4>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[svozza]

The image doesn't appear to have come through, you might have to add it through the GitHub WebUI.

[jlong-inteprosfed]

[svozza]

Ah okay, so it looks like there's something going wrong with the discovery process and it can't tell if the IAM role is deployed or not. Can you do me a favour and get the logs from the discovery prcoess that runs as an ECS task? It's probably one of the issues listed in the Troubleshooting section of the docs but I'll know for sure once I see the logs. Follow the steps in the discovery process debugging documentation (don't worry about the lambda instructions, we only care about ECS).

[jlong-inteprosfed]

There were no streams configured for /ecs/workload-discovery-task. I started a tail, created a test stream, and removed, then re-imported the account. No longs have been generated at this point

[svozza]

Interesting. Can you follow these steps to see why the tasks are failing to launch:

• Sign in to the Amazon Elastic Container Service console.
• Select the cluster named workload-discovery-cluster.
• Choose the Tasks tab.
• Select the Stopped button in the Desired task status panel.
• Click on one of the stopped tasks an see if there are any errors displayed

Also, has the solution been deployed to an existing VPC?

[jlong-inteprosfed]

ResourceInitializationError: unable to pull secrets or registry auth: The task cannot pull registry auth from Amazon ECR: There is a connection issue between the task and Amazon ECR. Check your task network configuration.
This is what I found

[jlong-inteprosfed]

RequestError: send request failed caused by: Post "https://api.ecr.us-east-1.amazonaws.com/": dial tcp :443: i/o timeout

[svozza]

I've only seen this error when the solution has been deployed to an existing VPC that does not allow requests to route to to ECR. Has the solution been deployed to existing VPC?

[jlong-inteprosfed]

It has been Justin Long Director of Engineering - IntePros Federal 501 School Street, SW Suite 601 Washington, DC 20024 Email: ***@***.******@***.***> www.inteprosfed.com<http://www.inteprosfed.com/> [Image]

…

________________________________ From: Stefano Vozza ***@***.***> Sent: Wednesday, April 30, 2025 2:09 PM To: aws-solutions/workload-discovery-on-aws ***@***.***> Cc: Justin Long ***@***.***>; Author ***@***.***> Subject: Re: [aws-solutions/workload-discovery-on-aws] At least one account is missing the Workload Discovery IAM Role. Visit the Accounts page to see what accounts are affected and verify that the AWS CloudFormation StackSets stack instance in these accounts d... You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> [https://avatars.githubusercontent.com/u/8573472?s=20&v=4]svozza left a comment (aws-solutions/workload-discovery-on-aws#589)<#589 (comment)> I've only seen this error when the solution has been deployed to an existing VPC that does not allow requests to route to to ECR. Has the solution been deployed to existing VPC? — Reply to this email directly, view it on GitHub<#589 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BFWGNDMNASSUAXRCVAU5T6324EGUBAVCNFSM6AAAAAB4GA4HW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNBSHA3TOMZXHA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[svozza]

For the solution to work correctly the discovery process needs to be able to route to all these APIs as documented here: https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/aws-apis.html.

I'm assuming you don't have a NAT Gateway in this VPC? If so you will have to add a VPC endpoint for both ECR API and the ECR Docker endpoint. You can see how we do it in CloudFormation when the solution deploys its own VPC:

https://github.com/aws-solutions/workload-discovery-on-aws/blob/main/source/cfn/templates/vpc.template#L395

[jlong-inteprosfed]

There are 2 NAT Gateways. They are added to 2 different route tables, and the two private subnets are assigned to each route table. They are private NAT Gateways, do I need public NAT Gateways?

[svozza]

You need public NAT Gateways if you don't want to use VPC endpoints for all the services in the document I linked too. All Amazon SDK calls go to the internet so they either need to route through an internet gateway at some point or go through a VPC endpoint.