Configuration question/feedback

[FrankHassanabad]

I might be missing something, but when using the aws-sdk-kotlin, I really want to utilize two things out of the box:

• Use ENV variable configuration without adding code touch points to read ENV variables
• 100% AWS profile support to where I can through a single AWS_PROFILE env variable, configure everything through my local .aws/config unless I decide to pass down another AWS Env/Java System variable to "override a profile setting"

This will make things easier for when people are using multiple profiles such as localstack profile for switching between them as developers. As a dev, locally I want to just change my AWS_PROFILE env variable and have it take effect in my environment rather than multiple ones.

To give an example of this operating as I expect it to... When I use the JavaScript JDK from: https://github.com/aws/aws-sdk-js-v3

I am able to do configuration with no code touch points like so:

// Notice that I do not have to pass a region or endpoint url or anything in here. I decide using ENV variable to set just `AWS_PROFILE` and can switch all profile settings through my `~/.aws/config` without other env variable settings
export const s3Client = new S3Client({});

☝️ , then from the outside of my container/process I can pass in just the AWS_PROFILE env variable and the AWS-SDK-JS will use my profile information and configure it from there. However, when using the aws-sdk-kotlin I am finding:

• I always have to pass at least 1 argument of region
• It does not fully support all the profile information/use the correct endpoint resolver code meaning I have to also pass down the endpointUrl if I want to use localstack

So, maybe I'm missing something, but this is what I have to add as a code touch point right now:

    return SecretsManagerClient {
        // NOTE: Really wish it would accept these as ENV variables, but it does not. Since ENV variables can only load
        // a system properties we have to do these tricks for devs load a custom .env file
        region = System.getenv("AWS_REGION") ?: System.getProperty("aws.region")
       // I have to pass this down even though I have it in my localstack profile as it does not resolve this as expected
        endpointUrl =
            System.getenv("AWS_ENDPOINT_URL")
                ?.let { Url.parse(it) }
                ?: System.getProperty("aws.endpointUrl")?.let { Url.parse(it) }
    }

And I have to keep these 3 env variables configured:

• AWS_REGION
• AWS_PROFILE
• AWS_ENDPOINT_URL

at all times so that I can switch between using localstack profile as well as my regular SSO sign on profile that I have. I would prefer:

• Keep only the AWS_PROFILE env configuration variable and all my profile information lives in my ~/.aws/config
• I don't have to ever add any code to read standard env variables when using the client API. It always reads from them as it's fall back defaults.

Here is my localstack profile (for reference). The JS SDK isn't perfect in that I have to add a few extra endpoint url's in the profile sections, but it does work with them once I get them correct without having to add code to read them.

[profile localstack]
region=us-east-2
output=json
endpoint_url=http://localhost:4566
services=localstack-s3

[services localstack-s3]
s3 =
  endpoint_url=http://s3.localhost.localstack.cloud:4566
secrets_manager =
  endpoint_url=http://secretsmanager.localhost.localstack.cloud:4566

Ref: https://docs.localstack.cloud/user-guide/integrations/aws-cli/#configuring-a-custom-profile

[aajtodd]

You can use fromEnvironment() to infer configuration from the environment including environment variables, JVM system properties, and shared config (~/.aws/config).

Check out the developer guide section on environment based config and let us know if you still have questions.

[FrankHassanabad]

Appreciate the fast response. Yes, the environment based config did work!

Just my opinion but the default docs and setup should be just fromEnvironment(), it is so much more friendlier and encourages people to not hard code regions, etc...

Thanks for all the hard work on this project.

[ianbotsf]

@FrankHassanabad we're always looking for ways to improve our documentation. Which docs/setup/examples did you find that could be improved by pointing readers to fromEnvironment()?

[FrankHassanabad]

Online here:
https://docs.aws.amazon.com/sdk-for-kotlin/latest/developer-guide/kotlin_secrets-manager_code_examples.html

And then when it points to the github examples:
https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/kotlin/services/secretsmanager/src/main/kotlin/com/kotlin/secrets/CreateSecret.kt#L50-L53

I typically follow the outlined examples (and will get code review questions if I'm deviating from online examples which makes sense), but was wanting to configure everything from either the profile or from env variables pushed down into our containers directly from our integration with AWS as that seems like the "best practices" and most "flexible" and "easiest" way to configure it.

So, instead of my code using the above linked examples, I just use the suggested:

SecretsManagerClient.fromEnvironment()

which seems more inline with the "best practices" way? Looking through the other Kotlin examples, they all seem to be instead showing a manual way of configuration, which makes me "feel" like I'm doing something wrong, but the more I become involved with AWS services and development, the more I want the fromEnvironment() way as the better way of configuring things.

Across different services and programming language integration points with AWS with different engineers, we're having to stay on top of ensuring they don't hard code a value such as a region or use different env variables that aren't standard AWS variables. Usually they're going down the wrong decision tree like I did when starting with the examples.

And so far it feels like the best way to configure AWS services is through the environment and locally using profiles as a developer to manage different test instances easier and test/staging/prod rollouts easier.

[ianbotsf]

Thanks for the additional detail @FrankHassanabad. I agree with you that the published samples should demonstrate best practices to reduce friction in the user experience. We're going to take a deep dive through our examples catalog and make some improvements in line with best practices.

[ianbotsf]

For tracking: #1268