Make sure that temporary credentials will be valid for the lifetime of the token

[rclark]

go-pgmultiauth/aws.go

Lines 33 to 41 in d14e8b7

	creds := c.awsConfig.Credentials
	region := *c.awsConfig.Region
	authToken, err := rdsutils.BuildAuthToken(
	fmt.Sprintf("%s:%d", c.host, c.port),
	region,
	c.user,
	creds,
	)

From AWS Docs

If you are using temporary credentials when creating an IAM database authentication token, the temporary credentials must still be valid when using the IAM database authentication token to make a connection request.

The aws.Config may wrap a cache around its credential providers. That means that if an application is using the default credential provider chain, and if it is perhaps using that aws.Config struct for other AWS clients in other parts of the application, then when the time comes for the database token to be refreshed, it is possible that the aws.Config.Credentials.Retrieve() method will return cached credentials.

These could be temporary credentials -- and usually will be, since applications usually leverage an IAM Role. So you can imagine a situation where:

1. A token needs to be refreshed
2. Inside the call to BuildAuthToken, there is a call to .Retrieve() credentials. This provides cached credentials that were previously provisioned.
3. We get a token that is presumed to be good for 15 minutes
4. 3 minutes later, those temporary credentials expire
5. 2 minutes after that, thinking that the token is still valid, another connect attempt tries to use it
6. That fails to connect. In fact all connection attempts now will fail until the 15 minute token timer expires.

---

I'm mentioning this because it looks like we are observing it happen in the Terraform Registry. Here is our implementation that appears to be falling victim to this.

Now, we're using github.com/aws/aws-sdk-go-v2 and here, it looks like you're using github.com/aws/aws-sdk-go. Sidebar: I would encourage you not to use the old version of the SDK, its going to be deprecated this July.

In v2 though, I can tell you how I think this should be solved. Basically, fetch credentials and check their expiration before you call BuildAuthToken. Then check the expiration time on those credentials and make sure that they won't expire before your 15 minute token timer would.

If they WOULD expire before then, you can invalidate the cache. That will mean the next time .Retrieve() gets called (inside the BuildAuthToken function), a new set of temporary credentials will be obtained. By default they expire in 1 hour. Here's how you'd do the invalidation:

type credentialsCache interface {
  Invalidate()
}

func (c awsTokenConfig) generateToken() (*authToken, error) {
...

  // If the .Credentials do not implement the Invalidate() method, then you're
  // not dealing with a provider that caches, and therefore you shouldn't have
  // to worry about it.
  if cache, ok := c.awsConfig.Credentials.(credentialsCache); ok {
    cache.Invalidate()
  }

  // ... now go on to BuildAuthToken using c.awsConfig.Credentials