Does KEDA support IAM based authentication for AWS postgres

[smartaquarius10]

We have hosted KEDA on EKS, and the triggers will be managed through a PostgreSQL database on AWS.

Instead of using a username and password, is it possible to authenticate using an IAM role ARN (pod identity)? Does the PostgreSQL scaler support this?

Thanks.

[JorTurFer]

Hello
There is a PR on going adding support -> #7097

[smartaquarius10]

@JorTurFer Thanks. It seems approval will take some time right as I there are few review comments.

So for now going with secret is the only option then.

I have one more query. Are we planning to add Postgres connection reusability as well rather than opening a new connection with every scaled object. Do we have that in roadmap

[JorTurFer]

That's an interesting improvement that we discussed some time ago -> #6360

I think that this could be done at some point too without that huge refactor as improvement

[smartaquarius10]

@JorTurFer Hmm, correct — that scaler service is needed to handle the connection pool. For example, we have more than 1,000 deployments, and each ScaledObject opens a new connection to the database, which doesn’t seem scalable.

Our use case is simple: wanted to scale down all deployments in non-production environments to zero replicas. For this, I’m planning to use a small Postgres instance, but it comes with the limitation of supporting only a limited number of connections.

For now, we’ll use AWS RDS Proxy, as it can handle polling with its queue mechanism. Hopefully, KEDA will include this functionality soon — will have to wait until then.

Meanwhile, will subscribe to this PR #7097 as well. Once done will switch to IAM. For now, using secrets seems to be the only option

[JorTurFer]

Our use case is simple: wanted to scale down all deployments in non-production environments to zero replicas. For this, I’m planning to use a small Postgres instance, but it comes with the limitation of supporting only a limited number of connections.

For now, we’ll use AWS RDS Proxy, as it can handle polling with its queue mechanism. Hopefully, KEDA will include this functionality soon — will have to wait until then.

Something that you could do as workaround in the meantime is building a simple API on top of your database and use metrics api scaler (to scale based on a rest api) or external metrics (to use gRPC) to query your API directly instead of the database.

[smartaquarius10]

@JorTurFer Yup, we actually thought about that. But two things came up:
1. Trust me, we really want to avoid reinventing the wheel.
2. If we build anything custom, maintaining vulnerabilities becomes a real headache. The moment an issue arises, we start getting alerts. There are already so many things running on the platform—adding another one would be quite a challenge.

That’s why we keep looking for existing solutions. RDS Proxy might offer some relief. Otherwise, the last option would be to go with AWS DynamoDB, but we’d prefer not to since we want to stay cloud-agnostic.

[JorTurFer]

Would you be willing to work on the connection pool for postgresql scaler? (not in the auth part that it's ongoing but in the pool)
We can ship it as part of January release -> #7171

If not but it's still needed, we can open another issue to handle the pooling part in specific and keep it to the community

[smartaquarius10]

@JorTurFer Thank you so much — I’d really love to get involved! The only challenge is that I’m completely new to open-source contributions, so I’m not yet familiar with the dos and don’ts.

For example, regarding this issue — where should I start? Should I go through the entire KEDA codebase first, or focus on something specific? Would really like investing time and effort into open-source work. If there’s no strict time constraint, I’d be happy to jump in and learn as I go, since this would be my first contribution.

I’ve actually raised this issue earlier, on August 1.

Would really appreciate if you could share a few pointers on where to begin, that would be incredibly helpful. Meanwhile, we can open issue to the community—if someone else completes it up before I do, at least we’ll make progress toward the January release deadline.

[smartaquarius10]

@JorTurFer hope you are doing great. Sorry for disturbing. In free time, I was analyzing this to learn as a fresh start. Just a small question what's the meaning of this statement. What is the meaning of pool part here.

(not in the auth part that it's ongoing but in the pool)

[JorTurFer]

Hey!
Sorry because I thought that I answered but obviously I didn't 🤦

There are 2 topics:

• IAM auth
• Reusing connections

For the IAM auth, there is already a PR on-going (the one I sent) so I'd just wait for it before implementing it again.

For reusing connections (which is a nice feature by itself), I suggest to create a connection pool that it's reused for postgres trigger where you store the connection by target
Let's say that you have something like this where you keep X connections active per target and you create a collection of targets. You could have X connections in the pool for Server A, another X for server B, etc.
In the trigger, instead of building the connection during trigger creation, you can request the connection for the target to the pool and rely on it to manage connections. Let's say that you have 100 psql triggers pointing to the same server, using this approach you can define the size of the pool (e.g: 10 connections) and just share them across all the triggers.

About how to start, I'd create a pkg inside scalers folder for connection pool (because this can be extended for other connections in the future) and build the psql pool struct there. Then I'd build a struct on top of psql pool to manage the collection of pools that KEDA will manage (a psql pool per target) and the code to add and get connection from the pool.

[smartaquarius10]

@JorTurFer Sorry for the delay in getting back. Was working on this task. I’ve completed the code changes today and tested with 7 scaled objects in EKS together, but only 2 connections were created with postgres in AWS.

The changes I have covered are:

Used pgxpool

• A new connection pool is created for a unique target here its DB.
• Pool initialized successfully with maxConns (either the default pgxpool or value read from ConfigMap).
• Another ScaledObject pointed to the same DB; reuses the same pool and increments the usage counter
• If ScaledObject deleted; it releases its reference, decreasing the usage counter.
• If the last ScaledObject referencing that pool is deleted and usage counter reaches zero, the connection with target is closed and DB connection disposed.
• During graceful shutdown, KEDA closes all active pools before termination.
• In addition, wrote a logic considering the extension of connection pool later for other scalers as you have guided before

First time contributing to open source. I hope the approach looks good and makes sense — I’ll test it further and raise a PR soon! :)