fix/postgresql/pam-authentication-failed
PostgreSQL error

PAM authentication failed for user

Updated Oct 12, 20263-min read
TL;DR

Amazon RDS and Aurora PostgreSQL check IAM logins through PAM. This error means the user has the rds_iam role, so the server expected an IAM auth token, but the token it received did not pass. The password itself is never the problem here; the token is.

  • ·The token was signed for a different region than the database
  • ·The token was signed for a different host or port than the RDS endpoint, for example a CNAME or a local tunnel
  • ·The IAM policy does not allow rds-db:connect for this database user and resource ID
  • ·The token is older than 15 minutes, or the client clock is off
  • ·IAM database authentication is turned off on the instance or cluster
CHECK FIRSTGenerate a fresh token with aws rds generate-db-auth-token using the exact RDS endpoint, port and region, and log in with psql within a minute. If that works, the problem is how your client builds or reuses the token.

What you're seeing

The connection reaches the server and TLS succeeds, then the login fails with SQLSTATE 28000. The server log on RDS records the same PAM failure.

text
psql: error: connection to server at "mydb.abc123xyz.us-east-1.rds.amazonaws.com" (10.0.1.25), port 5432 failed: FATAL:  PAM authentication failed for user "iam_user"

FATAL:  PAM authentication failed for user "iam_user"
Also seen as: pam authentication failed for user rds, fatal pam authentication failed for user rds iam, pam authentication failed for user aws rds, aws rds postgres pam authentication failed for user

What's causing this

Ranked most-likely first.

  1. 1

    Token signed for the wrong region

    The signature includes the region. A token signed in us-west-2 for a database in us-east-1 is rejected with exactly this error. This often happens when the AWS profile's default region differs from the database's.

  2. 2

    Token signed for the wrong host or port

    The token must be signed for the RDS endpoint and its real port. Signing for a custom DNS name, an IP address, or localhost and a forwarded port (SSM, SSH, kubectl) produces a token RDS will not accept.

  3. 3

    The IAM policy doesn't cover this user or resource

    rds-db:connect must name the database user and the resource ID, not the instance name: dbuser:db-XXXX/iam_user for an instance, dbuser:cluster-XXXX/iam_user for an Aurora cluster. A typo in either part fails the login.

  4. 4

    Expired token or a skewed clock

    Tokens are valid for 15 minutes. A connection pool that reconnects with a token created at startup fails later. A client clock that is several minutes off fails right away.

  5. 5

    IAM database authentication is off

    The instance or cluster setting must be on. Users with the rds_iam role cannot log in with a password either, so they fail in both cases.

How to fix it

Step 1: Confirm IAM authentication is on

Check the instance, or the cluster for Aurora. If it says false, turn it on with modify-db-instance or modify-db-cluster.

bash
aws rds describe-db-instances --db-instance-identifier mydb \
  --query 'DBInstances[0].IAMDatabaseAuthenticationEnabled'

aws rds describe-db-clusters --db-cluster-identifier my-aurora \
  --query 'DBClusters[0].IAMDatabaseAuthenticationEnabled'

Step 2: Check the user has the rds_iam role

Connect as the master user and confirm the grant. Without it, the server asks for a normal password instead of a token.

sql
SELECT pg_has_role('iam_user', 'rds_iam', 'member');
-- if false:
GRANT rds_iam TO iam_user;

Step 3: Check who you are and what the policy allows

Make sure the credentials in use belong to the role or user the policy is attached to, and that the policy names the resource ID.

bash
aws sts get-caller-identity

# Resource ID for an instance (db-...) or an Aurora cluster (cluster-...)
aws rds describe-db-instances --db-instance-identifier mydb \
  --query 'DBInstances[0].DbiResourceId'
aws rds describe-db-clusters --db-cluster-identifier my-aurora \
  --query 'DBClusters[0].DbClusterResourceId'

# The policy statement should allow:
# arn:aws:rds-db:us-east-1:123456789012:dbuser:db-ABCDEFGHIJKL/iam_user

Step 4: Sign a fresh token for the real endpoint and log in

Use the RDS endpoint, its real port and the database's region, and connect with TLS. If you reach RDS through a local tunnel, still sign for the RDS endpoint, not localhost.

bash
TOKEN=$(aws rds generate-db-auth-token \
  --hostname mydb.abc123xyz.us-east-1.rds.amazonaws.com \
  --port 5432 --region us-east-1 --username iam_user)

psql "host=mydb.abc123xyz.us-east-1.rds.amazonaws.com port=5432 \
  dbname=postgres user=iam_user password=$TOKEN \
  sslmode=verify-full sslrootcert=global-bundle.pem"

Step 5: Sync the clock and stop reusing old tokens

If the manual login works but your app fails later, the app is reusing a token past 15 minutes. Generate a new token for every new connection, and make sure the machine's clock syncs automatically.

Prevention

Treat the token like a one-time password: create it right before each new connection, never at startup. Pools need a password callback that signs a fresh token, not a fixed string.

Keep the region, host and port used for signing tied to the endpoint you connect to, and give IAM users a dedicated role with rds-db:connect scoped to their database user.

Debug PostgreSQL faster

1bench is a native GUI for PostgreSQL. Inspect queries, connections, and settings without leaving the app. See what's happening before you have to Google it.

Open PostgreSQL in 1bench