Redis Signer for connecting to Redis 7 using IAM authentication

[meenar-se]

Describe the feature

Need an implementation for Redis signer so that its useful to connect to Elastic cache redis version 7 or above using IAM Authentication.

Use Case

Recently AWS introduced an option to connect to Redis 7 using IAM authentication. But its not directly supported by the library. So we have slightly tweaked the RDS signer to support Redis as well. It will be useful if we add the Redis Signer as well in library itself.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

3.350.0

Environment details (OS name and version, etc.)

Macos 12.6.3

[RanVaknin]

Hi @meenar-se ,

Thanks for opening this feature request.

I think your request is reasonable, but like RDS signer, this would need to be a handwritten utility (whereas most of the SDK is code generated from the API models of the each service), this means that it will have to be properly designed and implemented in a uniform fashion across all SDKs.

Additionally, feature requests are accepted and implemented based on community engagement (comments, upvotes, or duplicate FRs). This helps us prioritize features in the most impactful way, and use the teams resources wisely.

I will transfer this FR to the cross SDK repo for it to gain traction there. This unfortunately means that it will not get prioritized right away, but it doesn't mean it wont in the future.

Since you marked the "I may be able to implement this feature request" checkbox, I'd encourage you to write your implementation here on this ticket for two reasons:

1. It will allow other community members that are facing the same issue to use your solution as a workaround.
2. When time comes to execute on this feature request, it will be a good starting points for one of the devs to refer to and test against.

Thanks again 😄 Ran

[meenar-se]

Implementation

Configuration file: runtimeConfig.ts

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { Hash } from '@aws-sdk/hash-node';
import { loadConfig } from '@aws-sdk/node-config-provider';
import { SignerConfig } from './Signer';

/**
 * @internal
 */
export const getRuntimeConfig = (config: SignerConfig) : SignerConfig => ({
  runtime: 'node',
  sha256: config?.sha256 ?? Hash.bind(null, 'sha256'),
  credentials: config?.credentials ?? fromNodeProviderChain(),
  region: config?.region ?? loadConfig(NODE_REGION_CONFIG_OPTIONS, NODE_REGION_CONFIG_FILE_OPTIONS),
  expiresIn: 900,
  ...config,
} as SignerConfig);

Signer Implementation: Signer.ts

import { SignatureV4 } from '@aws-sdk/signature-v4';
import {
  AwsCredentialIdentity,
  AwsCredentialIdentityProvider,
  ChecksumConstructor,
} from '@aws-sdk/types';
import { formatUrl } from '@aws-sdk/util-format-url';

import { getRuntimeConfig as __getRuntimeConfig } from './runtimeConfig';
import { defaultProvider } from '@aws-sdk/credential-provider-node';

export interface SignerConfig {
  /**
   * The AWS credentials to sign requests with. Uses the default credential provider chain if not specified.
   */
  credentials?: AwsCredentialIdentity | AwsCredentialIdentityProvider
  /**
   * The hostname of the database to connect to.
   */
  hostname: string
  /**
   * The port number the database is listening on.
   */
  port?: number
  /**
   * The region the database is located in. Uses the region inferred from the runtime if omitted.
   */
  region?: string
  /**
   * The SHA256 hasher constructor to sign the request.
   */
  sha256?: ChecksumConstructor
  /**
   * The username to login as.
   */
  username: string
  runtime?: string
  expiresIn?: number
}

/**
 * The signer class that generates an auth token to a database.
 */
export class Signer {

  private readonly protocol: string = 'http:';
  private readonly service: string = 'elasticache';

  public constructor(public configuration: SignerConfig) {
    this.configuration = __getRuntimeConfig(configuration);
    
  }

  public async getAuthToken(): Promise<string> {
    const signer = new SignatureV4({
      service: this.service,
      region: this.configuration.region!,
      credentials: this.configuration.credentials ?? defaultProvider(),
      sha256: this.configuration.sha256!,
    });

    const request = new HttpRequest({
      method: 'GET',
      protocol: this.protocol,
      hostname: this.configuration.hostname,
      port: this.configuration.port,
      query: {
        Action: 'connect',
        User: this.configuration.username,   
      },
      headers: {
        host: `${this.configuration.hostname}`,
      },
    });

    const presigned = await signer.presign(request, {
      expiresIn: this.configuration.expiresIn,
    });
    const format = formatUrl(presigned).replace(`${this.protocol}//`, '');
    console.log(format);
    
    return format;
  }
}

Usage example

import { createClient } from "redis";
import { Signer } from "./Signer";


export const redisConnect = async () => {
  console.log("calling redis connect");
  const credentials = await generateAssumeRoleCreds();
  const sign = new Signer({
    region: region,
    hostname: `${replicationGroupId}`,
    username: userId,
    credentials: credentials,
  });
  const presignedUrl = await sign.getAuthToken();
  const redisConfig = {
    url: `redis://master.xxx.xxx.xxxx.use1.cache.amazonaws.com:6379`,
    password: presignedUrl,
    username: userId,
    socket: {
      tls: true,
      rejectUnauthorized: false,
    },
  };
  const redisClient = await createClient(redisConfig);
  try {
    await redisClient.connect();
    console.log(await redisClient.get("key"));
  } catch (error) {
    console.log("Error catched " + error);
  }
};

[meenar-se]

I can also work on the implementing this feature in SDK. We can have the signer as common utility and use it for REDIS and RDS. Please suggest.

[X-Guardian]

HI @meenar-se, did you ever get your sample code to work? I've tried it on Elasticache redis/valkey serverless and non-severless and always get ErrorReply: WRONGPASS invalid username-password pair or user is disabled.

For serverless, it looks like an additional ResourceType=ServerlessCache query parameter is needed, but I've tried it with and without. Ref: https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/auth-iam.html#auth-iam-Connecting

[X-Guardian]

I got it working using https://github.com/aws-samples/elasticache-iam-auth-demo-app as reference. Here is my updated code including serverless support:

Configuration file: runtimeConfig.ts

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { NODE_REGION_CONFIG_FILE_OPTIONS, NODE_REGION_CONFIG_OPTIONS } from '@smithy/config-resolver';
import { Hash } from '@smithy/hash-node';
import { loadConfig } from '@smithy/node-config-provider';
import { SignerConfig } from './signer';

/**
 * @internal
 */
export const getRuntimeConfig = (config: SignerConfig) => {
  return {
    runtime: 'node',
    sha256: config?.sha256 ?? Hash.bind(null, 'sha256'),
    credentials: config?.credentials ?? fromNodeProviderChain(),
    region: config?.region ?? loadConfig(NODE_REGION_CONFIG_OPTIONS, NODE_REGION_CONFIG_FILE_OPTIONS),
    ...config,
  };
};

Signer Implementation: signer.ts

import { formatUrl } from '@aws-sdk/util-format-url';
import { HttpRequest } from '@smithy/protocol-http';
import { SignatureV4 } from '@smithy/signature-v4';
import {
  AwsCredentialIdentity,
  AwsCredentialIdentityProvider,
  ChecksumConstructor,
  HashConstructor,
  Provider,
} from '@smithy/types';

import { getRuntimeConfig as __getRuntimeConfig } from './runtimeConfig';

export enum ClusterType {
  normal = 'normal',
  serverless = 'serverless',
}

export interface SignerConfig {
  /**
   * The AWS credentials to sign requests with. Uses the default credential provider chain if not specified.
   */
  credentials?: AwsCredentialIdentity | AwsCredentialIdentityProvider;
  /**
   * The hostname of the cache to connect to.
   */
  cacheName: string;
  /**
   * The resource type of the cluster.
   */
  resourceType: string;
  /**
   * The region the database is located in. Uses the region inferred from the runtime if omitted.
   */
  region?: string;
  /**
   * The SHA256 hasher constructor to sign the request.
   */
  sha256?: ChecksumConstructor | HashConstructor;
  /**
   * The username to login as.
   */
  username: string;
}

/**
 * The signer class that generates an auth token to a database.
 */
export class Signer {
  private readonly credentials: AwsCredentialIdentity | AwsCredentialIdentityProvider;
  private readonly cacheName: string;
  private readonly resourceType: string;
  private readonly protocol: string = 'https:';
  private readonly region: string | Provider<string>;
  private readonly service: string = 'elasticache';
  private readonly sha256: ChecksumConstructor | HashConstructor;
  private readonly username: string;

  constructor(configuration: SignerConfig) {
    const runtimeConfiguration = __getRuntimeConfig(configuration);

    this.credentials = runtimeConfiguration.credentials;
    this.cacheName = runtimeConfiguration.cacheName;
    this.resourceType = runtimeConfiguration.resourceType;
    this.region = runtimeConfiguration.region;
    this.sha256 = runtimeConfiguration.sha256;
    this.username = runtimeConfiguration.username;
  }

  public async getAuthToken(): Promise<string> {
    const signer = new SignatureV4({
      service: this.service,
      region: this.region,
      credentials: this.credentials,
      sha256: this.sha256,
    });

    const request = new HttpRequest({
      method: 'GET',
      protocol: this.protocol,
      hostname: this.cacheName,
      query: {
        Action: 'connect',
        User: this.username,
        // add ResourceType property if serverless
        ...(this.resourceType === 'serverless' ? { ResourceType: 'ServerlessCache' } : {}),
      },
      headers: {
        host: this.cacheName,
      },
    });

    const presigned = await signer.presign(request, {
      expiresIn: 900,
    });

    return formatUrl(presigned).replace(`${this.protocol}//`, '');
  }
}

Usage example

import { fromNodeProviderChain } from '@aws-sdk/credential-providers';
import { createClient } from 'redis';
import { Signer, ClusterType } from './signer';

const options = {
  hostname: 'valkey-serverless.id.serverless.euw2.cache.amazonaws.com',
  cacheName: 'valkey-serverless',
  region: 'eu-west-2',
  clusterType: ClusterType.serverless,
  username: 'valkey-serverless-user',
};

export const redisConnect = async () => {
  let credentials = fromNodeProviderChain();

  console.log('calling redis connect');
  const sign = new Signer({
    credentials: credentials,
    cacheName: options.cacheName,
    region: options.region,
    resourceType: options.clusterType,
    username: options.username,
  });
  const presignedUrl = await sign.getAuthToken();
  const redisConfig = {
    url: options.hostname,
    password: presignedUrl,
    username: options.username,
    socket: {
      tls: true,
      rejectUnauthorized: false,
    },
  };
  const redisClient = createClient(redisConfig);
  try {
    await redisClient.connect();
    console.log(await redisClient.ping());
  } catch (error) {
    console.log('Error catched ' + error);
  }
};

[zachary-blackbird]

Note that IAM creds expire (I think every 15 minutes), so something needs to refresh them. The redis library does not have a refresh built in (but one is proposed). I guess the application layer just needs to reinitialize the redis client on credential timeout.

[lpavliuk]

Just in case someone is trying to implement IAM auth to AWS Elasticache in NodeJS with auto-reconnect when the token expires, here is the implementation that works for me.

IAMAuthTokenRequest.js

const { CrtSignerV4 } = require('@aws-sdk/signature-v4-crt');
const { fromEnv } = require('@aws-sdk/credential-providers');
const url = require('url');

class IAMAuthTokenRequest {
    static REQUEST_METHOD = 'GET';
    static REQUEST_PROTOCOL = 'http:';
    static PARAM_ACTION = 'Action';
    static PARAM_USER = 'User';
    static PARAM_RESOURCE_TYPE = 'ResourceType';
    static RESOURCE_TYPE_SERVERLESS_CACHE = 'ServerlessCache';
    static ACTION_NAME = 'connect';
    static SERVICE_NAME = 'elasticache';
    static TOKEN_EXPIRY_SECONDS = 900;

    constructor({userId, cacheName, region, isServerless}) {
        this.userId = userId;
        this.cacheName = cacheName;
        this.region = region || process.env.AWS_REGION;
        this.isServerless = isServerless || false;
    }

    async getAuthToken(credentials = fromEnv()) {
        let request = {
            method: IAMAuthTokenRequest.REQUEST_METHOD,
            protocol: IAMAuthTokenRequest.REQUEST_PROTOCOL,
            hostname: this.cacheName,
            headers: {
                host: this.cacheName
            },
            query: {
                [IAMAuthTokenRequest.PARAM_ACTION]: IAMAuthTokenRequest.ACTION_NAME,
                [IAMAuthTokenRequest.PARAM_USER]: this.userId,
                ...this.isServerless ? {
                    [IAMAuthTokenRequest.PARAM_RESOURCE_TYPE]: IAMAuthTokenRequest.RESOURCE_TYPE_SERVERLESS_CACHE
                } : {}
            }
        };

        let signed_request = await this.sign(request, credentials);

        return url
            .format(signed_request)
            .toString()
            .replace(`${request.protocol}//`, '');
    }

    async sign(request, credentials) {
        const signer = new CrtSignerV4({
            credentials,
            service: IAMAuthTokenRequest.SERVICE_NAME,
            region: this.region
        });

        return await signer.presign(request, {
            expiresIn: IAMAuthTokenRequest.TOKEN_EXPIRY_SECONDS,
        });
    }
}

module.exports = IAMAuthTokenRequest;

RedisTokenConnector.js

const Redis = require("ioredis");
const StandaloneConnector = require("ioredis/built/connectors/StandaloneConnector").default;

/**
 * RedisTokenConnector is a Redis connector that fetches the password
 * dynamically on each connect() call, and hacks it in the Redis instance.
 */
class RedisTokenConnector extends StandaloneConnector {
    /**
     * @param {Object} options - The options for the RedisTokenConnector.
     * @param {Object} options.tokenConnector - The token connector options.
     * @param {Object} options.tokenConnector.redisRef - Reference to an existing RedisRef instance.
     * @param {Function} options.tokenConnector.getToken - Function used to retrieve the connection password.
     */
    constructor(options) {
        super(options);

        this.redisRef = options.tokenConnector.redisRef;
        this.getToken = options.tokenConnector.getToken;
    }

    /**
     * Connects to the Redis instance, fetching the password dynamically.
     * @param {Function} _ - Error emitter function.
     * @returns {Promise<NetStream>} - The network stream.
     */
    async connect(_) {
        const token = await this.getToken();

        const condition = this.redisRef.current?.condition;
        if (!condition) throw new Error("expected redis.condition to be set at this point");

        if (condition.auth === undefined || typeof condition.auth === "string") {
            condition.auth = token; // password only
        } else if (Array.isArray(condition.auth)) {
            condition.auth = [condition.auth[0], token]; // [username, password]
        }

        return super.connect(_);
    }
}

module.exports = RedisTokenConnector;

Usage example:

const Redis = require('ioredis');
const RedisTokenConnector = require('./RedisTokenConnector');
const IAMAuthTokenRequest = require('./IAMAuthTokenRequest');

let iamAuthTokenRequest = new IAMAuthTokenRequest({
    userId: '<CACHE_USERNAME>',
    cacheName: '<CACHE_NAME>',
    region: 'ap-southeast-2',
    isServerless: false, // "false" in case you use Replication Group
});

const redisRef = { current: null };
const redisClient = new Redis({
    host: '<CACHE_ENDPOINT>',
    port: 6379,
    username: '<CACHE_USERNAME>',
    tls: {}, // Required if TLS encryption is enabled
    tokenConnector: {
        redisRef,
        getToken: async () => await iamAuthTokenRequest.getAuthToken()
    },
    Connector: RedisTokenConnector
    // ...other options
});
// IMPORTANT, if you do not set redisRef.current you'll get an error when connecting
redisRef.current = redisClient;

Also, a big thank you to the answers that helped me understand and implement it in NodeJS:

• https://github.com/aws/aws-sdk-js-v3/issues/6611#issuecomment-2448580661
• https://github.com/redis/ioredis/issues/1738#issuecomment-1969925020

[taraspos]

Potentially node-redis should release support of credential provider somewhere soon:

• https://github.com/redis/node-redis/pull/2877#pullrequestreview-2583227433
• https://github.com/redis/node-redis/issues/821

[taraspos]

Seems like official redis client starting from 5.0.0-next.6 has support of credentialProvider:

• https://github.com/redis/node-redis/tree/master/packages/entraid#service-principal-authentication
• https://github.com/redis/node-redis/pull/2877#issuecomment-2642798575

[aBurmeseDev]

As mentioned above, it appears Redis have added support of credential provider. We currently don't have plans for this at the moment and might revisit in the future. Closing as not planned.

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.