IAM Authentication in AWS/GCP - dynamic passwords and connection pooling

[williamdenton]

Issue description

Both aws and gcp allow use am IAM credentials to connect to the database. This is great as you don't have static passwords to manage, the password auth is delegated to IAM instead.

Both GCP and AWS provide tooling for generating a password on the client, this password can then be used in the regular way in the connection string. Where this is a pain is for long lived processes, the password is only valid for a short period of time, which means a new password must be generated occasionally and the connection string updated. This breaks connection pooling as the password forms part of the connection string, and this the pool of connections.

https://cloud.google.com/sql/docs/mysql/iam-logins https://docs.aws.amazon.com/cli/latest/reference/rds/generate-db-auth-token.html

I have previously implemented support for a callback that supplies a password on demand which is executed when the pooled connection is being opened by the driver (rather than an already open pooled connection that is being supplied to the application) in postgres for .Net see https://github.com/npgsql/npgsql/issues/2500

I'm new to Go and don't yet have the skills to contribute this change yet, but suggesting this would be a useful feature to add to this driver.

Summary of feature:

• Connection string is supplied without a password eg: user@protocol(host)
• somehow we supply a callback/higher order function to generate the password, maybe to the open method?
• when the driver needs to open a new connection the password func is called with necessary argument (host, db, user) and password is generated
• the new connection is added to the pool identified by the original connection string - without password
• connection pooling is not broken

[methane]

You can write a custom connector that wraps Config object. Isn't it enough?

[williamdenton]

hey @methane thanks for your fast reply. I appreciate you pointing me towards the custom connector.

as i mentioned im pretty new to go, so i've done some digging on what a connector is. Do you mean customising this code? It feels like this would be a bit hacky and not resharable in the community for all to benefit from. https://github.com/go-sql-driver/mysql/blob/a477f69f3c2abaf4646680bdc3a65d5172a6566e/connector.go#L91-L95

or perhaps a custom plugin that could be added here https://github.com/go-sql-driver/mysql/blob/0c62bb2791485d4260371bcc6017321de93b2430/auth.go#L263-L270

I found these documents but they appear to suffer from connection pooling issues i'm concerned about on the face of it as they are simply constructing the dsn with the custom password and would result in many pools being created and not being reused.

• https://cloud.google.com/sql/docs/mysql/samples/cloud-sql-mysql-databasesql-connect-connector
• https://docs.amazonaws.cn/en_us/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Go.html

[methane]

Using DSN is classic way. There is no flexibility on it.

Connector is new way to open connection pool (e.g. sql.DB). See this instead:

• Connector
• sql.OpenDB()

You can write a custom connector that wraps existing connector.

type MyConnector struct {
    m      sync.Mutex
    config *mysql.Config
}

func (c *MyConnector) Connect(ctx context.Context) (driver.Conn, error) {
    c.Lock()
    cc, err := mysql.NewConnector(c.config)
    c.Unlock()
    if err != nil {
        return nil, err
    }
    return cc.Connect(ctx)  // Safe for concurrent use. Cache cc if performance is important.
}

// You can customize anything in the Config object without reopening connection pool.

func (c *MyConnector) UpdatePassword(password string) {
    c.Lock()
    defer c.Unlock()
    c.config.Passwd = password
}

func (c *MyConnector) UpdateHost(host string) {
    c.Lock()
    defer c.Unlock()
    c.config.Addr = fmt.Sprintf("%s:3306", host)
}

[shogo82148]

Here is my implementation for Amazon RDS

https://github.com/shogo82148/rdsmysql