No pgxDriver.OpenConnector Function

[Osiris1975]

Hello,

Thank you for making this tutorial! I have been struggling exactly with this problem for a couple of days. My service uses GORM (not sure if that matters). I tried to implement my own version of the connect function. I noticed that in the latest version of stdlib package, there is no OpenConnector function. So where you have this:

	pgxConnector := &stdlib.Driver{}
	connector, err := pgxConnector.OpenConnector(connectionString)

I have this:

	pgxDriver := &stdlib.Driver{}
	connector, err := pgxDriver.Open(psqlUrl.String())

And as a result, there's no Connect function available on the connector object, and no way to pass it a context. TBH I still don't completely grasp what context is being used for in this function. Could you explain?

Also, if you have any experience with how this solution can be integrated with GORM, that would be super helpful.

[califlower]

Warning, a little bit long:

Super glad that this is helping! I've been trying to throw solutions I find for silly problems online because I realized how often I turn to google for this stuff and find other people's solutions

---

Anyways, thank you for bringing this up. Since I'm about to head out for the weekend, I'm not going to be able to check this. But I'll take a look on Tuesday and follow up with you here.

So GORM shouldn't matter for getting the driver to work, but I have not used it. We use sqlx. Could you give me the version of pgx you're using, and as long as it's not under an NDA or something, could you paste a larger portion of code here so I can get a better picture? Or paste in the imports you're using, it's possible that ours differs.

Looking at the stdlib https://github.com/jackc/pgx/blob/master/stdlib/sql.go#L192 it still seems to have the open connector function.

If you look thru the connect function. In this particular instance context it's used for logging from what I can tell. But in general context is used for either setting deadlines/timeouts and logging especially when it comes to things like http connections.

I've done a weird amount of hitting my head on a wall to get this to work in my code at work, so I'd love to be able to help you out with this so it doesn't feel all for nothing

[califlower]

This is our specific code for IAM connections. It might be a little bit confusing but maybe the import section, or one of the other parts will help for now

package db

import (
	"context"
	"****** internal company product name import"
	"database/sql"
	"database/sql/driver"
	"fmt"
	"net"
	"os"
	"strings"
	"time"

	"github.com/aws/aws-sdk-go-v2/aws/external"
	v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4"
	"github.com/aws/aws-sdk-go-v2/aws/stscreds"
	"github.com/aws/aws-sdk-go-v2/service/rds/rdsutils"
	"github.com/aws/aws-sdk-go-v2/service/sts"
	"github.com/jackc/pgx/v4/stdlib"
	"github.com/jmoiron/sqlx"
	"golang.org/x/xerrors"
)

type IAMAuth struct {
	DatabaseUser string
	//DatabaseHost is the database endpoint. No port or http type required.
	DatabaseHost string
	DatabasePort string
	//DatabaseName is the name of the database you're connecting to. It's probably just called 'postgres'
	DatabaseName string

	//AmazonResourceName is the name of the role that contains the policy allowing iam authentication. Typically this is the ARN attached to the EC2
	AmazonResourceName string

	AuthTokenGenerator Generator
}

type iamDB struct {
	IAMAuth
}

type IAMAuthGenerator struct{}

type Generator interface {
	GetAuthToken(ctx context.Context, region, cname, port, user, arn string) (string, error)
}

//If not set, database can hang for an extremely long time trying to open a new connection
const databaseConnectionTimeoutMilliseconds = 5000


func (iam *IAMAuthGenerator) GetAuthToken(ctx context.Context, region, cname, port, user, arn string) (string, error) {
	cfg, err := external.LoadDefaultAWSConfig()

	if err != nil {
		log.Error(ctx, fmt.Sprintf("could not connect to db using iam auth: %v", err))
		return "", xerrors.Errorf("could not connect to db using iam auth: %w", err)
	}

	cfg.Region = region

	//Should only send a request if the config has expired tokens
	credProvider := stscreds.NewAssumeRoleProvider(sts.New(cfg), arn)

	signer := v4.NewSigner(credProvider)

	ctxWithTimeout, cancel := context.WithTimeout(ctx, databaseConnectionTimeoutMilliseconds*time.Millisecond)

	defer cancel()

	authToken, err := rdsutils.BuildAuthToken(ctxWithTimeout,
		fmt.Sprintf("%s:%s", cname, port),
		region, user, signer)

	return authToken, err
}

type LookupCNAME func(string) (string, error)

func (ia *IAMAuth) GetConnectionString(ctx context.Context, lookup LookupCNAME) (string, error) {

	cnameUntrimmed, err := lookup(ia.DatabaseHost)

	if err != nil {
		log.Error(ctx, fmt.Sprintf("could not lookup cname during iam auth: %v", err))
		return "", xerrors.Errorf("could not lookup cname during iam auth: %w", err)
	}
	//Trim the trailing dot from the cname
	cname := strings.TrimRight(cnameUntrimmed, ".")
	splitCname := strings.Split(cname, ".")

	if len(splitCname) != 6 {
		return "", xerrors.New(fmt.Sprintf("cname not in AWS format, cname:%s ", cname))
	}

	region := splitCname[2]
	log.Info(ctx, fmt.Sprintf("opening connection to cname=%s, region=%s", cname, region))

	authToken, err := ia.AuthTokenGenerator.GetAuthToken(ctx, region, cname, ia.DatabasePort, ia.DatabaseUser, ia.AmazonResourceName)

	if err != nil {
		log.Error(ctx, fmt.Sprintf("could not build auth token: %v", err))
		return "", xerrors.Errorf("could not build auth token: %w", err)
	}

	var postgresConnection strings.Builder

	postgresConnection.WriteString(
		fmt.Sprintf("user=%s dbname=%s sslmode=%s port=%s host=%s password=%s",
			ia.DatabaseUser,
			ia.DatabaseName,
			"require",
			ia.DatabasePort,
			cname,
			authToken))

	return postgresConnection.String(), nil

}
func (id *iamDB) Connect(ctx context.Context) (driver.Conn, error) {

	connectionString, err := id.IAMAuth.GetConnectionString(ctx, net.LookupCNAME)

	if err != nil {
		log.Error(ctx, fmt.Sprintf("could not get connection string: %v", err))
		return nil, xerrors.Errorf("could not get connection string: %w", err)
	}
	pgxConnector := &stdlib.Driver{}

	connector, err := pgxConnector.OpenConnector(connectionString)

	if err != nil {
		return nil, err
	}

	return connector.Connect(ctx)

}

func (id *iamDB) Driver() driver.Driver {
	return id
}

// driver.Driver interface
func (id *iamDB) Open(name string) (driver.Conn, error) {
	return nil, xerrors.New("driver open method unsupported")
}

func (ia IAMAuth) Connect(ctx context.Context) (*sqlx.DB, error) {
	db := sql.OpenDB(&iamDB{ia})

	err := db.Ping()

	if err != nil {
		return nil, xerrors.Errorf("could not ping db: %w", err)
	}

	return sqlx.NewDb(db, driverName), nil
}

[Osiris1975]

Thanks so much for you willingness to help! This helped a lot and we made a lot of progress today. I'm not 100% sure how much of my company's code I'm allowed to share but we came up with something very similar. Something you might be interested in and what we used to cycle the auth token every 15 minutes is this library:

https://github.com/robfig/cron

[MatteoGioioso]

Hello, thanks for this. However for me is not working, and I am not sure what I am doing wrong:

First of all this method signature in my case is different, it does not require context , but it is imported still from "github.com/aws/aws-sdk-go-v2/service/rds/rdsutils" same as in the code

authToken, err := rdsutils.BuildAuthToken( // No context in my method signature
		fmt.Sprintf("%s:%s", cname, port),
		region, user, signer)

Also the credentials type is wrong:

	arn := fmt.Sprintf("arn:aws:rds-db:%v:%v:dbuser:*/%v", Region, AccountId, User)
	cfg.Region = Region
	credProvider := stscreds.NewAssumeRoleProvider(sts.New(cfg), arn)
	signer := v4.NewSigner(credProvider)

This result in a compiling error:

Cannot use 'signer' (type *Signer) as type aws.CredentialsProvider Type does not implement 'aws.CredentialsProvide

Maybe the code you posted is outdated?

Thanks

[califlower]

Hello, thanks for this. However for me is not working, and I am not sure what I am doing wrong:

First of all this method signature in my case is different, it does not require context , but it is imported still from "github.com/aws/aws-sdk-go-v2/service/rds/rdsutils" same as in the code

authToken, err := rdsutils.BuildAuthToken( // No context in my method signature
		fmt.Sprintf("%s:%s", cname, port),
		region, user, signer)

Also the credentials type is wrong:

	arn := fmt.Sprintf("arn:aws:rds-db:%v:%v:dbuser:*/%v", Region, AccountId, User)
	cfg.Region = Region
	credProvider := stscreds.NewAssumeRoleProvider(sts.New(cfg), arn)
	signer := v4.NewSigner(credProvider)

This result in a compiling error:

Cannot use 'signer' (type *Signer) as type aws.CredentialsProvider Type does not implement 'aws.CredentialsProvide

Maybe the code you posted is outdated?

Thanks

Unfortunately, AWS has completely changed their function signatures as of one of their newer library updates without documenting the change whatsoever (which they shouldn’t be doing according to the Golang specs, it should be a new version).

Since this is a breaking undocumented change, and based on some of their comments towards questions asking for better password lifecycle management I’d recommend using a different solution for authentication and steer clear of the AWS sdk if you can. Its clear that IAM authentication is a bit of an afterthought. If I do get the time to look for a solution I’ll update it

[califlower]

https://github.com/aws/aws-sdk-go/issues/3107

This issue seems to have some sample code. If you do find a solution though please update this issue with a working example

[MatteoGioioso]

Your solution was actually suggested to me by the AWS customer support.

Thank you very much for that link: https://github.com/aws/aws-sdk-go/issues/3107 I have managed to hack a solution, but that one is cleaner.

[califlower]

Yeah, I think they technically simplified the way stuff is done I just wish it was documented and didn’t break the existing project setup. If that worked for you, I’m going to update this article later today with his changes. Thank you