pgx icon indicating copy to clipboard operation
pgx copied to clipboard

Scanning of char into string possible in TextFormat but not in BinaryFormat

Open Simerax opened this issue 6 months ago • 2 comments
trafficstars

Describe the bug First of I think this is a bug but im not sure.
If you scan a 'char' into a go string it works if the wire? format is text and not binary. As soon as it is binary it does not work anymore.

Workaround

We just fixed our code to use rune instead of string which is probably better anyway but I thought i'd share the problem.

The only problem really is that as the user of the library you don't really know in advance if the database will use the binary or text format. At least that seems to happen in our actual code. On Server startup everything works, later during runtime the same query starts failing all of a sudden.

To Reproduce Steps to reproduce the behavior:

package main

import (
	"context"
	"os"

	"github.com/jackc/pgx/v5"
)

func main() {

	ctx := context.Background()

	conn, err := pgx.Connect(ctx, os.Getenv("DATABASE_URL"))
	if err != nil {
		panic(err)
	}
	defer conn.Close(ctx)

	rows, err := conn.Query(ctx, `SELECT confupdtype FROM pg_constraint LIMIT 1;`)
	if err != nil {
		panic(err)
	}
	defer rows.Close()

	for rows.Next() {
		updateType := ""
		if err := rows.Scan(&updateType); err != nil {
			panic(err)
		}
	}
}

go.mod

module bugreport

go 1.22.2

require github.com/jackc/pgx/v5 v5.7.4

require (
	github.com/jackc/pgpassfile v1.0.0 // indirect
	github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
	golang.org/x/crypto v0.31.0 // indirect
	golang.org/x/text v0.21.0 // indirect
)

Please run your example with the race detector enabled. For example, go run -race main.go or go test -race.

Expected behavior I expect the rows.Scan to work regardless of the format. Especially since I as the user of the library don't choose the format.

Actual behavior

panic: can't scan into dest[0] (col: confupdtype): cannot scan char (OID 18) in binary format into *string

goroutine 1 [running]:
main.main()
        .../pgx_bug/main.go:29 +0x30a
exit status 2

Version

  • Go: go version go1.22.2 windows/amd64
  • PostgreSQL: PostgreSQL 17.2 on x86_64-windows, compiled by msvc-19.41.34123, 64-bit
  • pgx: v5.7.4

Additional context Add any other context about the problem here.

Simerax avatar May 09 '25 10:05 Simerax

We just fixed our code to use rune instead of string which is probably better anyway but I thought i'd share the problem.

Using rune or byte is the correct solution.

If you scan a 'char' into a go string it works if the wire? format is text and not binary. As soon as it is binary it does not work anymore.

What's happening is the QCharCodec, which is the code the handles encoding and decoding (quoted) char, does not support string. It only supports rune and byte.

However, the pgx type system as a whole always supports a text formatted value being scanned into a string. This is useful because it allows unknown types to at least be handled as strings. But it also allows scanning a char into a string even though that wasn't really intended.

The only problem really is that as the user of the library you don't really know in advance if the database will use the binary or text format. At least that seems to happen in our actual code. On Server startup everything works, later during runtime the same query starts failing all of a sudden.

That shouldn't happen. pgx should only use a different format if you use a different QueryExecMode or are manually specifying the formats with QueryResultFormats.

jackc avatar May 10 '25 14:05 jackc

However, the pgx type system as a whole always supports a text formatted value being scanned into a string. This is useful because it allows unknown types to at least be handled as strings. But it also allows scanning a char into a string even though that wasn't really intended.

Okay this makes sense.

That shouldn't happen. pgx should only use a different format if you use a different QueryExecMode or are manually specifying the formats with QueryResultFormats.

I checked our code and actually noticed that we use pgx.QueryExecModeExec where it works and not where it doesn't work.

Im fine with closing this unless you want to document or change anything about it.

Simerax avatar May 12 '25 07:05 Simerax