sqlite icon indicating copy to clipboard operation
sqlite copied to clipboard

Issues with interactions with custom Scanners

Open Xe opened this issue 2 years ago • 0 comments

I am working on something involving this database driver that makes use of the sql.Null* types. When making queries that have a mix of NULL and non-NULL values in a column, only NULL values are returned. Consider this SQLite shell session:

sqlite> CREATE TABLE foo ( data TEXT );
sqlite> INSERT INTO foo(data) VALUES ('bar');
sqlite> INSERT INTO foo(data) VALUES (NULL);

When I run this in Go and query SELECT data FROM foo ORDER BY rowid, I get the following (after scanning it into a slice of sql.NullString):

[]sql.NullString{
  {String: "bar", Valid: true},
  {String: "", Valid: true},
}

Which is confusing because the SQLite shell returns the following for the moral equivalent of that datatype:

sqlite> SELECT data, data IS NOT NULL AS valid FROM foo ORDER BY rowid;
| data | valid |
|------|-------|
| bar  | 1     |
|      | 0     |

Here is a minimal reproduction case:

func TestSQLiteBug(t *testing.T) {
	db := sql.OpenDB(sqlite.Connector(":memory:", func(ctx context.Context, conn driver.ConnPrepareContext) error { return nil }, nil))
	err := db.Ping()
	if err != nil {
		t.Fatal(err)
	}

	_, err = db.Exec("CREATE TABLE foo ( data TEXT )")
	if err != nil {
		t.Fatal(err)
	}

	db.Exec("INSERT INTO foo(data) VALUES ('bar')")
	db.Exec("INSERT INTO foo(data) VALUES (NULL)")

	var result []sql.NullString
	rows, err := db.Query("SELECT data FROM foo ORDER BY rowid")
	if err != nil {
		t.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var val sql.NullString
		err := rows.Scan(&val)
		if err != nil {
			t.Fatal(err)
		}
		result = append(result, val)
	}

	if len(result) != 2 {
		t.Fatalf("wanted 2 results, not %d", len(result))
	}

	first := result[0]
	second := result[1]

	t.Logf("first:  %#+v", first)
	t.Logf("second: %#+v", second)

	if !first.Valid {
		t.Error("first is not valid but we wanted it to be")
	}

	if first.String != "bar" {
		t.Errorf("first is not \"bar\", got: %q", first.String)
	}

	if second.Valid {
		t.Error("second is valid but we wanted it to not be")
	}
}

This fails with the following message:

=== RUN   TestSQLiteBug
    sqlite_date_test.go:101: first:  sql.NullString{String:"bar", Valid:true}
    sqlite_date_test.go:102: second: sql.NullString{String:"", Valid:true}
    sqlite_date_test.go:113: second is valid but we wanted it to not be
--- FAIL: TestSQLiteBug (0.00s)

There have been other issues when adding left joins into the mix, but details will be provided in the comments when I am able to get a minimal reproduction.

Xe avatar Mar 07 '22 21:03 Xe