bigquery-emulator icon indicating copy to clipboard operation
bigquery-emulator copied to clipboard

TO_JSON_STRING with DATE field does not produce valid JSON - quotes missing

Open pavelfomin opened this issue 2 months ago • 3 comments

What happened?

The following test fails

    def "use Job.getQueryResults with TO_JSON_STRING"() {
        given:
        QueryJobConfiguration queryJobConfiguration = QueryJobConfiguration
            .newBuilder("SELECT TO_JSON_STRING(t,true) from (SELECT * FROM dataset.table) as t")
            .build()

        JobId jobId = JobId.newBuilder().setProject(project).setRandomJob().build()
        Job job = bigQuery.create(JobInfo.newBuilder(queryJobConfiguration)
            .setJobId(jobId)
            .build())

        when:
        TableResult tableResult = job.getQueryResults()
        List<Map> results = tableResult.streamAll().map(f ->
            objectMapper.readValue(f[0].value.toString(), Map)
        ).toList()

        then:
        results.size() == 2
    }

This is because DATE field value is missing quotes:

{"id":1,"name":"alice","dob":1990-01-01}

Note. Date field value is missing quotes. Works as expected in real BQ.

What did you expect to happen?

Date field in the JSON should be quoted.

How can we reproduce it (as minimally and precisely as possible)?

git clone -b feature/TO_JSON_STRING [email protected]:pavelfomin/spring-boot-bq-example.git
./gradlew clean build

Anything else we need to know?

Possibly related to #218, #265, #32

pavelfomin avatar Nov 06 '25 14:11 pavelfomin

Yes, you're correct in that this is related to those issues! @coreyp1 put together a patch based off of my team's original PR here: https://github.com/goccy/go-zetasqlite/pull/111

I'll revisit this issue soon.

ohaibbq avatar Nov 06 '25 18:11 ohaibbq

This was fixed and released in the Recidiviz fork of the emulator v0.6.6-recidiviz.1 https://github.com/Recidiviz/bigquery-emulator/releases/tag/v0.6.6-recidiviz.1

ohaibbq avatar Nov 13 '25 03:11 ohaibbq

Thank you @ohaibbq! I can confirm that the fix is working on our side.

pavelfomin avatar Nov 13 '25 15:11 pavelfomin