cockroachdb
roachtest: schemachange/mixed-versions failed [attempting to append refresh spans after the tracked timestamp has moved forward]
roachtest.schemachange/mixed-versions failed with artifacts on release-23.2 @ f88e669339210a94016ce6555168dfffa5df8159:
(mixedversion.go:594).Run: mixed-version test failure while running step 8 (run "run schemachange workload and validation in mixed version"): full command output in run_062138.872700845_n4_COCKROACHRANDOMSEED3.log: COMMAND_PROBLEM: exit status 1
test artifacts and logs in: /artifacts/schemachange/mixed-versions/run_1
Parameters:
ROACHTEST_arch=amd64ROACHTEST_cloud=gceROACHTEST_coverageBuild=falseROACHTEST_cpu=4ROACHTEST_encrypted=falseROACHTEST_metamorphicBuild=falseROACHTEST_ssd=0
Same failure on other branches
- #124722 roachtest: schemachange/mixed-versions failed [debug doctor validation: existing descriptors that still need to be dropped] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-23.1.22-rc]
This test on roachdash | Improve this report!
Jira issue: CRDB-39421
"message": "***UNEXPECTED ERROR; Failed to generate a random operation: Collect: CollectRows: \"WITH \\\"descriptors\\\" AS (SELECT\\n\\t\\tdescriptor.id,\\n\\t\\t\\\"parentSchemaID\\\" AS schema_id,\\n\\t\\tnamespace.name AS name,\\n\\t\\tcrdb_internal.pb_to_json('desc', descriptor) AS descriptor\\n\\tFROM system.descriptor\\n\\tJOIN system.namespace ON namespace.id = descriptor.id\\n\\tWHERE \\\"parentID\\\" = (SELECT id FROM system.namespace WHERE name = current_database() AND \\\"parentID\\\" = 0)\\n\\t),\\n\\\"enums\\\" AS (SELECT id, schema_id, name, descriptor-\u003e'type' AS descriptor FROM descriptors WHERE descriptor ? 'type'),\\n\\\"enum_members\\\" AS (SELECT *, jsonb_array_elements(descriptor-\u003e'enumMembers') AS member FROM enums)\\nSELECT\\n\\t\\t\\t\\tquote_ident(schema_id::REGNAMESPACE::TEXT) || '.' || quote_ident(name) AS name,\\n\\t\\t\\t\\tquote_literal(member-\u003e\u003e'logicalRepresentation') AS value,\\n\\t\\t\\t\\tCOALESCE(member-\u003e\u003e'direction' = 'REMOVE', false) AS dropping,\\n\\t\\t\\t\\tCOALESCE(json_array_length(descriptor-\u003e'referencingDescriptorIds') \u003e 0, false) AS has_references\\n\\t\\t\\tFROM enum_members\\n\\t\" []: ERROR: internal error: attempting to append refresh spans after the tracked timestamp has moved forward. batchTimestamp: 1717914114.781309202,0 refreshedTimestamp: 1717914114.854425156,1 (SQLSTATE XX000)",
This was supposed to be fixed by https://github.com/cockroachdb/cockroach/pull/123154.
According to mixed-version-test.log, this test was using v23.1.22:
[mixed-version-test] 06:19:54 mixedversion.go:591: mixed-version test plan for upgrading from "v23.1.22" to "release-23.2":
#123154 is included in that version. @yuzefovich curious if you have any other ideas on this.
I'll take a look at some point, but it might be some time before I get to this.
roachtest.schemachange/mixed-versions failed with artifacts on release-23.2 @ 924730113eefd257bd9cff3dffcc5241aa6aceaa:
(mixedversion.go:620).Run: mixed-version test failure while running step 17 (run "run schemachange workload and validation in mixed version"): full command output in run_062434.598064605_n4_COCKROACHRANDOMSEED2.log: COMMAND_PROBLEM: exit status 1
test artifacts and logs in: /artifacts/schemachange/mixed-versions/run_1
Parameters:
ROACHTEST_arch=amd64ROACHTEST_cloud=gceROACHTEST_coverageBuild=falseROACHTEST_cpu=4ROACHTEST_encrypted=falseROACHTEST_metamorphicBuild=falseROACHTEST_ssd=0
Same failure on other branches
- #126374 roachtest: schemachange/mixed-versions failed [C-test-failure O-roachtest O-robot T-sql-foundations branch-master release-blocker]
- #126065 roachtest: schemachange/mixed-versions failed [count-leases: column "crdb_region" does not exist] [C-test-failure O-roachtest O-robot T-sql-foundations branch-release-23.1.23-rc]
This test on roachdash | Improve this report!
Hm, I don't have any bright ideas. I confirmed that on 23.1.22 and 23.2.5 versions we no longer use the streamer for the query in question (referenced here). Unfortunately, this slipped off my radar, and the artifacts for the failure are gone, so I'm closing this as unreproducible.