[Bug]: Improve the spanner IT time

[liferoad]

Related Template(s)

spanner*

Template Version

latest

What happened?

job-logs.txt From the recent release log, when running Spanner ITs, it took 1h31m. We should optimize the running test.

Time(s)    Test_Name
1,704.025  com.google.cloud.teleport.v2.templates.MySQLSourceDbToSpannerSimpleIT
1,432.843  com.google.cloud.teleport.v2.templates.SpannerToSourceDbWideRowBasicIT
1,326.577  com.google.cloud.teleport.v2.templates.PostgreSQLDataTypesIT
1,278.379  com.google.cloud.teleport.v2.templates.endtoend.ForwardAndReverseMigrationEndToEndIT
1,276.618  com.google.cloud.teleport.v2.templates.MySQLDataTypesIT
1,101.849  com.google.cloud.teleport.v2.templates.DataStreamToSpannerWideRowForMaxTableNameWithMaxColumnNameIT
1,068.503  com.google.cloud.teleport.v2.templates.DataStreamToSpannerWideRowForMax16KeyTablePerDatabaseIT
1,059.708  com.google.cloud.teleport.v2.templates.DataStreamToSpannerWideRowForMaxColumnsPerTablesIT
1,054.155  com.google.cloud.teleport.v2.templates.SpannerToSourceDbTimezoneIT
1,034.398  com.google.cloud.teleport.v2.templates.DataStreamToSpannerWideRowForMax9MibTablePerDatabaseIT
996.763    com.google.cloud.teleport.v2.templates.GCSToSourceDbInterleaveMultiShardIT
994.383    com.google.cloud.teleport.v2.templates.DataStreamToSpannerWideRowForMaxTableNameIT
975.837    com.google.cloud.teleport.v2.templates.MySQLForeignKeyDependencyIT
957.755    com.google.cloud.teleport.v2.templates.GCSToSourceDbInterleaveIT
914.849    com.google.cloud.teleport.v2.templates.MySQLSourceDBToSpannerWideRowInterleaveDepthIT
914.376    com.google.cloud.teleport.v2.templates.SpannerChangeStreamToGcsMultiShardIT
913.715    com.google.cloud.teleport.v2.templates.SpannerToSourceDbInterleaveMultiShardIT
912.042    com.google.cloud.teleport.v2.templates.TimezoneIT
904.415    com.google.cloud.teleport.v2.templates.GCSToSourceDbWithReaderIT
883.866    com.google.cloud.teleport.v2.templates.SpannerToSourceDbIT
875.497    com.google.cloud.teleport.v2.templates.SpannerToCassandraSourceDbIT
866.845    com.google.cloud.teleport.v2.templates.MySQLIdentitySchemaMapperWithTransformationIT
844.141    com.google.cloud.teleport.v2.templates.DataStreamToSpannerShardedMigrationWithMigrationShardIdColumnIT
841.219    com.google.cloud.teleport.v2.templates.PostgreSQLWithUniformizationIT
835.774    com.google.cloud.teleport.v2.templates.CassandraAllDataTypesIT
829.831    com.google.cloud.teleport.v2.templates.SpannerToCassandraSourceDBCustomTransformationIT
824.275    com.google.cloud.teleport.v2.templates.MySQLIdentitySchemaMapperIT
811.819    com.google.cloud.teleport.v2.templates.MySQLStringOverridesSchemaMapperIT
807.204    com.google.cloud.teleport.v2.templates.MySQLFileOverridesSchemaMapperIT
806.921    com.google.cloud.teleport.v2.templates.MySQLSessionSchemaMapperWithTransformationIT
806.897    com.google.cloud.teleport.v2.templates.MySQLSourceDBToSpannerWideRowMaxSizeStringIT
798.366    com.google.cloud.teleport.v2.templates.MySQLDDLIT
797.364    com.google.cloud.teleport.v2.templates.MySQLSessionSchemaMapperWithTableFilterIT
795.596    com.google.cloud.teleport.v2.templates.MySQLSessionSchemaMapperIT
795.433    com.google.cloud.teleport.v2.templates.MySQLSingleShardIT
794.264    com.google.cloud.teleport.v2.templates.MySQLToSpannerWiderowForMaxColumnsPerTableIT
791.747    com.google.cloud.teleport.v2.templates.PostgreSQLReservedKeywordsIT
788.259    com.google.cloud.teleport.v2.templates.MySQLSourceDBToSpannerWideRowMaxColumnAndTableSizeIT
783.82     com.google.cloud.teleport.v2.templates.MySQLSourceDbToSpannerWideRowMaxColumnsTableKeyIT
775.308    com.google.cloud.teleport.v2.templates.MySQLSourceDbToSpannerWideRowMaxSizeTableKeyIT
774.154    com.google.cloud.teleport.v2.templates.SpannerToSourceDbCustomShardIT
773.361    com.google.cloud.teleport.v2.templates.SpannerToMySqlSourceDbWideRowMaxColumnsIT
770.478    com.google.cloud.teleport.v2.templates.SpannerToCassandraSourceDbMaxColumnsIT
766.912    com.google.cloud.teleport.v2.templates.PostgreSQLIdentitySchemaMapperIT
764.168    com.google.cloud.teleport.v2.templates.PostgreSQLNamespaceIT
748.35     com.google.cloud.teleport.v2.templates.MySQLCustomTransformationsNonShardedIT
736.117    com.google.cloud.teleport.v2.templates.SpannerToCassandraSourceDbMaxColumnsSizeIT
665.913    com.google.cloud.teleport.v2.templates.SpannerToSourceDbDatatypeIT
652.85     com.google.cloud.teleport.v2.templates.SpannerChangeStreamToGcsSingleShardIT
625.725    com.google.cloud.teleport.v2.templates.SpannerToCassandraSourceDbWideRow10MbIT
622.319    com.google.cloud.teleport.v2.templates.DataStreamToSpannerShardedMigrationWithoutMigrationShardIdColumnIT
616.5      com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseShardedMigrationWithoutMigrationShardIdColumnIT
616.091    com.google.cloud.teleport.v2.templates.SpannerChangeStreamToGcsCustomShardIT
612.72     com.google.cloud.teleport.v2.templates.SpannerToMySqlSourceDbWideRow10MbIT
564.574    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseShardedMigrationWithMigrationShardIdColumnIT
538.91     com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseEventsIT
522.797    com.google.cloud.teleport.v2.templates.GCSToSourceDbWithoutReaderIT
522.715    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseFileOverridesIT
496.858    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseMixedIT
493.773    com.google.cloud.teleport.v2.templates.DataStreamToSpannerMixedIT
488.191    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabasePKFocusedIT
471.673    com.google.cloud.teleport.v2.templates.DataStreamToSpannerSessionIT
463.04     com.google.cloud.teleport.v2.templates.DatastreamToSpannerSingleDFShardedMigrationIT
452.56     com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseSessionIT
452.085    com.google.cloud.teleport.v2.templates.DataStreamToSpannerDDLIT
443.389    com.google.cloud.teleport.v2.templates.DataStreamToSpannerEventsIT
442.165    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseDDLIT
409.714    com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseStringOverridesIT
399.11     com.google.cloud.teleport.v2.templates.SeparateShadowTableDatabaseSingleDFShardedMigrationIT
371.438    com.google.cloud.teleport.v2.templates.DataStreamToSpannerFileOverridesIT
368.936    com.google.cloud.teleport.v2.templates.DataStreamToSpannerStringOverridesIT
56.208     com.google.cloud.teleport.v2.templates.spanner.DatastreamToSpannerExceptionClassifierIT

Relevant log output

[liferoad]

https://github.com/GoogleCloudPlatform/DataflowTemplates/actions/runs/15567937299

[kennknowles]

Are they running single-threaded? I know we sometimes have done that to reduce quota-based flakiness. I hope we can set them back to parallel, if so. I doubt we have one job that takes 1.5hrs.