redpanda
redpanda copied to clipboard
[WIP] Flexible partition core assignment dev
Backports Required
- [ ] none - not a bug fix
- [ ] none - this is a backport
- [ ] none - issue does not exist in previous branches
- [ ] none - papercut/not impactful enough to backport
- [ ] v23.3.x
- [ ] v23.2.x
- [ ] v23.1.x
Release Notes
/ci-repeat 1
new failures in https://buildkite.com/redpanda/redpanda/builds/43449#018cd692-23f7-4377-bc13-aaddd17f54af:
"rptest.tests.recovery_mode_test.DisablingPartitionsTest.test_disable"
new failures in https://buildkite.com/redpanda/redpanda/builds/43449#018cd6bc-6d39-499e-88c5-a1ecf7233903:
"rptest.tests.internal_topic_protection_test.InternalTopicProtectionLargeClusterTest.test_schemas_topic"
new failures in https://buildkite.com/redpanda/redpanda/builds/43449#018cd6bc-6d3d-402c-9543-c0b85726bcfa:
"rptest.tests.cloud_storage_timing_stress_test.CloudStorageTimingStressTest.test_cloud_storage_with_partition_moves.cleanup_policy=delete"
new failures in https://buildkite.com/redpanda/redpanda/builds/43570#018ceaf2-3c48-433a-9651-a5de608a0c33:
"rptest.tests.recovery_mode_test.RecoveryModeTest.test_recovery_mode"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d1469-c13e-47a9-8a19-708c0ad2c65f:
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_multiple_nodes_to_the_cluster.partition_count=20"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_and_upgrade"
"rptest.tests.e2e_shadow_indexing_test.EndToEndSpilloverTest.test_spillover.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_flipping_decommission_recommission.node_is_alive=True"
"rptest.tests.rack_aware_replica_placement_test.RackAwarePlacementTest.test_rack_awareness_after_node_operations.rack_layout=ooooFF"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d1469-c134-48b8-b6a5-5ef3cfea2aea:
"rptest.tests.cloud_storage_scrubber_test.CloudStorageScrubberTest.test_scrubber.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.full_node_recovery_test.FullNodeRecoveryTest.test_node_recovery.recovery_type=full"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_rebalancing_node.shutdown_decommissioned=False"
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_node_with_unavailable_node"
"rptest.tests.controller_snapshot_test.ControllerSnapshotTest.test_join_restart_catch_up"
"rptest.tests.rack_aware_replica_placement_test.RackAwarePlacementTest.test_rack_awareness_after_node_operations.rack_layout=xxYYzz"
"rptest.tests.recovery_mode_test.DisablingPartitionsTest.test_disable"
"rptest.tests.e2e_shadow_indexing_test.EndToEndSpilloverTest.test_spillover.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.upgrade_test.UpgradeFromPriorFeatureVersionCloudStorageTest.test_rolling_upgrade.cloud_storage_type=CloudStorageType.S3"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d1469-c13a-4571-9cd0-cddec7c8108d:
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_finishes_after_manual_cancellation.delete_topic=False"
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_nodes_to_cluster.partition_count=20"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_flipping_decommission_recommission.node_is_alive=False"
"rptest.tests.e2e_shadow_indexing_test.EndToEndThrottlingTest.test_throttling.cloud_storage_type=CloudStorageType.S3"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f748-43f5-b899-c3f89c7502e9:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.NO_TIRED_STORAGE.cleanup_policy=compact.delete"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_decommission.kill_same_node=False.decommission_first=True"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_fuzz_admin_ops"
"rptest.tests.consumer_group_test.ConsumerGroupTest.test_dead_group_recovery.static_members=True"
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_node_with_unavailable_node"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_rebalancing_node.shutdown_decommissioned=True"
"rptest.tests.rack_aware_replica_placement_test.RackAwarePlacementTest.test_rack_awareness_after_node_operations.rack_layout=xxYYzz"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_shadow_indexing.num_to_upgrade=0.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_cross_shard.num_to_upgrade=0.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.e2e_shadow_indexing_test.EndToEndThrottlingTest.test_throttling.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.cloud_retention_test.CloudRetentionTimelyGCTest.test_retention_with_node_failures.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.tx_coordinator_migration_test.TxCoordinatorMigrationTest.test_migrating_tx_manager_coordinator.with_failures=False.upgrade=False"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f73c-4046-b176-5e65714d3f18:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.TIRED_STORAGE.cleanup_policy=compact"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_rack_awareness"
"rptest.tests.partition_move_interruption_test.PartitionMoveInterruption.test_cancelling_partition_move.replication_factor=1.unclean_abort=True.recovery=restart_recovery.compacted=True"
"rptest.tests.partition_move_interruption_test.PartitionMoveInterruption.test_cancelling_partition_move.replication_factor=1.unclean_abort=False.recovery=restart_recovery.compacted=True"
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_nodes_to_cluster.partition_count=1"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_cross_shard.num_to_upgrade=2.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_finishes_after_manual_cancellation.delete_topic=False"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_recommissioning_one_of_decommissioned_nodes"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_flipping_decommission_recommission.node_is_alive=False"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_shadow_indexing.num_to_upgrade=2.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.tx_coordinator_migration_test.TxCoordinatorMigrationTest.test_migrating_tx_manager_coordinator.with_failures=False.upgrade=True"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f741-4d6f-a8c0-26bf8ee3b2d9:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=node_add.test_mode=TestMode.NO_TIRED_STORAGE.cleanup_policy=compact.delete"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact.delete"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.TIRED_STORAGE.cleanup_policy=compact.delete"
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.num_to_upgrade=0.with_tiered_storage=False"
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.num_to_upgrade=0.with_tiered_storage=False"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_rack_constraint_repair"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_unavailable_nodes"
"rptest.tests.partition_move_interruption_test.PartitionMoveInterruption.test_cancelling_partition_move_x_core.replication_factor=3.unclean_abort=False.recovery=restart_recovery.compacted=True"
"rptest.tests.scaling_up_test.ScalingUpTest.test_adding_nodes_to_cluster.partition_count=20"
"rptest.tests.scaling_up_test.ScalingUpTest.test_topic_hot_spots"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_and_upgrade"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_finishes_after_manual_cancellation.delete_topic=True"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_flipping_decommission_recommission.node_is_alive=True"
"rptest.tests.rack_aware_replica_placement_test.RackAwarePlacementTest.test_rack_awareness_after_node_operations.rack_layout=ABCDEF"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_shadow_indexing.num_to_upgrade=2.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_cross_shard.num_to_upgrade=2.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.services_self_test.BucketScrubSelfTest.test_missing_segment.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.tx_coordinator_migration_test.TxCoordinatorMigrationTest.test_migrating_tx_manager_coordinator.with_failures=True.upgrade=False"
new failures in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f745-43f7-9b34-ed82986ad2be:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=node_add.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=node_add.test_mode=TestMode.TIRED_STORAGE.cleanup_policy=compact"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.NO_TIRED_STORAGE.cleanup_policy=compact"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_decommission.kill_same_node=False.decommission_first=False"
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.num_to_upgrade=0.with_tiered_storage=True"
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.num_to_upgrade=0.with_tiered_storage=True"
"rptest.tests.partition_move_interruption_test.PartitionMoveInterruption.test_cancellations_interrupted_with_restarts.replication_factor=3"
"rptest.tests.consumer_group_test.ConsumerGroupTest.test_dead_group_recovery.static_members=False"
"rptest.tests.scaling_up_test.ScalingUpTest.test_fast_node_addition"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_multiple_decommissions"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_movement_cancellations"
"rptest.tests.rack_aware_replica_placement_test.RackAwarePlacementTest.test_rack_awareness_after_node_operations.rack_layout=ooooFF"
"rptest.tests.partition_balancer_test.PartitionBalancerTest.test_full_nodes"
"rptest.tests.recovery_mode_test.DisablingPartitionsTest.test_disable"
"rptest.tests.cloud_retention_test.CloudRetentionTimelyGCTest.test_retention_with_node_failures.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_shadow_indexing.num_to_upgrade=0.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.partition_movement_test.SIPartitionMovementTest.test_cross_shard.num_to_upgrade=0.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.controller_snapshot_test.ControllerSnapshotTest.test_join_restart_catch_up"
"rptest.tests.tx_coordinator_migration_test.TxCoordinatorMigrationTest.test_migrating_tx_manager_coordinator.with_failures=True.upgrade=True"
"rptest.tests.workload_upgrade_runner_test.RedpandaUpgradeTest.test_workloads_through_releases.cloud_storage_type=CloudStorageType.S3"
new failures in https://buildkite.com/redpanda/redpanda/builds/43839#018d189f-da1e-4176-a91b-5511ef40a06e:
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.num_to_upgrade=0.with_tiered_storage=True"
new failures in https://buildkite.com/redpanda/redpanda/builds/44519#018d6007-f136-4b9e-aa8d-f87e4681540a:
"rptest.tests.topic_delete_test.TopicDeleteAfterMovementTest.topic_delete_orphan_files_after_move_test"
"rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_orphan_files_test"
new failures in https://buildkite.com/redpanda/redpanda/builds/44519#018d6018-daad-43ca-8af8-d27f817a1261:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=node_add.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact.delete"
"rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_orphan_files_test"
"rptest.tests.tx_coordinator_migration_test.TxCoordinatorMigrationTest.test_migrating_tx_manager_coordinator.with_failures=True.upgrade=False"
new failures in https://buildkite.com/redpanda/redpanda/builds/44519#018d6018-dab3-4cdb-8f67-56de5d45682f:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact.delete"
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.TIRED_STORAGE.cleanup_policy=compact.delete"
new failures in https://buildkite.com/redpanda/redpanda/builds/44519#018d6018-daaa-42c9-9740-049f6120660b:
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_multiple_decommissions"
"rptest.tests.topic_delete_test.TopicDeleteAfterMovementTest.topic_delete_orphan_files_after_move_test"
new failures in https://buildkite.com/redpanda/redpanda/builds/44694#018d7a30-9b2f-409a-b481-943f855f8c98:
"rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=node_add.test_mode=TestMode.NO_TIRED_STORAGE.cleanup_policy=compact.delete"
"rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.num_to_upgrade=0.with_tiered_storage=False"
new failures in https://buildkite.com/redpanda/redpanda/builds/44811#018d83ba-3d9e-4775-831a-b2c132e3ba99:
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=IDEMPOTENT.cleanup_policy=delete"
new failures in https://buildkite.com/redpanda/redpanda/builds/45062#018daf68-c2b7-4a6e-a284-b5eb30f08f76:
"rptest.tests.consumer_group_test.ConsumerGroupTest.test_dead_group_recovery.static_members=True"
"rptest.tests.topic_creation_test.RecreateTopicMetadataTest.test_recreated_topic_metadata_are_valid.replication_factor=5"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=ACKS_ALL.cleanup_policy=compact"
"rptest.tests.consumer_group_recovery_tool_test.ConsumerOffsetsRecoveryToolTest.test_consumer_offsets_partition_count_change"
"rptest.tests.partition_movement_test.PartitionMovementTest.test_deletion_stops_move.num_to_upgrade=0"
"rptest.tests.rpk_topic_test.RpkToolTest.test_consume_from_partition"
"rptest.tests.internal_topic_protection_test.InternalTopicProtectionTest.kafka_nodelete_topics_test.client_type=rpk"
"rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_orphan_files_test"
new failures in https://buildkite.com/redpanda/redpanda/builds/45062#018daf68-c2af-4255-b09c-3c06e7ca0137:
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_working_node.delete_topic=True.tick_interval=3600000"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=ACKS_ALL.cleanup_policy=delete"
"rptest.tests.simple_e2e_test.SimpleEndToEndTest.test_consumer_interruption"
"rptest.tests.partition_movement_test.PartitionMovementTest.test_deletion_stops_move.num_to_upgrade=2"
"rptest.tests.topic_delete_test.TopicDeleteCloudStorageTest.drop_lifecycle_marker_test.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.create_partitions_test.CreatePartitionsTest.test_create_partitions"
"rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_test.with_restart=False"
"rptest.tests.transactions_test.TransactionsTest.delete_topic_with_active_txns_test"
new failures in https://buildkite.com/redpanda/redpanda/builds/45062#018daf68-c2b2-4edc-b397-b8e44977a2c1:
"rptest.tests.read_replica_e2e_test.TestReadReplicaService.test_writes_forbidden.partition_count=10.cloud_storage_type=CloudStorageType.ABS"
"rptest.tests.nodes_decommissioning_test.NodesDecommissioningTest.test_decommissioning_working_node.delete_topic=True.tick_interval=5000"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=ACKS_1.cleanup_policy=compact"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=IDEMPOTENT.cleanup_policy=compact"
"rptest.tests.partition_movement_test.PartitionMovementTest.test_overlapping_changes.num_to_upgrade=0"
"rptest.tests.compatibility.arroyo_test.ArroyoTest.test_arroyo_test_suite"
"rptest.tests.topic_delete_test.TopicDeleteCloudStorageTest.drop_lifecycle_marker_test.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.admin_operations_test.AdminOperationsTest.test_admin_operations"
"rptest.tests.cluster_metrics_test.ClusterMetricsTest.partition_count_decreases_on_deletion_test"
"rptest.tests.group_membership_test.GroupMetricsTest.test_topic_recreation"
"rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_test.with_restart=True"
"rptest.tests.tx_admin_api_test.TxAdminTest.test_delete_topic_from_ongoin_tx"
new failures in https://buildkite.com/redpanda/redpanda/builds/45062#018daf68-c2b5-43f0-aad9-54af3c5ac3bd:
"rptest.tests.audit_log_test.AuditLogTestKafkaApi.test_management"
"rptest.tests.read_replica_e2e_test.TestReadReplicaService.test_writes_forbidden.partition_count=10.cloud_storage_type=CloudStorageType.S3"
"rptest.tests.consumer_group_test.ConsumerGroupTest.test_dead_group_recovery.static_members=False"
"rptest.tests.topic_creation_test.RecreateTopicMetadataTest.test_recreated_topic_metadata_are_valid.replication_factor=3"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=ACKS_1.cleanup_policy=delete"
"rptest.tests.topic_creation_test.TopicRecreateTest.test_topic_recreation_while_producing.workload=IDEMPOTENT.cleanup_policy=delete"
"rptest.tests.topic_delete_test.TopicDeleteStressTest.stress_test"
"rptest.tests.cluster_quota_test.ClusterQuotaPartitionMutationTest.test_partition_throttle_mechanism"
"rptest.tests.group_membership_test.GroupMetricsTest.test_check_value"
"rptest.tests.internal_topic_protection_test.InternalTopicProtectionTest.kafka_nodelete_topics_test.client_type=kafka_tools"
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43570#018ceb04-a064-463c-93a3-aacb9a579cf3
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43570#018ceb04-a067-4572-9ffb-be4da8e68b3f
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43816#018d1469-c134-48b8-b6a5-5ef3cfea2aea
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f741-4d6f-a8c0-26bf8ee3b2d9
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f748-43f5-b899-c3f89c7502e9
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43816#018d147a-f73c-4046-b176-5e65714d3f18
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43839#018d189f-da1b-4e0f-8a60-add02f2f72e7
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43839#018d189f-da22-4872-b2c1-866c0b99c23a
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/44519#018d6007-f132-4c76-8eef-f41856e30d85
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/44811#018d83e2-1fee-4197-be5e-0f3e49aeae7d
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/45035#018da73a-3f41-4747-a677-c0a06a01846b
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/ci-repeat
/dt
/ci-repeat