Revert "Revert "ref: Deprecate TOPIC_PARTITION_COUNTS setting (#5868)""

[lynnagara]

This reverts commit ed0426b511ad37718edc7b20237bca09de74817f.

It was reverted due to sentry CI issue. We can bring this back once https://github.com/getsentry/sentry/pull/71191 is merged

[onkar]

@lynnagara should this PR be closed or are you planning to take up again?

[onewland]

@lynnagara should this PR be closed or are you planning to take up again?

This is still valid, it fixes the issue that caused INC-1165

[codecov[bot]]

:x: 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
235	1	234	6

View the top 1 failed test(s) by shortest run time

tests.clickhouse.test_native::test_concurrency_limit

Stack Traces | 0.013s run time

Traceback (most recent call last):
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.../site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File ".../local/lib/python3.11....../site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File ".../local/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File ".../local/lib/python3.11....../site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11.../site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
            ^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11.../site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
            ^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File ".../local/lib/python3.11....../site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File ".../local/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../tests/clickhouse/test_native.py", line 82, in test_concurrency_limit
    assert connection.execute.call_count == 2, "Expected two attempts"
AssertionError: Expected two attempts
assert 1 == 2
 +  where 1 = <Mock name='mock.execute' id='140712714698064'>.call_count
 +    where <Mock name='mock.execute' id='140712714698064'> = <Mock id='140713342606608'>.execute

To view more test analytics, go to the Test Analytics Dashboard _{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[onewland]

To repro locally, run:

docker run --rm -p 127.0.0.1:1218:1218 --network devservices --name snuba-snuba-1 -e DEFAULT_BROKERS=kafka-kafka-1:9092 -e REDIS_HOST=redis-redis-1 -e SNUBA_SETTINGS=docker -e CLICKHOUSE_HOST=clickhouse c22b79361144 # this hash is the result of `docker build .` in snuba

you'll see this log in snuba when running the failing Sentry test:

2025-05-22 18:13:14,120 partitions_number kafka config: {'bootstrap.servers': 'kafka-kafka-1:9092', 'security.protocol': 'plaintext', 'ssl.ca.location': '', 'ssl.certificate.location': '', 'ssl.key.location': '', 'log_level': 6}
%3|1747937594.126|FAIL|rdkafka#producer-2| [thrd:127.0.0.1:9092/1001]: 127.0.0.1:9092/1001: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT)
%3|1747937594.126|FAIL|rdkafka#producer-2| [thrd:127.0.0.1:9092/1001]: 127.0.0.1:9092/1001: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)
...
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.11/site-packages/confluent_kafka/admin/__init__.py", line 264, in _make_futmap_result_from_list
    results = f.result()
              ^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
cimpl.KafkaException: KafkaError{code=_TIMED_OUT,val=-185,str="Timed out waiting for controller"}
172.18.0.1 - - [22/May/2025:18:13:14 +0000] "POST /events/events/subscriptions HTTP/1.1" 500 255 "-" "python-urllib3/2.2.2"

as the new code path tries to connect to 127.0.0.1:9092 but needs to connect to kafka:9092 or kafka-kafka-1:9092

[github-actions[bot]]

This PR has a migration; here is the generated SQL for ./snuba/migrations/groups.py

[getsentry-bot]

PR reverted: e8188c4b7feab59ab0d3d618ca05fc409eef4f61