tvalentyn

[Task]: Document compatibility asipirations & test coverage policies for optional Beam dependencies

1

### What needs to happen? Certain aspects of Beam functionality depend on actively evolving libraries, for example RunInference model handlers might require dependencies like PyTorch or Tensorflow. We should document...

### What happened? ``` > Task :sdks:python:test-suites:direct:py38:tensorflowInferenceTest FAILED File "", line 671, in _load_unlocked File "/runner/_work/beam/beam/build/gradleenv/1398941892/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 186, in exec_module exec(co, module.__dict__) File "/runner/_work/beam/beam/build/gradleenv/1398941892/lib/python3.8/site-packages/typeguard/__init__.py", line 21, in from ._importhook import...

[Failing Test]: PreCommit YAML Xlang Direct fails on GHA

1

### What happened? Sample run: https://github.com/apache/beam/actions/runs/9087570294/job/24975542386 Stacktrace: INFO apache_beam.utils.subprocess_server:subprocess_server.py:213 Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory ``` INFO root:external.py:1017 Starting a JAR-based expansion service from JAR /runner/_work/beam/beam/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.57.0-SNAPSHOT.jar INFO apache_beam.utils.subprocess_server:subprocess_server.py:203 Starting service...

[Failing Test]: Beam ML unit tests fail with Keras 3.3.3 and TF 2.16.1

2

### What happened? Sample failure: ``` $ pytest apache_beam/ml/inference/tensorflow_inference_test.py::TFRunInferenceTest::test_predict_numpy_with_batch_size ================================================================================================================================== test session starts =================================================================================================================================== platform linux -- Python 3.9.18, pytest-7.4.4, pluggy-1.5.0 rootdir: /home/valentyn/projects/beam/beam/beam/sdks/python configfile: pytest.ini plugins: xdist-3.6.1, requests-mock-1.12.1, timeout-2.3.1, hypothesis-6.102.1...

ATM this PR is in the phase of scoping what's not working #29149 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and...

[Failing Test]: Onnx inference unit tests are failing.

1

### What happened? Due to test configuration issues, the tests were not part of tox unit test suite. Enabling them causes failures: ``` FAILED apache_beam/ml/inference/onnx_inference_test.py::OnnxPytorchRunInferencePipelineTest::test_invalid_input_type - AttributeError: 'TestOnnxModelHandler' object has...

### What happened? With `xgboost==2.0.3` the apache_beam/ml/inference/xgboost_inference_test.py test fails with `AttributeError: 'XGBClassifier' object has no attribute 'inference_calls`. ### Issue Failure Failure: Test is continually failing ### Issue Priority Priority: 2...

https://github.com/apache/beam/pull/30639 added an codepath to fail early when SDK emits a large element. This PR adds a warning when we detect large elements that can be problematic, but still within...

### What happened? Installing apache-beam into a clean environment now shows following logs: ``` pip install --no-cache-dir --pre apache-beam[gcp]==2.56.0 ... Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub=2.1.0->apache-beam[gcp]==2.56.0) Downloading grpcio_status-1.64.1-py3-none-any.whl.metadata (1.1 kB) Downloading grpcio_status-1.64.0-py3-none-any.whl.metadata...