ModuleNotFoundError for packages belonging to the same API namespace

[vam-google]

Most likely it is caused by proto-plus incompatibility with nested packages API layout.

Affected APIs:

   //google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_test
    //google/cloud/dialogflow/cx/v3/cx_py_gapic_test
    //google/cloud/networkmanagement/v1beta1/networkmanagement_py_gapic_test
    //google/cloud/asset/v1p7beta1/asset_py_gapic_test
    //google/cloud/bigquery/dataexchange/v1beta1/dataexchange_py_gapic_test
    //google/cloud/gkehub/v1/gkehub_py_gapic_test
    //google/identity/accesscontextmanager/v1/accesscontextmanager_py_gapic_test
    //google/cloud/gkehub/v1alpha/gkehub_py_gapic_test
    //google/cloud/oslogin/v1/oslogin_py_gapic_test
    //google/cloud/oslogin/v1beta/oslogin_py_gapic_test
    //google/devtools/clouddebugger/v2/clouddebugger_py_gapic_test
    //google/devtools/testing/v1/testing_py_gapic_test
    //google/api/serviceusage/v1beta1/serviceusage_py_gapic_test
    //google/cloud/gkehub/v1beta/gkehub_py_gapic_test
    //google/cloud/networkmanagement/v1/networkmanagement_py_gapic_test

[noahdietz]

Adding //google/api/apikeys/v2:apikeys_py_gapic_test to the list.

[parthea]

Adding //google/api/apikeys/v2:apikeys_py_gapic_test to the list.

//google/api/apikeys/v2:apikeys_py_gapic_test passed in cl/468190444

[parthea]

For //google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_test and //google/cloud/dialogflow/cx/v3/cx_py_gapic_test, I was able to get the py_test rule to pass. The issue is that we're running pytest on the entire generated gapic, instead of only the tests directory. In this case, pytest is trying to run tests on the test_case.py which lives under types (not part of tests) https://github.com/googleapis/python-dialogflow-cx/blob/main/google/cloud/dialogflowcx_v3beta1/types/test_case.py

The actual tests that we want to run live in the tests folder so we only need to run pytest on that folder: https://github.com/googleapis/python-dialogflow-cx/tree/main/tests

Sample fix

(py39) partheniou@partheniou-vm-3:~/git/gapic-generator-python$ git diff rules_python_gapic/
diff --git a/rules_python_gapic/pytest.py b/rules_python_gapic/pytest.py
index 934fdfa6..9751a22a 100644
--- a/rules_python_gapic/pytest.py
+++ b/rules_python_gapic/pytest.py
@@ -4,8 +4,10 @@ import os
 
 
 if __name__ == '__main__':
+    # The generated gapic will be in a directory with suffix `srcjar.py`
+    src_dir = os.path.abspath(__file__).replace("pytest.py", "srcjar.py")
     sys.exit(pytest.main([
         '--disable-pytest-warnings',
         '--quiet',
-        os.path.dirname(os.path.abspath(__file__))
+        f"{src_dir}/tests"
     ]))

Local testing

(py39) partheniou@partheniou-vm-3:~/git/googleapis$ bazel run //google/cloud/dialogflow/cx/v3beta1:cx_py_gapic_test
INFO: Analyzed target //google/cloud/dialogflow/cx/v3beta1:cx_py_gapic_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //google/cloud/dialogflow/cx/v3beta1:cx_py_gapic_test up-to-date:
  bazel-bin/google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_pytest.py
  bazel-bin/google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_test.py
  bazel-bin/google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_test
INFO: Elapsed time: 0.222s, Critical Path: 0.01s
INFO: 2 processes: 2 internal.
INFO: Build completed successfully, 2 total actions
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh google/cloud/dialogflow/cx/v3beta1/cx_py_gapic_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //google/cloud/dialogflow/cx/v3beta1:cx_py_gapic_test
-----------------------------------------------------------------------------
........................................................................ [  1%]
........................................................................ [  3%]
........................................................................ [  5%]
........................................................................ [  6%]
........................................................................ [  8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
...............................                                          [100%]
4207 passed, 10 warnings in 31.12s

[parthea]

I'm going to remove the following APIs from the list as the specific issue is tracked in https://github.com/googleapis/gapic-generator-python/issues/1734

//google/cloud/dialogflow/cx/v3beta1:cx_py_gapic_test
//google/cloud/dialogflow/cx/v3:cx_py_gapic_test
//google/cloud/networkmanagement/v1beta1:networkmanagement_py_gapic_test
//google/cloud/networkmanagement/v1:networkmanagement_py_gapic_test

[parthea]

Removing //google/cloud/asset/v1p7beta1/asset_py_gapic_test from the list. The issue will be fixed in cl/555934590

[parthea]

Removing //google/api/serviceusage/v1beta1/serviceusage_py_gapic_test from the list. The issue will be fixed in cl/555937290

[parthea]

Removing //google/devtools/clouddebugger/v2/clouddebugger_py_gapic_test as the API is shut down: https://cloud.google.com/stackdriver/docs/deprecations/debugger-deprecation