Error installing TensorFlow Object Detection API in Google Colab

[eldivategar]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [x] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• [x] I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [x] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/packages/tf2/setup.py

2. Describe the bug

I am experiencing an issue when trying to install the TensorFlow Object Detection API on TensorFlow 2. I am following the installation steps as mentioned in the documentation, but I encounter the following error:

%cd models/research

!protoc object_detection/protos/*.proto --python_out=.

!cp object_detection/packages/tf2/setup.py .
!python -m pip install .

The specific error message I am receiving is:

Collecting pyyaml<5.4.0,>=5.1 (from tf-models-official>=2.5.1->object-detection==0.1) Using cached PyYAML-5.3.1.tar.gz (269 kB) error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. Preparing metadata (setup.py) ... error error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.

3. Steps to reproduce

• Clone the TensorFlow Object Detection API repository.
• Navigate to the models/research directory.
• Execute the provided commands for installation.
• Observe the error mentioned above.

4. Expected behavior

I expected the installation to complete successfully without any errors.

5. System information

• OS Platform and Distribution (Google Colab):
• Mobile device name if the issue happens on a mobile device: N/A
• TensorFlow installed from (source or binary): Use Google Colab environment
• TensorFlow version (use command below): 2.13.0 (as it's in a Google Colab environment)
• Python version: 3.10.12
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: Tesla T4, 13gb ram (as it's in a Google Colab environment)

[vankhoa01]

Yeah, I had same issue. Please let me know if you have any ideas to resolve this. Thank you so much

[adammasyk]

+1

[MarcoPrassel]

+1

[fbernaly]

+1

I have encountered the same issue when installing in colab

pip install "tf-models-official==2.13.*"

I am getting this:

Collecting tf-models-official==2.13.*
  Using cached tf_models_official-2.13.1-py2.py3-none-any.whl (2.6 MB)
Requirement already satisfied: Cython in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (3.0.2)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (9.4.0)
Requirement already satisfied: gin-config in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (0.5.0)
Requirement already satisfied: google-api-python-client>=1.6.7 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (2.84.0)
Collecting immutabledict (from tf-models-official==2.13.*)
  Using cached immutabledict-3.0.0-py3-none-any.whl (4.0 kB)
Requirement already satisfied: kaggle>=1.3.9 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (1.5.16)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (3.7.1)
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (1.23.5)
Requirement already satisfied: oauth2client in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (4.1.3)
Requirement already satisfied: opencv-python-headless in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (4.8.0.76)
Requirement already satisfied: pandas>=0.22.0 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (1.5.3)
Requirement already satisfied: psutil>=5.4.3 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (5.9.5)
Requirement already satisfied: py-cpuinfo>=3.3.0 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (9.0.0)
Requirement already satisfied: pycocotools in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.13.*) (2.0.7)
Collecting pyyaml<5.4.0,>=5.1 (from tf-models-official==2.13.*)
  Using cached PyYAML-5.3.1.tar.gz (269 kB)
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I want to learn how to fine tune BERT but so far I have not been able to pass the setup in these tutorial because installing that package is failing:

• https://www.tensorflow.org/text/tutorials/classify_text_with_bert
• https://www.tensorflow.org/tfmodels/nlp/fine_tune_bert

I found similar issues in other repos that use that package but so far no response:

• https://github.com/tensorflow/text/issues/1213
• https://github.com/tensorflow/text/issues/1147

Please, someone help!

[murtomanuel]

Im having the same issue

[laxmareddyp]

Hi @fbernaly,

We are aware that the recent Cython version update in Colab has caused compatibility issues with the PIP package, specifically requiring PyYAML>=6.0.0. Rest assured, we are actively addressing this issue, and you can expect to see the necessary changes in the upcoming release of the tf-models-official pip package. In the meantime, if you are using Colab, you have the option to clone the models repository and manually update the PyYAML version to >=6.0.0 in the requirement.txt file.

We hope this information proves helpful. Thank you for your understanding and patience.

[QuanNguyenAUT]

hi @laxmareddyp,

Appreciate your quick support. I mostly run on Colab, so this issue has been interrupted me too much. I did follow your suggestion which sets 'PyYAML>=6.0.0' in object_detection/packages/tf2/setup.py I see it can install object detection API properly. However, I can't export a model from a checkpoint by below error

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 35, in <module>
    from official.legacy.image_classification.efficientnet import efficientnet_model
ModuleNotFoundError: No module named 'official.legacy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/content/models/research/object_detection/exporter_main_v2.py", line 104, in <module>
    from object_detection import exporter_lib_v2
  File "/usr/local/lib/python3.10/dist-packages/object_detection/exporter_lib_v2.py", line 21, in <module>
    from object_detection.builders import model_builder
  File "/usr/local/lib/python3.10/dist-packages/object_detection/builders/model_builder.py", line 70, in <module>
    from object_detection.models import ssd_efficientnet_bifpn_feature_extractor as ssd_efficientnet_bifpn
  File "/usr/local/lib/python3.10/dist-packages/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 37, in <module>
    from official.vision.image_classification.efficientnet import efficientnet_model
  File "/usr/local/lib/python3.10/dist-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 37, in <module>
    from official.vision.image_classification import preprocessing
  File "/usr/local/lib/python3.10/dist-packages/official/vision/image_classification/preprocessing.py", line 25, in <module>
    from official.vision.image_classification import augment
  File "/usr/local/lib/python3.10/dist-packages/official/vision/image_classification/augment.py", line 31, in <module>
    from tensorflow.python.keras.layers.preprocessing import image_preprocessing as image_ops
ModuleNotFoundError: No module named 'tensorflow.python.keras.layers.preprocessing'

Could you help me to handle this bug? Thank you so much.

[laxmareddyp]

Hi @QuanNguyenAUT,

I believe the checkpoints you are using are possibly out of date

Thanks.

[QuanNguyenAUT]

Hi @laxmareddyp ,

I don't think so, because it is my current task. I run it everyday, so it is so strange. I did verify again with a succeed checkpoint, however it still failed.

Thanks.

[laxmareddyp]

Hi @QuanNguyenAUT ,

The error seems to be occurring in two projects: 'official' and 'research'. Which project did you load the model from? PIP install only supports the 'official' project, not the 'research' project.

Thanks.

[QuanNguyenAUT]

Hi @laxmareddyp ,

So far, I have worked on 'Research' project until yesterday. Could you show me how to work on TF2 object detection API based on Colab?

Thanks.

[vankhoa01]

Hi @laxmareddyp , I tried to clone the models repository and manually update the PyYAML version to >=6.0.0 , then I can pass this issue. But after that, I tried to test the model builder, then I got other issue like this: Do you have any solution to fix it ? Thank you so much

[eldivategar]

Hi @vankhoa01 ,

I tried uninstalling the default Cython from colab first and cloning the repo then installing the requirements. Once that was done, I tried running the test model builder. The details are below.

!pip uninstall Cython -y

!git clone https://github.com/tensorflow/models.git

%cd models/research

!protoc object_detection/protos/*.proto --python_out=.

!cp object_detection/packages/tf2/setup.py .
!python -m pip install .

installation results:

... Successfully built object-detection avro-python3 crcmod dill hdfs pyyaml seqeval docopt Installing collected packages: sentencepiece, docopt, crcmod, zstandard, tensorflow-model-optimization, tensorflow-io-gcs-filesystem, pyyaml, pyparsing, portalocker, orjson, objsize, immutabledict, fasteners, fastavro, dnspython, dill, Cython, colorama, avro-python3, tensorflow_io, sacrebleu, pymongo, hdfs, seqeval, lvis, apache-beam, tensorflow-text, tf-models-official, object-detection Attempting uninstall: tensorflow-io-gcs-filesystem Found existing installation: tensorflow-io-gcs-filesystem 0.33.0 Uninstalling tensorflow-io-gcs-filesystem-0.33.0: Successfully uninstalled tensorflow-io-gcs-filesystem-0.33.0 Attempting uninstall: pyyaml Found existing installation: PyYAML 6.0.1 Uninstalling PyYAML-6.0.1: Successfully uninstalled PyYAML-6.0.1 Attempting uninstall: pyparsing Found existing installation: pyparsing 3.1.1 Uninstalling pyparsing-3.1.1: Successfully uninstalled pyparsing-3.1.1 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. flax 0.7.2 requires PyYAML>=5.4.1, but you have pyyaml 5.3.1 which is incompatible. Successfully installed Cython-3.0.2 apache-beam-2.50.0 avro-python3-1.10.2 colorama-0.4.6 crcmod-1.7 dill-0.3.1.1 dnspython-2.4.2 docopt-0.6.2 fastavro-1.8.3 fasteners-0.18 hdfs-2.7.2 immutabledict-3.0.0 lvis-0.5.3 object-detection-0.1 objsize-0.6.1 orjson-3.9.7 portalocker-2.7.0 pymongo-4.5.0 pyparsing-2.4.7 pyyaml-5.3.1 sacrebleu-2.2.0 sentencepiece-0.1.99 seqeval-1.2.2 tensorflow-io-gcs-filesystem-0.34.0 tensorflow-model-optimization-0.7.5 tensorflow-text-2.13.0 tensorflow_io-0.34.0 tf-models-official-2.13.1 zstandard-0.21.0

Test model builder results:

[QuanNguyenAUT]

Hi @eldivategar , you are a legend.

It truly works at my end.

Appreciate your research. Thanks.

[MrDBC]

@fbernaly

im working on the same bert model as you are. It was working fine few days ago. and now it happened

[sineeli]

Hi @QuanNguyenAUT, @vankhoa01, @MrDBC, @fbernaly, @adammasyk

If you are still not using research folder or in production environments, I suggest to use the latest official repo which has proper release updates and will be useful to slowly move from research to official. There are already some reference notebooks which can help you start with official usage.

1. Image Classification
2. Object Detection
3. Semantic Segmentation
4. Instance Segmentation

Let us if you find these useful and further discuss if you have any doubts.

Thanks

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[cuoicungtui]

You can try this way, I got the error `# Clone the tensorflow models repository from GitHub !pip uninstall Cython -y # Temporary fix for "No module named 'object_detection'" error !git clone --depth 1 https://github.com/tensorflow/models

cd models/research/ protoc object_detection/protos/*.proto --python_out=.

Modify setup.py file to install the tf-models-official repository targeted at TF v2.8.0

'tf-models-official==2.8.0', s)

Install the Object Detection API

Need to do a temporary fix with PyYAML because Colab isn't able to install PyYAML v5.4.1

!pip install pyyaml==5.3 !pip install /content/models/research/

Need to downgrade to TF v2.8.0 due to Colab compatibility bug with TF v2.10 (as of 10/03/22)

!pip install tensorflow==2.8.0`