Traceback info when converting model and build android apk

[jiahaolu97]

Hey , I am a newcomer to deep learning frameworks and I am very confused by some Traceback information. i couldn't convert a model , nor could i build an android apk.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
• NDK version(e.g., 15c): r16b
• GCC version(if compiling for host, e.g., 5.4.0): 5.5.0
• MACE version (Use the command: git describe --long --tags): v0.10.0-82-g19bf36b
• Python version(2.7): 3.6.5
• Bazel version (e.g., 0.13.0): 0.13.0

Describe the problem

1. bash tools/build-standalone-lib.sh succeed

2. python tools/converter.py convert --config=/path/to/mace-models/mobilenet-v2/mobilenet-v2.yml
   got Traceback information

3. ./build.sh dynamic [in mace/examples/android]
   I got same Traceback information as in 2 , and i didn't get a "build " folder under android folder.

To Reproduce

Steps to reproduce the problem:

1. cd /path/to/mace
2. python tools/converter.py convert --config_file=/path/to/your/mace-models/mobilenet-v2/mobilenet-v2.yml

3. cd mace/examples/android
4. ./build.sh dynamic

Error information / logs

Both actions got the similar Traceback information as following

LOGs
---------------------------------------------------
                Common Configuration               
---------------------------------------------------
|        key        |            value            |
===================================================
|       library_name|                 mobilenet-v2|
---------------------------------------------------
|        target_abis| ['armeabi-v7a', 'arm64-v8a']|
---------------------------------------------------
|        target_socs|                           []|
---------------------------------------------------
| model_graph_format|                         code|
---------------------------------------------------
|  model_data_format|                         code|
---------------------------------------------------

* Generate mace engine creator source
Generate mace engine creator source done!

**********************************************
          Convert mobilenet_v2 model          
**********************************************

INFO: Analysed target //mace/python/tools:converter (0 packages loaded).
INFO: Found 1 target...
Target //mace/python/tools:converter up-to-date:
  bazel-bin/mace/python/tools/converter
INFO: Elapsed time: 0.269s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action

Traceback (most recent call last):
  File "/home/royljh/mace/bazel-bin/mace/python/tools/converter.runfiles/mace/mace/python/tools/converter.py", line 23, in <module>
    from mace.proto import mace_pb2
  File "/home/royljh/mace/bazel-bin/mace/python/tools/converter.runfiles/mace/mace/proto/mace_pb2.py", line 6, in <module>
    from google.protobuf.internal import enum_type_wrapper
  File "/usr/local/lib/python3.6/dist-packages/google/protobuf/__init__.py", line 37, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2927, in <module>
    @_call_aside
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2913, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2952, in _initialize_master_working_set
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 956, in subscribe
    callback(dist)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2952, in <lambda>
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2515, in activate
    declare_namespace(pkg)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2097, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2047, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2066, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
Traceback (most recent call last):
  File "tools/converter.py", line 1300, in <module>
    flags.func(flags)
  File "tools/converter.py", line 774, in convert_func
    convert_model(configs, flags.cl_mem_type)
  File "tools/converter.py", line 704, in convert_model
    ",".join(model_config.get(YAMLKeyword.graph_optimize_options, [])))
  File "/home/royljh/mace/tools/sh_commands.py", line 532, in gen_model_code
    _fg=True)
  File "/usr/local/lib/python3.6/dist-packages/sh-1.12.14-py3.6.egg/sh.py", line 1413, in __call__
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/python bazel-bin/mace/python/tools/converter -u --platform=tensorflow --model_file=builds/downloads/d4e627ce1b15ed6f96aefffe315a8dc8.pb --weight_file= --model_checksum=369f9a5f38f3c15b4311c1c84c032ce868da9f371b5f78c13d3ea3c537389bb4 --weight_checksum= --input_node=input --input_data_formats=NHWC --output_node=MobilenetV2/Predictions/Reshape_1 --output_data_formats=NHWC --check_node= --runtime=cpu+gpu --template=mace/python/tools --model_tag=mobilenet_v2 --input_shape=1,224,224,3 --input_range= --output_shape=1,1001 --check_shape= --dsp_mode=0 --embed_model_data=True --winograd=0 --quantize=0 --quantize_range_file= --change_concat_ranges=0 --obfuscate=0 --output_dir=mace/codegen/models/mobilenet_v2 --model_graph_format=code --data_type=fp16_fp32 --graph_optimize_options= --cl_mem_type=image

  STDOUT:


  STDERR:

Additional context

Add any other context about the problem here, e.g., what you have modified about the code.

I have edited build-standalone-lib.sh and converter.py to get rid of the opencl acceleration part - I annotated line 47-59 and line 75-87 in build-standalone-lib.sh and I change all opencl=true to opencl=false - I annotated line 181-182 in converter.py

I have modify the gradle version

• in mace/examples/android/gradle/wrapper/gradle-wrapper.properties , I change the line distributionUrl=https://services.gradle.org/distributions/gradle-4.1-all.zip to gradle-5.0-all.zip
  And I change line 10 mace/examples/android/build.gradle from gradle:3.0.1 to gradle 3.1.1

I hope I have my problems clearly stated , and I really appreciate your efforts in this project .

[yejw5]

It seems that this crash can be fixed by update pip and setuptools: pip3 install -U setuptools pip

[jiahaolu97]

Thank you for your answering !! Now I can successfully convert models , and I can successfully
Build //mace/codegen:generated_models with ABI arm64-v8a However , I still failed to build an apk due to some android-sdk lisence problems

Error information is as follows :

Starting a Gradle Daemon (subsequent builds will be faster)

Configure project :app File /root/.android/repositories.cfg could not be loaded. Checking the license for package Android SDK Build-Tools 27.0.3 in /home/royljh/Software/android-sdk/android-sdk-linux/licenses Warning: License for package Android SDK Build-Tools 27.0.3 not accepted. Checking the license for package Android SDK Platform 26 in /home/royljh/Software/android-sdk/android-sdk-linux/licenses Warning: License for package Android SDK Platform 26 not accepted.

FAILURE: Build failed with an exception.

• What went wrong: A problem occurred configuring project ':app'.

Failed to install the following Android SDK packages as some licences have not been accepted. platforms;android-26 Android SDK Platform 26 build-tools;27.0.3 Android SDK Build-Tools 27.0.3 To build this project, accept the SDK license agreements and install the missing components using the Android Studio SDK Manager. Alternatively, to transfer the license agreements from one workstation to another, see http://d.android.com/r/studio-ui/export-licenses.html

Using Android SDK: /home/royljh/Software/android-sdk/android-sdk-linux

• Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

• Get more help at https://help.gradle.org

BUILD FAILED in 3m 42s

[jiahaolu97]

Should I close this issue and post the android-sdk problems into a new issue ?

[lee-bin]

How did you install the Android SDK? You can find how we install it in MACE docker around here.