Trying to convert caffe flatten layer gives incorrect output shape

Before you open an issue, please make sure you have tried the following steps:

1. Make sure your environment is the same with (https://mace.readthedocs.io/en/latest/installation/env_requirement.html).
2. Have you ever read the document for your usage?
3. Check if your issue appears in HOW-TO-DEBUG or FAQ.
4. The form below must be filled.

---

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS 10.13.6
• NDK version(e.g., 15c): r16b
• GCC version(if compiling for host, e.g., 5.4.0):
• MACE version (Use the command: git describe --long --tags): v0.10.0
• Python version(2.7): 2.7
• Bazel version (e.g., 0.13.0): 0.20.0-homebrew

Model deploy file (*.yml)

......

Describe the problem

• Flatten layer is just returning the same shape as that of the input.
• Looking at the transformer https://github.com/XiaoMi/mace/blob/master/mace/python/tools/converter_tool/transformer.py#L1804 if we are flattening on the channel axis in NHWC format, the shape_tensor variable would end up as 0,0,0,-1
• The current reshape implementation https://github.com/XiaoMi/mace/blob/master/mace/ops/reshape.cc#L51 in this case would set the output to be of the same size as input. As a result, a concat followed by flatten layer is failing due to dimension mismatch.

To Reproduce

Steps to reproduce the problem:

• Any model where 2 or more layers with different dimensions are flattened and then concatenated would fail at runtime (tested on GPU)

Additional context

• No modifications to the code other than some header includes to compile with clang/libc++ toolchain.

The flatten op is converted from caffe's flatten layer, and caffe has the format NCHW. So if you are flattening on the channel axis, the shape_tensor should be [0, -1]? Is it convenient for you to provide your model and yml files so that I can debug it later? By the way, do you test this on CPU? Because of some format problem, reshape op should not be used on GPU, we will fix this soon.

@lydoc I was trying to flatten the input to a 4D tensor such that all dims except the channel are 1, which is why I was expecting [1,1,1,-1] instead of [0, -1] Regarding GPU usage, I wasn't aware that reshape wasn't yet supported fully on GPU since reading the .cc seemed like it is enabled. That would also explain why model validation fails if I use reshape on GPU :) I am looking forward to the GPU fix. Should I close this issue out till then, or just update the issue to request support for reshape on GPU?

If you want all dims except the channel are 1, why not use reshape? Beacuse from the caffe's definition( http://caffe.berkeleyvision.org/tutorial/layers/flatten.html ): The Flatten layer is a utility layer that flattens an input of shape n * c * h * w to a simple vector output of shape n * (chw), so I think if you flatten a 4D tensor on channel axis, the output should be 2D, right? As for Reshape op, because of the data format problem, we may only support it on CPU. Reshape op will be fallback to CPU when you use GPU to run the whole model.

I see, does it fallback to CPU automatically and run the other supported layers on the GPU? I tried using reshape with [1,1,1,-1]as the shape tensor, and now I am able to run inference properly, but the outputs don't match against caffe. I'll try to create a small reproducible version of the problem and share it here. Do you have a rough timeline for when reshape on GPU might be supported?

Yes, Reshape will fall back to CPU automatically. And I think it may not be supported on GPU recently since you can use cpu+gpu runtime.

I see.. and in case of cpu+gpu mode, would the full inference be run on CPU or just the reshape part?

Hmm, actually, when you use cpu mode, the whole model will be run on CPU, when you use gpu mode, ops which support gpu runtime will run on GPU, and ops which don't support gpu will fall back to cpu, when you use cpu+gpu mode, the model will run on CPU and GPU separately. By the way, we fixed some bugs just now, could you try the latest master branch?

Just tried with latest master, I now get an error at https://github.com/XiaoMi/mace/blob/master/mace/core/tensor.h#L251 I am still trying to run my model (which has a reshape layer) in GPU mode. If I understand correctly, the reshape should auto fallback to CPU and rest should run on GPU. Also, a separate question I had is whether the input to reshape layer transposed before switching between CPU and GPU mode (since the underlying layout is different)? Let me know if I can provide any additional info. Thanks!

Is it convenient for you to share your model so that I can reproduce this issue? For another question, the reshape layer's format has been transposed before fallback because of the different layout.