PaddleSeg icon indicating copy to clipboard operation
PaddleSeg copied to clipboard

ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0

Open firaterdem opened this issue 2 years ago • 26 comments

问题确认 Search before asking

Bug描述 Describe the Bug

In Google Colab Quick Start code is giving the following error message at the training part. How can I solve it?

2023-07-26 12:56:35 [INFO] [TRAIN] epoch: 8, iter: 500/1000, loss: 0.0543, lr: 0.005369, batch_cost: 0.1380, reader_cost: 0.04589, ips: 28.9826 samples/sec | ETA 00:01:09 2023-07-26 12:56:35 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "/content/drive/MyDrive/SmartPol_PaddleSeg/PaddleSeg/tools/train.py", line 195, in <module> main(args) File "/content/drive/MyDrive/SmartPol_PaddleSeg/PaddleSeg/tools/train.py", line 170, in main train( File "/usr/local/lib/python3.10/dist-packages/paddleseg/core/train.py", line 315, in train mean_iou, acc, _, _, _ = evaluate( File "/usr/local/lib/python3.10/dist-packages/paddleseg/core/val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( File "/usr/local/lib/python3.10/dist-packages/paddleseg/utils/metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) File "/usr/local/lib/python3.10/dist-packages/paddle/tensor/manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ../paddle/phi/infermeta/multiary.cc:954)

复现环境 Environment

  • OS: Linux (Google Colab)
  • PaddlePaddle: 2.5.0
  • PaddleSeg: 2.8.0
  • Python: 3.10.6
  • CUDA: 11.8

Bug描述确认 Bug description confirmation

  • [X] 我确认已经提供了Bug复现步骤、代码改动说明、以及环境信息,确认问题是可以复现的。I confirm that the bug replication steps, code change instructions, and environment information have been provided, and the problem can be reproduced.

是否愿意提交PR? Are you willing to submit a PR?

  • [ ] 我愿意提交PR!I'd like to help by submitting a PR!

firaterdem avatar Jul 26 '23 17:07 firaterdem

You can attempt to fix the bug using this method, but finding the root cause of the issue will require additional effort.

Asthestarsfalll avatar Jul 27 '23 07:07 Asthestarsfalll

You can attempt to fix the bug using this method, but finding the root cause of the issue will require additional effort.

@Asthestarsfalll I corrected the related part of the code ("/usr/local/lib/python3.10/dist-packages/paddleseg/utils/metrics.py") as suggested:

pred_area.append(paddle.sum(paddle.cast(pred_i, "int64")).unsqueeze(0))

But it is still giving the same error.

firaterdem avatar Jul 28 '23 19:07 firaterdem

@firaterdem Can you share the Colab notebook so that I can investigate the reason?

Asthestarsfalll avatar Jul 29 '23 07:07 Asthestarsfalll

@firaterdem Can you share the Colab notebook so that I can investigate the reason?

@Asthestarsfalll sure, here is the notebook:

https://colab.research.google.com/drive/1Uu4eUHcDNA7LThE5KGexG5HdEZR-vB2t?usp=sharing

firaterdem avatar Jul 29 '23 08:07 firaterdem

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: 2023-07-29_16-42

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

Asthestarsfalll avatar Jul 29 '23 08:07 Asthestarsfalll

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: 2023-07-29_16-42

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

@Asthestarsfalll Yes, installing paddleseg in this way solved the problem. Thank you very much!

firaterdem avatar Jul 29 '23 19:07 firaterdem

以上回答已经充分解答了问题,如果有新的问题欢迎随时提交issue,或者在此条issue下继续回复~ 我们开启了飞桨套件的ISSUE攻关活动,欢迎感兴趣的开发者参加:https://github.com/PaddlePaddle/PaddleOCR/issues/10223

ToddBear avatar Aug 07 '23 06:08 ToddBear

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: 2023-07-29_16-42 You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

@Asthestarsfalll Yes, installing paddleseg in this way solved the problem. Thank you very much!

I have try this method, but not solve the problem. How can I solve this.

Jerry3062 avatar Aug 08 '23 10:08 Jerry3062

I have try this method, but not solve the problem. How can I solve this.

should give more details

Asthestarsfalll avatar Aug 08 '23 10:08 Asthestarsfalll

I have try this method, but not solve the problem. How can I solve this.

should give more details

I change my code from gitee to github and solve the problem, thanks.

Jerry3062 avatar Aug 09 '23 02:08 Jerry3062

我也遇到同樣的問題,已經按照上面的步驟無論是重新安裝或downgrad版本重跑數變,都是一樣的問題: ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

我是follow 官方教程:https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.8/docs/quick_start_cn.md

I0924 22:43:01.913477 15680 eager_method.cc:140] Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. 2023-09-24 22:43:01 [INFO] [TRAIN] epoch: 8, iter: 500/1000, loss: 0.0397, lr: 0.005369, batch_cost: 0.1381, reader_cost: 0.04371, ips: 28.9607 samples/sec | ETA 00:01:09 2023-09-24 22:43:01 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "C:\Users\a0922\Desktop\paddlepaddle\PaddleSeg\tools\train.py", line 195, in main(args) File "C:\Users\a0922\Desktop\paddlepaddle\PaddleSeg\tools\train.py", line 170, in main train( File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\train.py", line 315, in train mean_iou, acc, _, _, _ = evaluate( ^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\utils\metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddle\tensor\manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

jason660519 avatar Sep 24 '23 12:09 jason660519

@jason660519

是按照这种方式安装的吗,请确认安装成功了,而不是使用的pypi源当中的paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

Asthestarsfalll avatar Sep 24 '23 14:09 Asthestarsfalll

我是按照官方網站的教學: https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.8/docs/quick_start_cn.md 一步一步重複做了很多次, 每次在第三步驟執行: python tools/train.py --config configs/quick_start/pp_liteseg_optic_disc_512x512_1k.yml --save_interval 500 --do_eval --use_vdl --save_dir output 之後,都是一樣的結果。

我的硬體與安裝包如下: OS:Windows 10, AMD CPU, NVIDIA RTX 3090 24G GPU Compute Capability: 8.6 Driver API Version: 12.2, Runtime API Version: 12.0 cuDNN Version: 8.9. PaddlePaddle works well on 1 GPU. PaddlePaddle: 2.5.1 Paddleseg: 2.8.0

錯誤碼如下:

Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. (這一段的警告提示,應該無關訓練)

中止執行的時間點就是ve_interval 500的地方, iter設500 iter就停在500(epoch約跑到:8) iter設1000 iter就停在1000(epoch約跑到:17) Iter設10000 iter就停在10000(epoch約跑到:?? 截稿前,還在跑....)

epoch: 8, iter: 500/1000, loss: 0.0478, lr: 0.005369, batch_cost: 0.1344, reader_cost: 0.05471, ips: 29.7619 samples/sec | ETA 00:01:07 2023-09-25 16:04:53 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "C:\Users\a0922\Desktop\paddlepaddle\paddleseg\tools\train.py", line 195, in main(args) File "C:\Users\a0922\Desktop\paddlepaddle\paddleseg\tools\train.py", line 170, in main train( File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\train.py", line 315, in train mean_iou, acc, _, _, _ = evaluate( ^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\utils\metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddle\tensor\manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

jason660519 avatar Sep 25 '23 06:09 jason660519

@jason660519 不要使用pip安装,确保本地的包是通过python setup.py安装上的

Asthestarsfalll avatar Sep 25 '23 06:09 Asthestarsfalll

用 python setup.py 安裝不了,安裝到一半會跳出錯誤碼...

jason660519 avatar Sep 25 '23 06:09 jason660519

用 python setup.py 安裝不了,安裝到一半會跳出錯誤碼...

那就使用更低版本的paddle试试

Asthestarsfalll avatar Sep 25 '23 06:09 Asthestarsfalll

(paddle2-4) PS C:\Users\a0922\Desktop\paddle2-4> python -m pip install paddlepaddle-gpu==2.4.2.post117 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html Looking in links: https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html ERROR: Could not find a version that satisfies the requirement paddlepaddle-gpu==2.4.2.post117 (from versions: 2.5.1, 2.5.1.post102, 2.5.1.post112, 2.5.1.post116, 2.5.1.post117, 2.5.1.post120) ERROR: No matching distribution found for paddlepaddle-gpu==2.4.2.post117 (paddle2-4) PS C:\Users\a0922\Desktop\paddle2-4>

PS C:\Users\a0922\Desktop\paddle2-4> python -m pip install paddlepaddle-gpu==2.3.2.post116 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html Looking in links: https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html ERROR: Could not find a version that satisfies the requirement paddlepaddle-gpu==2.3.2.post116 (from versions: 2.5.1, 2.5.1.post102, 2.5.1.post112, 2.5.1.post116, 2.5.1.post117, 2.5.1.post120) ERROR: No matching distribution found for paddlepaddle-gpu==2.3.2.post116

即使降了一代版本,官網回應satisfy的paddle版本全都2.5.1版本以上 2.4.2 版本,官網系統回應找不到該版本。 2.3.2 版本,官網系統回應找不到該版本。 2.4.2 CPU的版本雖可以安裝...但PaddleSeg會裝不了。 已經一個禮拜了...真想放棄!

jason660519 avatar Sep 25 '23 07:09 jason660519

@jason660519 请给出本地安装paddleseg的报错信息

Asthestarsfalll avatar Sep 25 '23 08:09 Asthestarsfalll

方式安装paddleseg解决了这个问题。非常

解决了吗

yuzu16 avatar Jan 03 '24 03:01 yuzu16

@jason660519

是按照这种方式安装的吗,请确认安装成功了,而不是使用的pypi源使用的paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

这种方式安装的还是没有解决,

yuzu16 avatar Jan 03 '24 08:01 yuzu16

我已经尝试过这个方法,但没有解决问题。我该如何解决这个问题。

应该提供更多细节

我也是源代码下载 也是没解决问题 版本是2.8.0 是因为版本问题吗?

yuzu16 avatar Jan 03 '24 13:01 yuzu16

跟着官网的步骤,同样遇到了这个问题。

image image

FaberSober avatar Feb 12 '24 13:02 FaberSober

跟着官网的步骤,同样遇到了这个问题。

image image

使用更新的版本2.9呢?

Asthestarsfalll avatar Mar 05 '24 12:03 Asthestarsfalll

I have try this method, but not solve the problem. How can I solve this.

should give more details

I change my code from gitee to github and solve the problem, thanks.

where you change this code

pvinhd avatar Apr 02 '24 13:04 pvinhd

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: 2023-07-29_16-42

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

我按照您的方法重新装了paddleseg,还是报了这个错误,似乎不是metric的错误 Traceback (most recent call last): File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 197, in main() File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 193, in main run(FLAGS, cfg) File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 146, in run trainer.train(FLAGS.eval) File "/usr/local/lib/python3.10/dist-packages/ppdet/engine/trainer.py", line 539, in train outputs = model(data) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call return self.forward(*inputs, **kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/meta_arch.py", line 60, in forward out = self.get_loss() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/faster_rcnn.py", line 112, in get_ loss rpn_loss, bbox_loss = self._forward() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/faster_rcnn.py", line 80, in _forw ard rois, rois_num, rpn_loss = self.rpn_head(body_feats, self.inputs) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call return self.forward(*inputs, **kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 141, in fo rward rois, rois_num = self._gen_proposal(scores, deltas, anchors, inputs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 234, in _g en_proposal bs_rois_num_collect = paddle.concat(bs_rois_num_collect) File "/usr/local/lib/python3.10/dist-packages/paddle/tensor/manipulation.py", line 1263, in concat return _C_ops.concat(input, axis) ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0

messy2333 avatar Jul 01 '24 06:07 messy2333