mmocr icon indicating copy to clipboard operation
mmocr copied to clipboard
verified publisher icon open-mmlab

totaltext_converter.py

Open ConflictPeople opened this issue 3 years ago • 8 comments

转换totaltext数据集后，用于DBNet训练时出现错误：shapely.errors.TopologicalError: The operation 'GEOSIntersection_r' could not be performed. Likely cause is invalidity of the geometry <shapely.geometry.polygon.Polygon object at 0x7faa4c8a5050>

我认为可能是代码转换文件的问题：有些标注信息没有处理好导致的，因为有些样本是可以训练的，但是如果遇到不能处理的样本就会弹出这个错误？有大佬能帮忙解决一下，或者分享一下数据集吗？在线关注，谢谢！（同时，CTW1500训练1200个epoch的性能跟论文相比还有些差距，但损失已经不会下降了）

ConflictPeople avatar Jul 14 '22 01:07 ConflictPeople

2022-07-14 09:52:13,865 - mmocr - INFO - workflow: [('train', 1)], max: 1200 epochs 2022-07-14 09:52:13,865 - mmocr - INFO - Checkpoints will be saved to /home/wqb/WQB/mmocr/tools/totaltext_train_data by HardDiskBackend. 2022-07-14 09:52:19,986 - mmocr - INFO - Epoch [1][5/210] lr: 7.000e-03, eta: 3 days, 12:43:45, time: 1.210, data_time: 0.532, memory: 6249, loss_prob: 2.2807, loss_db: 0.3423, loss_thr: 0.8210, loss: 3.4440 2022-07-14 09:52:23,346 - mmocr - INFO - Epoch [1][10/210] lr: 7.000e-03, eta: 2 days, 17:52:12, time: 0.672, data_time: 0.097, memory: 6249, loss_prob: 2.1272, loss_db: 0.3664, loss_thr: 0.7378, loss: 3.2313 2022-07-14 09:52:26,414 - mmocr - INFO - Epoch [1][15/210] lr: 7.000e-03, eta: 2 days, 10:14:09, time: 0.614, data_time: 0.041, memory: 6249, loss_prob: 1.9986, loss_db: 0.3522, loss_thr: 0.7325, loss: 3.0833 2022-07-14 09:52:29,492 - mmocr - INFO - Epoch [1][20/210] lr: 7.000e-03, eta: 2 days, 6:26:59, time: 0.616, data_time: 0.047, memory: 6249, loss_prob: 2.1272, loss_db: 0.3776, loss_thr: 0.7285, loss: 3.2333 2022-07-14 09:52:32,655 - mmocr - INFO - Epoch [1][25/210] lr: 7.000e-03, eta: 2 days, 4:24:45, time: 0.632, data_time: 0.042, memory: 6249, loss_prob: 2.1731, loss_db: 0.3632, loss_thr: 0.7202, loss: 3.2565 2022-07-14 09:52:35,835 - mmocr - INFO - Epoch [1][30/210] lr: 7.000e-03, eta: 2 days, 3:05:15, time: 0.635, data_time: 0.048, memory: 6249, loss_prob: 2.0725, loss_db: 0.3612, loss_thr: 0.7560, loss: 3.1896 2022-07-14 09:52:39,009 - mmocr - INFO - Epoch [1][35/210] lr: 7.000e-03, eta: 2 days, 2:07:50, time: 0.634, data_time: 0.036, memory: 6249, loss_prob: 2.3256, loss_db: 0.4081, loss_thr: 0.8106, loss: 3.5443 2022-07-14 09:52:42,153 - mmocr - INFO - Epoch [1][40/210] lr: 7.000e-03, eta: 2 days, 1:22:21, time: 0.630, data_time: 0.049, memory: 6249, loss_prob: 1.9485, loss_db: 0.3328, loss_thr: 0.7281, loss: 3.0093 2022-07-14 09:52:45,346 - mmocr - INFO - Epoch [1][45/210] lr: 7.000e-03, eta: 2 days, 0:51:15, time: 0.639, data_time: 0.036, memory: 6249, loss_prob: 2.0033, loss_db: 0.3410, loss_thr: 0.7560, loss: 3.1003 2022-07-14 09:52:48,608 - mmocr - INFO - Epoch [1][50/210] lr: 7.000e-03, eta: 2 days, 0:32:00, time: 0.652, data_time: 0.044, memory: 6249, loss_prob: 1.9819, loss_db: 0.3399, loss_thr: 0.6499, loss: 2.9716 2022-07-14 09:52:51,764 - mmocr - INFO - Epoch [1][55/210] lr: 7.000e-03, eta: 2 days, 0:08:03, time: 0.631, data_time: 0.048, memory: 6249, loss_prob: 2.0467, loss_db: 0.3615, loss_thr: 0.7914, loss: 3.1995 2022-07-14 09:52:54,987 - mmocr - INFO - Epoch [1][60/210] lr: 7.000e-03, eta: 1 day, 23:52:45, time: 0.644, data_time: 0.047, memory: 6249, loss_prob: 2.1328, loss_db: 0.3525, loss_thr: 0.8070, loss: 3.2923 2022-07-14 09:52:58,124 - mmocr - INFO - Epoch [1][65/210] lr: 7.000e-03, eta: 1 day, 23:34:35, time: 0.628, data_time: 0.048, memory: 6249, loss_prob: 1.7559, loss_db: 0.2880, loss_thr: 0.7385, loss: 2.7824 TopologyException: Input geom 0 is invalid: Self-intersection at 201.36643885580986 748.04435560663774 2022-07-14 09:53:01,224 - mmocr - INFO - Epoch [1][70/210] lr: 7.000e-03, eta: 1 day, 23:16:35, time: 0.620, data_time: 0.043, memory: 6249, loss_prob: 1.5067, loss_db: 0.2551, loss_thr: 0.6357, loss: 2.3975 /home/wqb/WQB/mmocr/mmocr/datasets/pipelines/textdet_targets/base_textdet_targets.py:125: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. shrunk = np.array(pco.Execute(-distance)) Traceback (most recent call last): File "/home/wqb/WQB/mmocr/tools/train.py", line 271, in main() File "/home/wqb/WQB/mmocr/tools/train.py", line 267, in main meta=meta) File "/home/wqb/WQB/mmocr/mmocr/apis/train.py", line 166, in train_detector runner.run(data_loaders, cfg.workflow) # 开始运行程序 File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 130, in run epoch_runner(data_loaders[i], **kwargs) # 进行一个 epoch 的训练 File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 47, in train for i, data_batch in enumerate(self.data_loader): # 每次取出6张图片 File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 530, in next data = self._next_data() File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1224, in _next_data return self._process_data(data) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1250, in _process_data data.reraise() File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/_utils.py", line 457, in reraise raise exception shapely.errors.TopologicalError: Caught TopologicalError in DataLoader worker process 1. Original Traceback (most recent call last): File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/utils/data/dataset.py", line 416, in getitem return self.datasets[dataset_idx][sample_idx] File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmdet/datasets/custom.py", line 218, in getitem data = self.prepare_train_img(idx) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmdet/datasets/custom.py", line 241, in prepare_train_img return self.pipeline(results) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmdet/datasets/pipelines/compose.py", line 41, in call data = t(data) File "/home/wqb/WQB/mmocr/mmocr/datasets/pipelines/dbnet_transforms.py", line 83, in call self.may_augment_annotation(aug, shape, target_shape, results) File "/home/wqb/WQB/mmocr/mmocr/datasets/pipelines/dbnet_transforms.py", line 94, in may_augment_annotation masks = self.may_augment_poly(aug, shape, results[key]) File "/home/wqb/WQB/mmocr/mmocr/datasets/pipelines/dbnet_transforms.py", line 135, in may_augment_poly shape=img_shape)])[0].clip_out_of_image() File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/imgaug/augmentables/polys.py", line 1806, in clip_out_of_image return self.copy().clip_out_of_image() File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/imgaug/augmentables/polys.py", line 1779, in clip_out_of_image for poly in self.polygons File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/imgaug/augmentables/polys.py", line 1780, in for poly_clipped in poly.clip_out_of_image(self.shape)] File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/imgaug/augmentables/polys.py", line 591, in clip_out_of_image multipoly_inter_shapely = poly_shapely.intersection(poly_image) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/shapely/geometry/base.py", line 689, in intersection return geom_factory(self.impl['intersection'](self, other)) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/shapely/topology.py", line 72, in call self._check_topology(err, this, other) File "/home/wqb/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/shapely/topology.py", line 40, in _check_topology self.fn.name, repr(geom))) shapely.errors.TopologicalError: The operation 'GEOSIntersection_r' could not be performed. Likely cause is invalidity of the geometry <shapely.geometry.polygon.Polygon object at 0x7faa4c8a5050>

Process finished with exit code 1

ghost avatar Jul 14 '22 02:07 ghost

I have already try to add the .buffer(0.001) under the instantiation of Poly, but can't solve this problem...

ghost avatar Jul 14 '22 02:07 ghost

Hi @ConflictPeople Please share your email address, and I'll email it to you.

Mountchicken avatar Jul 14 '22 02:07 Mountchicken

Hi @ConflictPeople Please share your email address, and I'll email it to you. Thank you! My email is [email protected]

ghost avatar Jul 14 '22 02:07 ghost

@ConflictPeople Done

Mountchicken avatar Jul 14 '22 03:07 Mountchicken

@ConflictPeople Done

Hi, can you share the TotalText data set? I already have CTW, thank you.

ghost avatar Jul 14 '22 04:07 ghost

There are some invalid polygons in totaltext's annotation, which are not permitted in ImgAug by default. Try turn clip_invalid_poly off in the config:

dict(
        type='ImgAug',
        args=[['Fliplr', 0.5],
              dict(cls='Affine', rotate=[-10, 10]), ['Resize', [0.5, 3.0]]],
        clip_invalid_ploys=False)

gaotongxiao avatar Jul 16 '22 13:07 gaotongxiao

There are some invalid polygons in totaltext's annotation, which are not permitted in ImgAug by default. Try turn clip_invalid_poly off in the config:

dict(
        type='ImgAug',
        args=[['Fliplr', 0.5],
              dict(cls='Affine', rotate=[-10, 10]), ['Resize', [0.5, 3.0]]],
        clip_invalid_ploys=False)

Thank you for your help. It has been solved.

ghost avatar Jul 16 '22 13:07 ghost

您好，请问您是怎么解决这个问题的

catmiao108 avatar Feb 27 '23 14:02 catmiao108

In the latest 1.x branch, it can be elegantly solved by adding FixInvalidPolygon transform after LoadOCRAnnotations in both train and test pipeline. Here are examples:

https://github.com/open-mmlab/mmocr/blob/27b6a68586b9a040678fe083bcf60662ae1b9261/configs/textdet/dbnet/dbnet_resnet18_fpnc_1200e_totaltext.py#L21

https://github.com/open-mmlab/mmocr/blob/27b6a68586b9a040678fe083bcf60662ae1b9261/configs/textdet/dbnet/dbnet_resnet18_fpnc_1200e_totaltext.py#L51

gaotongxiao avatar Feb 27 '23 14:02 gaotongxiao