DAVAR-Lab-OCR
DAVAR-Lab-OCR copied to clipboard
hikopensource
LGPMA Training Error
Using bash dist_train.sh I get the below error: Any clue what the root cause of the error is?
CUDA_HOME: /usr/local/cuda-11.0 NVCC: Build cuda_11.0_bu.TC445_37.28845127_0 GCC: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10.0.2) PyTorch: 1.7.1+cu110 PyTorch compiling details: PyTorch built with:
- GCC 7.3
- CuDNN 8.0.5
- Magma 2.5.2
TorchVision: 0.8.2+cu110 OpenCV: 4.5.5 MMCV: 1.3.4 MMCV Compiler: GCC 8.5 MMCV CUDA Compiler: 11.0 DAVAROCR: 0.5.0+03a4c9c
2022-06-06 14:22:04,247 - davarocr - INFO - Distributed training: True 2022-06-06 14:22:04,565 - davarocr - INFO - Config: model = dict( type='LGPMA', pretrained= 'DAVAR-Lab-OCR/demo/table_recognition/lgpma/ckg/resnet50-19c8e357.pth', backbone=dict( type='ResNet', depth=50, num_stages=4, out_indices=(0, 1, 2, 3), frozen_stages=1, style='pytorch'), neck=dict( type='FPN', in_channels=[256, 512, 1024, 2048], out_channels=256, num_outs=5), rpn_head=dict( type='RPNHead', in_channels=256, feat_channels=256, anchor_generator=dict( type='AnchorGenerator', scales=[4, 8, 16], ratios=[0.05, 0.1, 0.2, 0.5, 1.0, 2.0], strides=[4, 8, 16, 32, 64]), bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[1.0, 1.0, 1.0, 1.0]), loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0), loss_bbox=dict( type='SmoothL1Loss', beta=0.1111111111111111, loss_weight=1.0)), roi_head=dict( type='LGPMARoIHead', bbox_roi_extractor=dict( type='SingleRoIExtractor', roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0), out_channels=256, featmap_strides=[4, 8, 16, 32]), bbox_head=dict( type='Shared2FCBBoxHead', in_channels=256, fc_out_channels=1024, roi_feat_size=7, num_classes=2, bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[0.1, 0.1, 0.2, 0.2]), reg_class_agnostic=False, loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0), loss_bbox=dict(type='SmoothL1Loss', beta=1.0, loss_weight=1.0)), mask_roi_extractor=dict( type='SingleRoIExtractor', roi_layer=dict(type='RoIAlign', output_size=14, sampling_ratio=0), out_channels=256, featmap_strides=[4, 8, 16, 32]), mask_head=dict( type='LPMAMaskHead', num_convs=4, in_channels=256, conv_out_channels=256, num_classes=2, loss_mask=dict( type='CrossEntropyLoss', use_mask=True, loss_weight=1.0), loss_lpma=dict(type='L1Loss', loss_weight=1.0))), global_seg_head=dict( type='GPMAMaskHead', in_channels=256, conv_out_channels=256, num_classes=1, loss_mask=dict(type='DiceLoss', loss_weight=1), loss_reg=dict( type='SmoothL1Loss', beta=0.1, loss_weight=0.01, reduction='sum')), train_cfg=dict( rpn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.7, neg_iou_thr=0.3, min_pos_iou=0.3, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=256, pos_fraction=0.5, neg_pos_ub=-1, add_gt_as_proposals=False), allowed_border=0, pos_weight=-1, debug=False), rpn_proposal=dict( nms_pre=2000, max_per_img=2000, nms_post=2000, nms=dict(type='nms', iou_threshold=0.5), min_bbox_size=0), rcnn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.5, neg_iou_thr=0.5, min_pos_iou=0.5, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=512, pos_fraction=0.25, neg_pos_ub=-1, add_gt_as_proposals=True), mask_size=28, pos_weight=-1, debug=False)), test_cfg=dict( rpn=dict( nms_pre=2000, nms_post=2000, max_per_img=2000, nms=dict(type='nms', iou_threshold=0.5), min_bbox_size=0), rcnn=dict( score_thr=0.05, nms=dict(type='nms', iou_threshold=0.1), max_per_img=1000, mask_thr_binary=0.5), postprocess=dict(type='PostLGPMA', refine_bboxes=False))) train_cfg = None test_cfg = None dataset_type = 'DavarCustomDataset' data_root = '' img_norm_cfg = dict( mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True) train_pipeline = [ dict(type='DavarLoadImageFromFile'), dict( type='DavarLoadTableAnnotations', with_bbox=True, with_enlarge_bbox=True, with_label=True, with_poly_mask=True, with_empty_bbox=True), dict( type='DavarResize', img_scale=[(360, 480), (960, 1080)], keep_ratio=True, multiscale_mode='range'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='GPMADataGeneration'), dict(type='DavarDefaultFormatBundle'), dict( type='DavarCollect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks', 'gt_semantic_seg']) ] val_pipeline = [ dict(type='DavarLoadImageFromFile'), dict( type='DavarLoadTableAnnotations', with_bbox=True, with_enlarge_bbox=True, with_label=True, with_poly_mask=True, with_empty_bbox=True), dict( type='MultiScaleFlipAug', scale_factor=1.5, flip=False, transforms=[ dict(type='DavarResize', keep_ratio=True), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='DavarDefaultFormatBundle'), dict( type='DavarCollect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ]) ] test_pipeline = [ dict(type='DavarLoadImageFromFile'), dict( type='MultiScaleFlipAug', scale_factor=1.5, flip=False, transforms=[ dict(type='DavarResize', keep_ratio=True), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='DavarDefaultFormatBundle'), dict(type='DavarCollect', keys=['img']) ]) ] data = dict( samples_per_gpu=3, workers_per_gpu=1, train=dict( type='DavarCustomDataset', ann_file= '/data/Davar/datalist_train_detection_pubtabnet.json', img_prefix='/data/pubtabnet', pipeline=[ dict(type='DavarLoadImageFromFile'), dict( type='DavarLoadTableAnnotations', with_bbox=True, with_enlarge_bbox=True, with_label=True, with_poly_mask=True, with_empty_bbox=True), dict( type='DavarResize', img_scale=[(360, 480), (960, 1080)], keep_ratio=True, multiscale_mode='range'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='GPMADataGeneration'), dict(type='DavarDefaultFormatBundle'), dict( type='DavarCollect', keys=[ 'img', 'gt_bboxes', 'gt_labels', 'gt_masks', 'gt_semantic_seg' ]) ]), val=dict( type='DavarCustomDataset', ann_file='/data/pubtabnet/PubTabNet_2.0.0.jsonl', img_prefix='/data/pubtabnet/val', pipeline=[ dict(type='DavarLoadImageFromFile'), dict( type='DavarLoadTableAnnotations', with_bbox=True, with_enlarge_bbox=True, with_label=True, with_poly_mask=True, with_empty_bbox=True), dict( type='MultiScaleFlipAug', scale_factor=1.5, flip=False, transforms=[ dict(type='DavarResize', keep_ratio=True), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='DavarDefaultFormatBundle'), dict( type='DavarCollect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ]) ]), test=dict( type='DavarCustomDataset', ann_file='/data/pubtabnet/PubTabNet_2.0.0.jsonl', img_prefix='/data/pubtabnet/val', pipeline=[ dict(type='DavarLoadImageFromFile'), dict( type='MultiScaleFlipAug', scale_factor=1.5, flip=False, transforms=[ dict(type='DavarResize', keep_ratio=True), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size_divisor=32), dict(type='DavarDefaultFormatBundle'), dict(type='DavarCollect', keys=['img']) ]) ], samples_per_gpu=1)) optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001) optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2)) lr_config = dict( policy='step', warmup='linear', warmup_iters=1000, warmup_ratio=0.3333333333333333, step=[6, 10]) runner = dict(type='EpochBasedRunner', max_epochs=12) checkpoint_config = dict( interval=1, filename_tmpl= 'DAVAR-Lab-OCR/demo/table_recognition/lgpma/configs/maskrcnn-lgpma-pub-e12-pub.pth' ) log_config = dict(interval=10, hooks=[dict(type='TextLoggerHook')]) dist_params = dict(backend='nccl') log_level = 'INFO' work_dir = '/DAVAR-Lab-OCR/demo/table_recognition/lgpma/configs' load_from = None resume_from = None workflow = [('train', 1)] gpu_ids = range(0, 8)
2022-06-06 14:22:04,844 - mmdet - INFO - load model from: DAVAR-Lab-OCR/demo/table_recognition/lgpma/ckg/resnet50-19c8e357.pth 2022-06-06 14:22:04,844 - mmdet - INFO - Use load_from_local loader 2022-06-06 14:22:04,844 - mmdet - INFO - Use load_from_local loader 2022-06-06 14:22:04,965 - mmdet - WARNING - The model and loaded state dict do not match exactly
unexpected key in source state_dict: fc.weight, fc.bias
Traceback (most recent call last): File "anaconda3/envs/davar/lib/python3.9/site-packages/mmcv/utils/registry.py", line 51, in build_from_cfg return obj_cls(**args) File "DAVAR-Lab-OCR/davarocr/davar_table/datasets/pipelines/gpma_data.py", line 50, in init lib = ctl.load_library(lib_name, lib_dir) File "anaconda3/envs/davar/lib/python3.9/site-packages/numpy/ctypeslib.py", line 163, in load_library raise OSError("no file with expected extension") OSError: no file with expected extension
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "anaconda3/envs/davar/lib/python3.9/site-packages/mmcv/utils/registry.py", line 51, in build_from_cfg return obj_cls(**args) File 'DAVAR-Lab-OCR/davarocr/davar_common/datasets/davar_custom.py", line 135, in init self.pipeline = Compose(pipeline) File "anaconda3/envs/davar/lib/python3.9/site-packages/mmdet/datasets/pipelines/compose.py", line 22, in init transform = build_from_cfg(transform, PIPELINES) File "anaconda3/envs/davar/lib/python3.9/site-packages/mmcv/utils/registry.py", line 54, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') OSError: GPMADataGeneration: no file with expected extension
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File"DAVAR-Lab-OCR//tools/train.py", line 253, in
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "anaconda3/envs/davar/lib/python3.9/site-packages/mmcv/utils/registry.py", line 51, in build_from_cfg return obj_cls(**args) File "DAVAR-Lab-OCR/davarocr/davar_common/datasets/davar_custom.py", line 135, in init self.pipeline = Compose(pipeline) File "anaconda3/envs/davar/lib/python3.9/site-packages/mmdet/datasets/pipelines/compose.py", line 22, in init transform = build_from_cfg(transform, PIPELINES) File "anaconda3/envs/davar/lib/python3.9/site-packages/mmcv/utils/registry.py", line 54, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') OSError: GPMADataGeneration: no file with expected extension
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DAVAR-Lab-OCR//tools/train.py", line 253, in
Traceback (most recent call last):
File "DAVAR-Lab-OCR//tools/train.py", line 253, in
The problem comes from 'OSError: GPMADataGeneration: no file with expected extension'. You should run the setup.sh to generate the ".so" file. If it is correctly excuted, there will be a ".so" file in "davarocr\davar_table\datasets\pipelines\lib".
bash setup.sh throws this error:
compilation terminated.
Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_det/core/post_processing/lib/tp_points_generate.cpp:21:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc'
to the PKG_CONFIG_PATH environment variable
Package 'opencv', required by 'virtual:world', not found
./davarocr/davar_det/core/post_processing/lib/east_postprocess.cpp:11:10: fatal error: opencv2/opencv.hpp: No such file or directory
#include <opencv2/opencv.hpp>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
Package opencv was not found in the pkg-config search path.
Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_spotting/core/post_processing/lib/bfs_search.cpp:13:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc'
to the PKG_CONFIG_PATH environment variable
Package 'opencv', required by 'virtual:world', not found
./davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp:24:10: fatal error: opencv2/opencv.hpp: No such file or directory
#include <opencv2/opencv.hpp>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
11
mkdir: cannot create directory ‘build’: File exists
setup.sh: line 60: cmake: command not found
make: *** No targets specified and no makefile found. Stop.
Could not find libwarpctc.so in ../build.
Build warp-ctc and set WARP_CTC_PATH to the location of libwarpctc.so (default is '../build')
It is because the algorithm of EAST and Warpctc require other dependencies. It won’t affect the model of LGPMA, You can simply ignore them by comment the related code in davarocr if any error occurs.
发件人: Yazdan @.> 发送时间: 2022年6月8日 2:13 收件人: hikopensource/DAVAR-Lab-OCR @.> 抄送: 乔梁6 @.>; Comment @.> 主题: Re: [hikopensource/DAVAR-Lab-OCR] LGPMA Training Error (Issue #89)
bash setup.sh throws this error: compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_det/core/post_processing/lib/tp_points_generate.cpp:21:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_det/core/post_processing/lib/east_postprocess.cpp:11:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_spotting/core/post_processing/lib/bfs_search.cpp:13:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. Package opencv was not found in the pkg-config search path. Perhaps you should add the directory containing opencv.pc' to the PKG_CONFIG_PATH environment variable Package 'opencv', required by 'virtual:world', not found ./davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp:24:10: fatal error: opencv2/opencv.hpp: No such file or directory #include <opencv2/opencv.hpp> ^~~~~~~~~~~~~~~~~~~~ compilation terminated. 11 mkdir: cannot create directory ‘build’: File exists setup.sh: line 60: cmake: command not found make: *** No targets specified and no makefile found. Stop. Could not find libwarpctc.so in ../build. Build warp-ctc and set WARP_CTC_PATH to the location of libwarpctc.so (default is '../build')
— Reply to this email directly, view it on GitHubhttps://github.com/hikopensource/DAVAR-Lab-OCR/issues/89#issuecomment-1149009491, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUSKD7MIOJ24DYOITDTB3KTVN6GLVANCNFSM5X7SJIRQ. You are receiving this because you commented.Message ID: @.***>
CONFIDENTIALITY NOTICE: This electronic message is intended to be viewed only by the individual or entity to whom it is addressed. It may contain information that is privileged, confidential and exempt from disclosure under applicable law. Any dissemination, distribution or copying of this communication is strictly prohibited without our prior permission. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, or if you have received this communication in error, please notify us immediately by return e-mail and delete the original message and any copies of it from your computer system. For further information about Hikvision company. please see our website at www.hikvision.comhttp://www.hikvision.com
I commented out the lines 26-31 and 36-61 in setup.sh but the ".so" file was not generated in this directory "davarocr\davar_table\datasets\pipelines\lib". What lines do I need to comment out since I just want to train the LGPMA?
You can run following command in your shell like
g++ -shared -o /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.so -fPIC /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp pkg-config --cflags --libs opencv
Change the "/path/to" as the absolute path to davarocr.
You can run following command in your shell like
g++ -shared -o /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.so -fPIC /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp
pkg-config --cflags --libs opencvChange the "/path/to" as the absolute path to davarocr.
I had the same training error with GPMADataGeneration and I run the command that you refer here with no error after installing C++ opencv. But i have the OSError where the .so file cannot be found.
My C++ opencv is installed locally instead of system-wide.
You can run following command in your shell like g++ -shared -o /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.so -fPIC /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp
pkg-config --cflags --libs opencvChange the "/path/to" as the absolute path to davarocr.I had the same training error with GPMADataGeneration and I run the command that you refer here with no error after installing C++ opencv. But i have the OSError where the .so file cannot be found.
My C++ opencv is installed locally instead of system-wide.
If there is .so file generated but cannot be found when running, you can also indentify its path in the model config (in lgpma_base.py) like:
train_pipeline = [
...
dict(type='GPMADataGeneration'
lib_name='gpma_data.so',
lib_dir='/path/to/pipelines/lib/'
),
...
]
I managed to solve this already thank you. The problem for me is the configuration of the C++ opencv.
The .so file is not generated. How can I generate it?
On Thu, Jun 16, 2022 at 8:26 AM Liang Qiao @.***> wrote:
You can run following command in your shell like g++ -shared -o /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.so -fPIC /path/to/davarocr/davar_table/datasets/pipelines/lib/gpma_data.cpp pkg-config --cflags --libs opencv Change the "/path/to" as the absolute path to davarocr.
I had the same training error with GPMADataGeneration and I run the command that you refer here with no error after installing C++ opencv. But i have the OSError where the .so file cannot be found.
My C++ opencv is installed locally instead of system-wide.
If there is .so file generated but cannot be found when running, you can also indentify its path in the model config (in lgpma_base.py) like:
train_pipeline = [ ... dict(type='GPMADataGeneration' lib_name='gpma_data.so', lib_dir='/path/to/pipelines/lib/' ), ... ]
— Reply to this email directly, view it on GitHub https://github.com/hikopensource/DAVAR-Lab-OCR/issues/89#issuecomment-1157601905, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECILPDPL235QJQFWPBCPLLVPMMPXANCNFSM5X7SJIRQ . You are receiving this because you authored the thread.Message ID: @.***>
I managed to solve this already thank you. The problem for me is the configuration of the C++ opencv.
I have the same problem about error of opencv, How did you solve it?
I managed to solve this already thank you. The problem for me is the configuration of the C++ opencv.
I have the same problem about error of opencv, How did you solve it?
set the PKG_CONFIG_PATH to the directory that contains a opencv.pc (or opencv4.pc) file, but not the file itself. Then, the opencv.pc file has to point to the place where you build and install the C++ opencv. An example of the opencv.pc file can be as follows,
prefix=/to/your/directory/opencv-4.1.1/build/install exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include/opencv4
Name: OpenCV Description: Open Source Computer Vision Library Version: 4.6.0 Libs: -L${exec_prefix}/lib -lopencv_highgui -lopencv_ml -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_video -lopencv_calib3d -lopencv_features2d -lopencv_dnn -lopencv_flann -lopencv_videoio -lopencv_imgcodecs -lopencv_imgproc -lopencv_core Libs.private: -ldl -lm -lpthread -lrt Cflags: -I${includedir}
If you installed C++ opencv systemwide, then the default prefix path should be to /usr/local. Otherwise, which you have installed locally, then you need to point it a local path.