MinerU icon indicating copy to clipboard operation
MinerU copied to clipboard
Published 1 week ago • verified publisher icon opendatalab

转换时间过慢问题

Open UFOyyds opened this issue 1 year ago • 7 comments

Description of the bug | 错误描述

一篇简短的PDF to markdown 等待时间有点久，不知什么原因？感谢给予指导

How to reproduce the bug | 如何复现

JSON文件中默认"max_time": 400，请问这个可以适度调节吗？

Operating system | 操作系统

Windows

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

0.6.x

Device mode | 设备模式

cpu

UFOyyds avatar Aug 05 '24 05:08 UFOyyds

能上传一些解析过程中的日志看看具体是具体哪个步骤耗时过长吗

myhloli avatar Aug 05 '24 06:08 myhloli

cpu是会比较慢的，你可以试试gpu环境，目前我也用的cpu，模型加载1分钟左右。不算模型加载的话使用ocr处理，一页大概要处理30-45秒，不过因为设备限制我也不确定gpu能快多少。

2257396011 avatar Aug 05 '24 06:08 2257396011

cpu是会比较慢的，你可以试试gpu环境，目前我也用的cpu，模型加载1分钟左右。不算模型加载的话使用ocr处理，一页大概要处理30-45秒，不过因为设备限制我也不确定gpu能快多少。

gpu正常情况一页1～2秒左右

myhloli avatar Aug 05 '24 06:08 myhloli

能上传一些解析过程中的日志看看具体是具体哪个步骤耗时过长吗

`(detectron2_env) C:\Users\XXXXX>magic-pdf pdf-command --pdf "pdf_path" --inside_model true 2024-08-05 14:10:24.153 | WARNING | magic_pdf.cli.magicpdf:get_model_json:312 - not found json "pdf_path".json existed 2024-08-05 14:10:24.154 | WARNING | magic_pdf.libs.config_reader:get_local_dir:64 - 'temp-output-dir' not found in magic-pdf.json, use '/tmp' as default 2024-08-05 14:10:24.378 | INFO | magic_pdf.libs.pdf_check:detect_invalid_chars:57 - cid_count: 0, text_len: 2, cid_chars_radio: 0.0 2024-08-05 14:10:24.378 | WARNING | magic_pdf.filter.pdf_classify_by_type:classify:334 - pdf is not classified by area and text_len, by_image_area: False, by_text: False, by_avg_words: False, by_img_num: True, by_text_layout: False, by_img_narrow_strips: True, by_invalid_chars: True INFO:datasets:PyTorch version 2.3.1 available. 2024-08-05 14:10:32.531 | INFO | magic_pdf.model.pdf_extract_kit:init:99 - DocAnalysis init, this may take some times. apply_layout: True, apply_formula: True, apply_ocr: True 2024-08-05 14:10:32.532 | INFO | magic_pdf.model.pdf_extract_kit:init:107 - using device: cpu 2024-08-05 14:10:32.532 | INFO | magic_pdf.model.pdf_extract_kit:init:109 - using models_dir: E:\PDF-Extract-Kit\models CustomVisionEncoderDecoderModel init CustomMBartForCausalLM init CustomMBartDecoder init [08/05 14:10:55 detectron2]: Rank of current process: 0. World size: 1 [08/05 14:10:56 detectron2]: Environment info:

sys.platform win32 Python 3.10.14 | packaged by Anaconda, Inc. | (main, May 6 2024, 19:44:50) [MSC v.1916 64 bit (AMD64)] numpy 1.26.4 detectron2 0.6 @D:\anaconda\envs\detectron2_env\lib\site-packages\detectron2 detectron2._C not built correctly: DLL load failed while importing _C: 找不到指定的程序。 DETECTRON2_ENV_MODULE PyTorch 2.3.1+cpu @D:\anaconda\envs\detectron2_env\lib\site-packages\torch PyTorch debug build False torch._C._GLIBCXX_USE_CXX11_ABI False GPU available No: torch.cuda.is_available() == False Pillow 10.4.0 torchvision 0.18.1+cpu @D:\anaconda\envs\detectron2_env\lib\site-packages\torchvision fvcore 0.1.5.post20221221 iopath 0.1.9 cv2 4.6.0

PyTorch built with:

  • C++ Version: 201703
  • MSVC 192930154
  • Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)
  • OpenMP 2019
  • LAPACK is enabled (usually provided by MKL)
  • CPU capability usage: AVX2
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /Zc:__cplusplus /bigobj /FS /utf-8 -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=0, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

[08/05 14:10:56 detectron2]: Command line arguments: {'config_file': 'D:\anaconda\envs\detectron2_env\lib\site-packages\magic_pdf\resources\model_config\layoutlmv3\layoutlmv3_base_inference.yaml', 'resume': False, 'eval_only': False, 'num_gpus': 1, 'num_machines': 1, 'machine_rank': 0, 'dist_url': 'tcp://127.0.0.1:57823', 'opts': ['MODEL.WEIGHTS', 'E:\PDF-Extract-Kit\models\Layout/model_final.pth']} [08/05 14:10:56 detectron2]: Contents of args.config_file=D:\anaconda\envs\detectron2_env\lib\site-packages\magic_pdf\resources\model_config\layoutlmv3\layoutlmv3_base_inference.yaml: AUG: DETR: true CACHE_DIR: ~/cache/huggingface CUDNN_BENCHMARK: false DATALOADER: ASPECT_RATIO_GROUPING: true FILTER_EMPTY_ANNOTATIONS: false NUM_WORKERS: 4 REPEAT_THRESHOLD: 0.0 SAMPLER_TRAIN: TrainingSampler DATASETS: PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000 PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000 PROPOSAL_FILES_TEST: [] PROPOSAL_FILES_TRAIN: [] TEST:

  • scihub_train TRAIN:
  • scihub_train GLOBAL: HACK: 1.0 ICDAR_DATA_DIR_TEST: '' ICDAR_DATA_DIR_TRAIN: '' INPUT: CROP: ENABLED: true SIZE:
    • 384
    • 600 TYPE: absolute_range FORMAT: RGB MASK_FORMAT: polygon MAX_SIZE_TEST: 1333 MAX_SIZE_TRAIN: 1333 MIN_SIZE_TEST: 800 MIN_SIZE_TRAIN:
  • 480
  • 512
  • 544
  • 576
  • 608
  • 640
  • 672
  • 704
  • 736
  • 768
  • 800 MIN_SIZE_TRAIN_SAMPLING: choice RANDOM_FLIP: horizontal MODEL: ANCHOR_GENERATOR: ANGLES:
      • -90
      • 0
      • 90 ASPECT_RATIOS:
      • 0.5
      • 1.0
      • 2.0 NAME: DefaultAnchorGenerator OFFSET: 0.0 SIZES:
      • 32
      • 64
      • 128
      • 256
      • 512 BACKBONE: FREEZE_AT: 2 NAME: build_vit_fpn_backbone CONFIG_PATH: '' DEVICE: cuda FPN: FUSE_TYPE: sum IN_FEATURES:
    • layer3
    • layer5
    • layer7
    • layer11 NORM: '' OUT_CHANNELS: 256 IMAGE_ONLY: true KEYPOINT_ON: false LOAD_PROPOSALS: false MASK_ON: true META_ARCHITECTURE: VLGeneralizedRCNN PANOPTIC_FPN: COMBINE: ENABLED: true INSTANCES_CONFIDENCE_THRESH: 0.5 OVERLAP_THRESH: 0.5 STUFF_AREA_LIMIT: 4096 INSTANCE_LOSS_WEIGHT: 1.0 PIXEL_MEAN:
  • 127.5
  • 127.5
  • 127.5 PIXEL_STD:
  • 127.5
  • 127.5
  • 127.5 PROPOSAL_GENERATOR: MIN_SIZE: 0 NAME: RPN RESNETS: DEFORM_MODULATED: false DEFORM_NUM_GROUPS: 1 DEFORM_ON_PER_STAGE:
    • false
    • false
    • false
    • false DEPTH: 50 NORM: FrozenBN NUM_GROUPS: 1 OUT_FEATURES:
    • res4 RES2_OUT_CHANNELS: 256 RES5_DILATION: 1 STEM_OUT_CHANNELS: 64 STRIDE_IN_1X1: true WIDTH_PER_GROUP: 64 RETINANET: BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_WEIGHTS:
    • 1.0
    • 1.0
    • 1.0
    • 1.0 FOCAL_LOSS_ALPHA: 0.25 FOCAL_LOSS_GAMMA: 2.0 IN_FEATURES:
    • p3
    • p4
    • p5
    • p6
    • p7 IOU_LABELS:
    • 0
    • -1
    • 1 IOU_THRESHOLDS:
    • 0.4
    • 0.5 NMS_THRESH_TEST: 0.5 NORM: '' NUM_CLASSES: 10 NUM_CONVS: 4 PRIOR_PROB: 0.01 SCORE_THRESH_TEST: 0.05 SMOOTH_L1_LOSS_BETA: 0.1 TOPK_CANDIDATES_TEST: 1000 ROI_BOX_CASCADE_HEAD: BBOX_REG_WEIGHTS:
      • 10.0
      • 10.0
      • 5.0
      • 5.0
      • 20.0
      • 20.0
      • 10.0
      • 10.0
      • 30.0
      • 30.0
      • 15.0
      • 15.0 IOUS:
    • 0.5
    • 0.6
    • 0.7 ROI_BOX_HEAD: BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_LOSS_WEIGHT: 1.0 BBOX_REG_WEIGHTS:
    • 10.0
    • 10.0
    • 5.0
    • 5.0 CLS_AGNOSTIC_BBOX_REG: true CONV_DIM: 256 FC_DIM: 1024 NAME: FastRCNNConvFCHead NORM: '' NUM_CONV: 0 NUM_FC: 2 POOLER_RESOLUTION: 7 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 SMOOTH_L1_BETA: 0.0 TRAIN_ON_PRED_BOXES: false ROI_HEADS: BATCH_SIZE_PER_IMAGE: 512 IN_FEATURES:
    • p2
    • p3
    • p4
    • p5 IOU_LABELS:
    • 0
    • 1 IOU_THRESHOLDS:
    • 0.5 NAME: CascadeROIHeads NMS_THRESH_TEST: 0.5 NUM_CLASSES: 10 POSITIVE_FRACTION: 0.25 PROPOSAL_APPEND_GT: true SCORE_THRESH_TEST: 0.05 ROI_KEYPOINT_HEAD: CONV_DIMS:
    • 512
    • 512
    • 512
    • 512
    • 512
    • 512
    • 512
    • 512 LOSS_WEIGHT: 1.0 MIN_KEYPOINTS_PER_IMAGE: 1 NAME: KRCNNConvDeconvUpsampleHead NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: true NUM_KEYPOINTS: 17 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 ROI_MASK_HEAD: CLS_AGNOSTIC_MASK: false CONV_DIM: 256 NAME: MaskRCNNConvUpsampleHead NORM: '' NUM_CONV: 4 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 RPN: BATCH_SIZE_PER_IMAGE: 256 BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_LOSS_WEIGHT: 1.0 BBOX_REG_WEIGHTS:
    • 1.0
    • 1.0
    • 1.0
    • 1.0 BOUNDARY_THRESH: -1 CONV_DIMS:
    • -1 HEAD_NAME: StandardRPNHead IN_FEATURES:
    • p2
    • p3
    • p4
    • p5
    • p6 IOU_LABELS:
    • 0
    • -1
    • 1 IOU_THRESHOLDS:
    • 0.3
    • 0.7 LOSS_WEIGHT: 1.0 NMS_THRESH: 0.7 POSITIVE_FRACTION: 0.5 POST_NMS_TOPK_TEST: 1000 POST_NMS_TOPK_TRAIN: 2000 PRE_NMS_TOPK_TEST: 1000 PRE_NMS_TOPK_TRAIN: 2000 SMOOTH_L1_BETA: 0.0 SEM_SEG_HEAD: COMMON_STRIDE: 4 CONVS_DIM: 128 IGNORE_VALUE: 255 IN_FEATURES:
    • p2
    • p3
    • p4
    • p5 LOSS_WEIGHT: 1.0 NAME: SemSegFPNHead NORM: GN NUM_CLASSES: 10 VIT: DROP_PATH: 0.1 IMG_SIZE:
    • 224
    • 224 NAME: layoutlmv3_base OUT_FEATURES:
    • layer3
    • layer5
    • layer7
    • layer11 POS_TYPE: abs WEIGHTS: OUTPUT_DIR: SCIHUB_DATA_DIR_TRAIN: ~/publaynet/layout_scihub/train SEED: 42 SOLVER: AMP: ENABLED: true BACKBONE_MULTIPLIER: 1.0 BASE_LR: 0.0002 BIAS_LR_FACTOR: 1.0 CHECKPOINT_PERIOD: 2000 CLIP_GRADIENTS: CLIP_TYPE: full_model CLIP_VALUE: 1.0 ENABLED: true NORM_TYPE: 2.0 GAMMA: 0.1 GRADIENT_ACCUMULATION_STEPS: 1 IMS_PER_BATCH: 32 LR_SCHEDULER_NAME: WarmupCosineLR MAX_ITER: 20000 MOMENTUM: 0.9 NESTEROV: false OPTIMIZER: ADAMW REFERENCE_WORLD_SIZE: 0 STEPS:
  • 10000 WARMUP_FACTOR: 0.01 WARMUP_ITERS: 333 WARMUP_METHOD: linear WEIGHT_DECAY: 0.05 WEIGHT_DECAY_BIAS: null WEIGHT_DECAY_NORM: 0.0 TEST: AUG: ENABLED: false FLIP: true MAX_SIZE: 4000 MIN_SIZES:
    • 400
    • 500
    • 600
    • 700
    • 800
    • 900
    • 1000
    • 1100
    • 1200 DETECTIONS_PER_IMAGE: 100 EVAL_PERIOD: 1000 EXPECTED_RESULTS: [] KEYPOINT_OKS_SIGMAS: [] PRECISE_BN: ENABLED: false NUM_ITER: 200 VERSION: 2 VIS_PERIOD: 0

[08/05 14:10:58 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from E:\PDF-Extract-Kit\models\Layout/model_final.pth ... [08/05 14:10:58 fvcore.common.checkpoint]: [Checkpointer] Loading from e:\PDF-Extract-Kit\models\Layout/model_final.pth ... 2024-08-05 14:11:00.031 | INFO | magic_pdf.model.pdf_extract_kit:init:132 - DocAnalysis init done! 2024-08-05 14:11:00.032 | INFO | magic_pdf.model.doc_analyze_by_custom_model:custom_model_init:92 - model init cost: 35.65274977684021 2024-08-05 14:11:35.095 | INFO | magic_pdf.model.pdf_extract_kit:call:143 - layout detection cost: 34.73

0: 1888x1344 (no detections), 5070.7ms Speed: 27.2ms preprocess, 5070.7ms inference, 1.0ms postprocess per image at shape (1, 3, 1888, 1344) 2024-08-05 14:11:42.570 | INFO | magic_pdf.model.pdf_extract_kit:call:173 - formula nums: 0, mfr time: 0.0 2024-08-05 14:11:50.457 | INFO | magic_pdf.model.pdf_extract_kit:call:250 - ocr cost: 7.89 2024-08-05 14:12:24.253 | INFO | magic_pdf.model.pdf_extract_kit:call:143 - layout detection cost: 33.79

0: 1888x1344 (no detections), 4877.3ms Speed: 26.4ms preprocess, 4877.3ms inference, 0.0ms postprocess per image at shape (1, 3, 1888, 1344) 2024-08-05 14:12:29.161 | INFO | magic_pdf.model.pdf_extract_kit:call:173 - formula nums: 0, mfr time: 0.0 2024-08-05 14:12:30.193 | INFO | magic_pdf.model.pdf_extract_kit:call:250 - ocr cost: 1.03 2024-08-05 14:12:30.194 | INFO | magic_pdf.model.doc_analyze_by_custom_model:doc_analyze:118 - doc analyze cost: 89.82563972473145 2024-08-05 14:12:30.260 | INFO | magic_pdf.pdf_parse_union_core:pdf_parse_union:221 - page_id: 0, last_page_cost_time: 0.0 2024-08-05 14:12:30.480 | INFO | magic_pdf.pdf_parse_union_core:pdf_parse_union:221 - page_id: 1, last_page_cost_time: 0.22 2024-08-05 14:12:30.802 | INFO | magic_pdf.para.para_split_v2:__connect_middle_align_text:682 - 1.0 2024-08-05 14:12:30.848 | INFO | magic_pdf.pipe.UNIPipe:pipe_mk_markdown:48 - uni_pipe mk mm_markdown finished 2024-08-05 14:12:30.865 | INFO | magic_pdf.pipe.UNIPipe:pipe_mk_uni_format:43 - uni_pipe mk content list finished 2024-08-05 14:12:30.867 | INFO | magic_pdf.cli.magicpdf:do_parse:165 - local output dir is '/tmp\magic-pdf\单个文件名\auto', you can found the result in it.`

UFOyyds avatar Aug 05 '24 06:08 UFOyyds

@UFOyyds 看了下你的log，一个2页的扫描版pdf，除去模型加载34秒，解析用时89秒，其中约70秒是在layout检测上，这个时间是符合预期的。使用纯cpu解析时，解析效率与cpu的核心数量和主频相关性极高。一般4核cpu单页layout时间30秒左右是正常的，8核cpu可以把这个时间缩短到15秒左右。 cpu解析方案只是为了兼容更多设备，为没有gpu设备的用户提供的一种快速体验方案，不适合在生产环境长时间和大量处理文档使用，这一点在readme中也已经做了说明。 如果需要较快的解析速度，一张大显存显卡是不可缺少的。具体显卡型号选择，可以参考

https://github.com/opendatalab/MinerU/blob/master/README_zh-CN_v2.md

中【安装前必看——软硬件环境支持说明】的表格。

myhloli avatar Aug 05 '24 07:08 myhloli

@UFOyyds 看了下你的log，一个2页的扫描版pdf，除去模型加载34秒，解析用时89秒，其中约70秒是在layout检测上，这个时间是符合预期的。使用纯cpu解析时，解析效率与cpu的核心数量和主频相关性极高。一般4核cpu单页layout时间30秒左右是正常的，8核cpu可以把这个时间缩短到15秒左右。 cpu解析方案只是为了兼容更多设备，为没有gpu设备的用户提供的一种快速体验方案，不适合在生产环境长时间和大量处理文档使用，这一点在readme中也已经做了说明。 如果需要较快的解析速度，一张大显存显卡是不可缺少的。具体显卡型号选择，可以参考

https://github.com/opendatalab/MinerU/blob/master/README_zh-CN_v2.md

中【安装前必看——软硬件环境支持说明】的表格。

感谢解答！

UFOyyds avatar Aug 05 '24 07:08 UFOyyds

JSON文件中默认"max_time": 400，请问这个可以适度调节吗？

这个是表格识别的超时时间，单位：s

papayalove avatar Aug 05 '24 09:08 papayalove