Union14M icon indicating copy to clipboard operation
Union14M copied to clipboard

请问mmcv、mmdet、mmengine需要的版本各是多少?

Open ggxxer opened this issue 1 year ago • 6 comments

感谢您的工作,我在尝试跑通您的代码,但是在我使用自己预训练的模型进行微调时,遇到了上述几个库之间版本不兼容的报错,后续查阅资料,下载了兼容的版本: mmcv 2.0.0rc4 https://github.com/open-mmlab/mmcv mmdet 3.0.0rc5 https://github.com/open-mmlab/mmdetection mmengine 0.10.5 https://github.com/open-mmlab/mmengine mmocr 1.0.0 /home/gx/Union14M-main/mmocr-dev-1.x 但出现了新的报错如下: Traceback (most recent call last): File "tools/train.py", line 113, in main() File "tools/train.py", line 102, in main runner = Runner.from_cfg(cfg) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 462, in from_cfg runner = cls( File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 429, in init self.model = self.build_model(model) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 836, in build_model model = MODELS.build(model) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/Union14M-main/mmocr-dev-1.x/mmocr/models/textrecog/recognizers/encoder_decoder_recognizer.py", line 52, in init self.backbone = MODELS.build(backbone) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg return build_from_cfg(cfg, registry, default_args) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/Union14M-main/mmocr-dev-1.x/mmocr/models/common/backbones/vit.py", line 57, in init super(VisionTransformer, self).init( TypeError: init() got an unexpected keyword argument 'vit_config' 查阅资料没有找到有效的解决方式,我猜测仍然是mm库之间的版本问题导致,所以想请问您训练时具体的版本号,谢谢

ggxxer avatar Sep 23 '24 04:09 ggxxer

Hi @ggxxer 看上去像是 timm 库的bug,按照这里的流程安装 timm 试试呢: https://github.com/Mountchicken/Union14M/blob/main/docs/pretrain.md#1-installation

Mountchicken avatar Sep 23 '24 06:09 Mountchicken

Hi @ggxxer 看上去像是 timm 库的bug,按照这里的流程安装 timm 试试呢: https://github.com/Mountchicken/Union14M/blob/main/docs/pretrain.md#1-installation

谢谢回复,timm已经按这个流程安装,问题仍然存在

ggxxer avatar Sep 23 '24 06:09 ggxxer

除此之外,我在使用您提供的预训练ViT-B进行微调时,也遇到了下面的报错,无法正常运行: Load pre-trained checkpoint from: /home/gx/Union14M-main/vit_base_checkpoint-19.pth _IncompatibleKeys(missing_keys=[], unexpected_keys=['mask_token']) 09/23 15:39:34 - mmengine - INFO - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used. 09/23 15:39:34 - mmengine - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) RuntimeInfoHook
(BELOW_NORMAL) LoggerHook

before_train: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(VERY_LOW ) CheckpointHook

before_train_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) DistSamplerSeedHook

before_train_iter: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook

after_train_iter: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_train_epoch: (NORMAL ) IterTimerHook
(NORMAL ) SyncBuffersHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

before_val: (VERY_HIGH ) RuntimeInfoHook

before_val_epoch: (NORMAL ) IterTimerHook
(NORMAL ) SyncBuffersHook

before_val_iter: (NORMAL ) IterTimerHook

after_val_iter: (NORMAL ) IterTimerHook
(NORMAL ) VisualizationHook
(BELOW_NORMAL) LoggerHook

after_val_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_val: (VERY_HIGH ) RuntimeInfoHook

after_train: (VERY_HIGH ) RuntimeInfoHook
(VERY_LOW ) CheckpointHook

before_test: (VERY_HIGH ) RuntimeInfoHook

before_test_epoch: (NORMAL ) IterTimerHook

before_test_iter: (NORMAL ) IterTimerHook

after_test_iter: (NORMAL ) IterTimerHook
(NORMAL ) VisualizationHook
(BELOW_NORMAL) LoggerHook

after_test_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook

after_test: (VERY_HIGH ) RuntimeInfoHook

after_run: (BELOW_NORMAL) LoggerHook

Traceback (most recent call last): File "tools/train.py", line 113, in main() File "tools/train.py", line 109, in main runner.train() File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1742, in train self._val_loop = self.build_val_loop( File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1562, in build_val_loop loop = LOOPS.build( File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/loops.py", line 347, in init super().init(runner, dataloader) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/base_loop.py", line 26, in init self.dataloader = runner.build_dataloader( File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1370, in build_dataloader dataset = DATASETS.build(dataset_cfg) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/Union14M-main/mmocr-dev-1.x/mmocr/datasets/dataset_wrapper.py", line 49, in init self.datasets.append(DATASETS.build(dataset)) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 247, in init self.full_init() File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 298, in full_init self.data_list = self.load_data_list() File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 435, in load_data_list annotations = load(self.ann_file) File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/fileio/io.py", line 852, in load with StringIO(file_backend.get_text(file)) as f: File "/home/gx/anaconda3/envs/maee/lib/python3.8/site-packages/mmengine/fileio/backends/local_backend.py", line 56, in get_text with open(filepath, encoding=encoding) as f: FileNotFoundError: [Errno 2] No such file or directory: '../data/common_benchmarks/CUTE80/annotation.json' 以下是环境中所有安装库的版本: Package Version Editable project location


absl-py 2.1.0 addict 2.4.0 albucore 0.0.17 albumentations 1.4.16 aliyun-python-sdk-core 2.15.2 aliyun-python-sdk-kms 2.16.5 annotated-types 0.7.0 asynctest 0.13.0 attrs 24.2.0 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 codecov 2.1.13 colorama 0.4.6 contourpy 1.1.1 coverage 7.6.1 crcmod 1.7 cryptography 43.0.1 cycler 0.12.1 eval_type_backport 0.2.0 exceptiongroup 1.2.2 filelock 3.14.0 flake8 7.1.1 fonttools 4.53.1 google-auth 2.34.0 google-auth-oauthlib 0.4.6 grpcio 1.66.1 idna 3.10 imageio 2.35.1 imgaug 0.4.0 importlib_metadata 8.5.0 importlib_resources 6.4.5 iniconfig 2.0.0 interrogate 1.7.0 isort 5.13.2 jmespath 0.10.0 kiwisolver 1.4.7 kwarray 0.7.0 lanms-neo 1.0.2 lazy_loader 0.4 lmdb 1.4.1 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.7.5 mccabe 0.7.0 mdurl 0.1.2 mmcv 2.0.0rc4 mmdet 3.0.0rc5 mmengine 0.10.5 mmocr 1.0.0 /home/gx/Union14M-main/mmocr-dev-1.x model-index 0.1.11 networkx 3.1 numpy 1.24.4 oauthlib 3.2.2 opencv-python 4.10.0.84 opencv-python-headless 4.10.0.84 opendatalab 0.0.10 openmim 0.3.9 openxlab 0.1.1 ordered-set 4.1.0 oss2 2.17.0 packaging 24.1 pandas 2.0.3 parameterized 0.9.0 pillow 10.4.0 pip 24.2 platformdirs 4.3.6 pluggy 1.5.0 protobuf 3.20.3 py 1.11.0 pyasn1 0.6.1 pyasn1_modules 0.4.1 pyclipper 1.3.0.post5 pycocotools 2.0.7 pycodestyle 2.12.1 pycparser 2.22 pycryptodome 3.20.0 pydantic 2.9.2 pydantic_core 2.23.4 pyflakes 3.2.0 Pygments 2.18.0 pyparsing 3.1.4 pytest 8.3.3 pytest-cov 5.0.0 pytest-runner 6.0.1 python-dateutil 2.9.0.post0 pytz 2023.4 PyWavelets 1.4.1 PyYAML 6.0.2 rapidfuzz 3.9.7 requests 2.28.2 requests-oauthlib 2.0.0 rich 13.4.2 rsa 4.9 safetensors 0.4.5 scikit-image 0.21.0 scipy 1.10.1 setuptools 60.2.0 shapely 2.0.6 six 1.16.0 tabulate 0.9.0 tensorboard 2.11.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 termcolor 2.4.0 terminaltables 3.1.10 tifffile 2023.7.10 timm 0.3.2 tomli 2.0.1 torch 1.13.0+cu117 torchaudio 0.13.0+cu117 torchvision 0.14.0+cu117 tqdm 4.65.2 typing_extensions 4.12.2 tzdata 2024.1 ubelt 1.3.6 urllib3 1.26.20 Werkzeug 3.0.4 wheel 0.44.0 xdoctest 1.2.0 yapf 0.40.2 zipp 3.20.2 还是想请问您训练时,mm系列库具体的版本号

ggxxer avatar Sep 23 '24 07:09 ggxxer

请问common_benchmarks应该放到什么位置?因为我依然是如下报错: Traceback (most recent call last): File "tools/train.py", line 113, in main() File "tools/train.py", line 109, in main runner.train() File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1742, in train self._val_loop = self.build_val_loop( File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1562, in build_val_loop loop = LOOPS.build( File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/runner/loops.py", line 347, in init super().init(runner, dataloader) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/runner/base_loop.py", line 26, in init self.dataloader = runner.build_dataloader( File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1370, in build_dataloader dataset = DATASETS.build(dataset_cfg) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/Union14M-main/mmocr-dev-1.x/mmocr/datasets/dataset_wrapper.py", line 49, in init self.datasets.append(DATASETS.build(dataset)) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(**args) # type: ignore File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 247, in init self.full_init() File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 298, in full_init self.data_list = self.load_data_list() File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 435, in load_data_list annotations = load(self.ann_file) File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/fileio/io.py", line 852, in load with StringIO(file_backend.get_text(file)) as f: File "/home/gx/anaconda3/envs/mmocr1.0/lib/python3.8/site-packages/mmengine/fileio/backends/local_backend.py", line 56, in get_text with open(filepath, encoding=encoding) as f: FileNotFoundError: [Errno 2] No such file or directory: '../data/common_benchmarks/CUTE80/annotation.json' 我注意到您在README中不仅提供了Union14M-U/L 的下载,还提供了common_benchmarks的下载,但我没有看到后者出现在前者的dataset structure当中,所以请问应该将其放在哪个位置来解决报错?

ggxxer avatar Sep 24 '24 04:09 ggxxer

  1. 对于mm的版本号
os.system('python -m mim install "mmcv==2.0.0rc4"')
os.system('python -m mim install mmengine==0.7.1')
os.system('python -m mim install "mmdet==3.0.0rc5"')
  1. 在 mmocr-dev-1.x 下面创建一个 data文件夹就行。可以参考这里的路径:https://github.com/Mountchicken/Union14M/tree/main/mmocr-dev-1.x/configs/textrecog/base/datasets

Mountchicken avatar Sep 24 '24 09:09 Mountchicken

  1. 对于mm的版本号
os.system('python -m mim install "mmcv==2.0.0rc4"')
os.system('python -m mim install mmengine==0.7.1')
os.system('python -m mim install "mmdet==3.0.0rc5"')
  1. 在 mmocr-dev-1.x 下面创建一个 data文件夹就行。可以参考这里的路径:https://github.com/Mountchicken/Union14M/tree/main/mmocr-dev-1.x/configs/textrecog/base/datasets

十分感谢您的回复,问题已经解决!

ggxxer avatar Sep 24 '24 11:09 ggxxer