layout-parser icon indicating copy to clipboard operation
layout-parser copied to clipboard

conflicting dependencies

Open bertsky opened this issue 3 years ago • 37 comments

If I follow the installation steps currently in the README, I get the following from the last pip command (i.e. when installing layout-parser a second time, after installing detectron2):

Installing collected packages: pycocotools, fvcore
  Attempting uninstall: pycocotools
    Found existing installation: pycocotools 2.0.2
    Uninstalling pycocotools-2.0.2:
      Successfully uninstalled pycocotools-2.0.2
  Attempting uninstall: fvcore
    Found existing installation: fvcore 0.1.2.post20210128
    Uninstalling fvcore-0.1.2.post20210128:
      Successfully uninstalled fvcore-0.1.2.post20210128
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ocrd-segment 0.1.3 requires pycocotools>=2.0.2, but you have pycocotools 2.0.1 which is incompatible.
detectron2 0.3 requires fvcore<0.1.3,>=0.1.2, but you have fvcore 0.1.1.post20200623 which is incompatible.
detectron2 0.3 requires pycocotools>=2.0.2, but you have pycocotools 2.0.1 which is incompatible.
Successfully installed fvcore-0.1.1.post20200623 pycocotools-2.0.1

So it seems we are in conflict with current master of detectron2 here.

Could you point me to the right version of detectron2 to fetch, or update layout-parser accordingly?

Also, why not make the detectron2 dependency explicit in setup.py?

bertsky avatar Jan 29 '21 16:01 bertsky

I am not sure if this is related: When I ignore the above errors and continue with detection, I see these:

>>> import layoutparser as lp
** fvcore version of PathManager will be deprecated soon. **
** Please migrate to the version in iopath repo. **
https://github.com/facebookresearch/iopath
>>> layout = model.detect(image)
/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/detectron2/modeling/roi_heads/fast_rcnn.py:154: UserWarning: This overload of nonzero is deprecated:
	nonzero()
Consider using one of the following signatures instead:
	nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  filter_inds = filter_mask.nonzero()

And layout is empty.

bertsky avatar Jan 29 '21 16:01 bertsky

Tried with the last stable release, i.e. detectron2==0.3, but still the same result.

bertsky avatar Jan 29 '21 19:01 bertsky

0.2.1 (on torch==0.1.6) is also the same.

Note, one cannot just ignore detectron2's fvcore requirement, since it is enforced:

assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2"
AssertionError: Requires fvcore>=0.1.2

bertsky avatar Jan 29 '21 20:01 bertsky

And layout is empty.

That might be a specific problem of the lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config model. I do get non-empty results for all the other models (including ResNet-50 based PubLayNet models) – but the results are pretty much useless for my historical documents.

I can provide samples from layoutparser.draw_box().

bertsky avatar Jan 29 '21 22:01 bertsky

Thanks for reporting this issue! There has been some updates for Detection, and the support for fvcore is somewhat problematic in the current version. I plan to remove the fvcore dependency in the next major release, which should be around within 1 month.

And speaking of the models

  1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:
    lp.Detectron2LayoutModel(
                config_path = xxx,
                extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)
    
  2. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models. But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:
    • for the pre-training issue you mentioned earlier, it should be fixed within 1wk
    • and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

lolipopshock avatar Jan 30 '21 00:01 lolipopshock

Many thanks for your fast response!

Good to know you are already onto these dependency conflicts.

So am I correct in inferring these warnings don't actually do any harm?

  1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:
    lp.Detectron2LayoutModel(
                config_path = xxx,
                extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)
    

Understood. I went as far down as lp.Detectron2LayoutModel('lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.01]), but still got no results there.

  1. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models.

With pleasure:

  • original: FILE_0002_ORIGINAL
  • on newspapernavigator Mask-RCNN model: FILE_0002_LP_newspapernavigator_red
  • on primalayout Mask-RCNN model: FILE_0002_LP_primalayout
  • on publaynet (small) Mask-RCNN model: FILE_0002_LP_publaynet

But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:

  • for the pre-training issue you mentioned earlier, it should be fixed within 1wk
  • and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

Wonderful!

bertsky avatar Jan 30 '21 01:01 bertsky

@bertsky, nice to virtually meet you. Found your Leipzig institutional affiliation via search engine sleuthing. :-) So wish you were on Twitter so you could be engaged in the fine-grained research chatter related to the intersection of Digital Humanities and AI/ML wrt digitization pipelines. As an independent Citizen Scientist, that communication channel is invaluable to me.

You must be buddies with Clemens Neudecker and the OCR-D network given your recent involvement in that mega project. I know them through involvement in the Time Machine EU project, NewsEye, DATeCH, and Andreas Maier’s Pattern Recognition lab at FAU etc. First noticed your GitHub comment about LAREX’s PAGEgts reading order issue. I resonated with that as magazines and newspapers have complex document structures where within page reading order is indeterminate while being determinant at the issue level.

It is not surprising to see you here kicking the tires of Zejiang @lolipopshock Layout-Parser. The PDF annotation tool Zejiang alludes to is #PAWLS from @allenai lab here: https://github.com/allenai/pawls. Looks awesome and could be very helpful to my #MAGAZINEgts research. Would corresponding to you via your Leipzig email address be okay? Or do you have a preferred way to chat about shared interests?

Jim-Salmons avatar Jan 30 '21 04:01 Jim-Salmons

BTW @bertsky, given my OCDish bent, I actually spent a looonnnggg day getting Layout-Parser installed and working on my Windows 10 development box. I told Zejiang @lolipopshock I’d contribute an installation note to that effect once I confirmed the procedure. So far I have not been successful rewalking that torturous process. :-/ Will likely wait for a Docker release so as not to need to knock myself out with future updates. :-)

Jim-Salmons avatar Jan 30 '21 04:01 Jim-Salmons

Many thanks for your fast response!

Good to know you are already onto these dependency conflicts.

So am I correct in inferring these warnings don't actually do any harm?

  1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:
    lp.Detectron2LayoutModel(
                config_path = xxx,
                extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)
    

Understood. I went as far down as lp.Detectron2LayoutModel('lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.01]), but still got no results there.

  1. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models.

With pleasure:

  • original: FILE_0002_ORIGINAL
  • on newspapernavigator Mask-RCNN model: FILE_0002_LP_newspapernavigator_red
  • on primalayout Mask-RCNN model: FILE_0002_LP_primalayout
  • on publaynet (small) Mask-RCNN model: FILE_0002_LP_publaynet

But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:

  • for the pre-training issue you mentioned earlier, it should be fixed within 1wk
  • and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

Wonderful!

Yes, please ignore the warnings. And the document is significantly different from the training datasets, so it's expected to generate the non-sense outputs.

lolipopshock avatar Jan 31 '21 01:01 lolipopshock

@Jim-Salmons, my pleasure!

Regarding communication channels, Twitter's not my cup of tea TBH, but you can reach me under the same nick at gitter.im. Probably not a replacement for public fine-grained research chatter, though. But as an open forum frequented by researchers and practitioners in the field alike, I can recommend and invite you to https://gitter.im/OCR-D/Lobby. (And yes, my Uni Leipzig email is also fine of course.)

Yes, I do know Clemens, and indeed continue to work on that mega-project (on and off, also hoping to stay involved in the next round).

I agree, reading order is especially tough on newspapers and magazines. Incidentally, I have recently come to use a nice new layout tool (a contribution from SBB to the Qurator project co-chaired by Clemens IIRC) which IMHO excels in that respect: eynollah.

PAWLS, I've heard of it, and welcome seeing a fresh new approach to GT production/annotation, with different focus than LAREX, Transkribus etc. But such tools only become interesing to me as soon as they support ALTO or PAGE.

But I must confess I don't know anything about your MAGAZINEgts yet, so I've got to catch up with reading in that area I guess...

bertsky avatar Jan 31 '21 22:01 bertsky

@lolipopshock

And the document is significantly different from the training datasets, so it's expected to generate the non-sense outputs.

I see. I had my hopes up a little for the HJDataset model, because they are equally historic and are similar in visual appearance. But of course, vertical text in horizontal columns and horizontal text in vertical columns are still very different kinds of layouts. Anyway, for the sake of completeness, here's the result of applying that model to my sample after 90° reflection: FILE_0002_LP_rot90_hjdataset

So I guess I should really start training my own models now. But I am wondering: Do you have any experience how well mixing different datasets (in the hope to increase generality) or transferring weights from a larger model like the full PubLayNet to a small domain model works?

bertsky avatar Jan 31 '21 23:01 bertsky

I noticed that I fed images in BGR instead of RGB (the cv2.imread example in the README does not explain this, should have read the other documentation first). But it does not make a difference. (Also, strangely, all models have input.format: BGR in their configuration...)

Then I tried to do binarization, deskewing and cropping externally. This did help somewhat:

  • on primalayout Mask-RCNN model: FILE_0002_LP_bin-sbb-deskew-crop_primalayout
  • on publaynet (full) Mask-RCNN model (non-empty now): FILE_0002_LP_bin-sbb-deskew-crop_publaynet

I wonder what the masks would look like. Is there a way to query / plot them?

Also, getting back to vertical text and in-domain data, here's a different set of examples:

original on HJDataset model
xueheng-001-000 xueheng-001-000_lpboxes
xueheng-002-001_2L xueheng-002-001_2L_lpboxes
xueheng-003-002_1R xueheng-003-002_1R_lpboxes
xueheng-004-003_1R xueheng-004-003_1R_lpboxes

Is that expected output, too? Why do I get the feeling I am making some user-side mistake?

bertsky avatar Feb 01 '21 18:02 bertsky

(In fairness, I do get good looking results on PubLayNet itself. Here are 3 images from the validation subset:

original image on publaynet (large) model
PMC4012781_00001 PMC4012781_00001_lpboxes
PMC5293783_00019 PMC5293783_00019_lpboxes
PMC6095085_00000 PMC6095085_00000_lpboxes

Looking forward to your expert opinion on my above question how this could become a base model for other domains...

bertsky avatar Feb 01 '21 18:02 bertsky

Hi. Thanks for this library. The idea seems awesome. However, I still haven't got the chance to use it as I'm having issues with the conflicted dependencies stated above.

Also, I'm having issues installing pycocotools on windows10.

ERROR: Failed building wheel for pycocotools
Running setup.py clean for pycocotools
Failed to build pycocotools
Installing collected packages: pycocotools, opencv-python, fvcore, layoutparser
Attempting uninstall: pycocotools
Found existing installation: pycocotools 2.0
Uninstalling pycocotools-2.0:
Successfully uninstalled pycocotools-2.0
Running setup.py install for pycocotools ... error

It seems that for pycocotools the fix in windows 10 is: https://github.com/cocodataset/cocoapi/issues/169

Thanks for reporting this issue! There has been some updates for Detection, and the support for fvcore is somewhat problematic in the current version. I plan to remove the fvcore dependency in the next major release, which should be around within 1 month.

Any update on this?

Thank you!

ieriii avatar Feb 26 '21 17:02 ieriii

Thanks for reporting the issue for installing pycocotools on win10 - Let me try to fix that.

Speaking on the next major update, we are currently working on more features and it should be a huge update! Please stay tuned and we will release it very soon!

lolipopshock avatar Mar 01 '21 15:03 lolipopshock

The installation issue should be fixed in the latest commit. However, as it hasn't been pushed to pypi, you might want to install via pip install "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser". See also #11 .

lolipopshock avatar Mar 30 '21 02:03 lolipopshock

There are still some issues with "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser".

I've a usable configuration with: fvcore==0.1.3.post20210310 torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html detectron2 0.3

CrisChir avatar Apr 08 '21 14:04 CrisChir

There are still some issues with "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser".

I've a usable configuration with: fvcore==0.1.3.post20210310 torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html detectron2 0.3

Thanks for reporting! Could you share the exact issues for using pip install "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser" for installation and I could look into that? Thanks!

lolipopshock avatar Apr 08 '21 14:04 lolipopshock

There are still some issues with "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser". I've a usable configuration with: fvcore==0.1.3.post20210310 torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html detectron2 0.3

Thanks for reporting! Could you share the exact issues for using pip install "git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser" for installation and I could look into that? Thanks!

#a message that a library is requiring fvcore==0.1.2 (detectron2 04 i think) or a ModuleNotFoundError (sorry for not saving the errors , but right now i'm afraid to replicate the configuration :) Any tentative to update fvcore or dectectron is giving the following messages: layoutparser 0.1.3 requires fvcore==0.1.1.post20200623, but you have fvcore 0.1.5.post20210402 which is incompatible. layoutparser 0.1.3 requires pycocotools==2.0.1, but you have pycocotools 2.0.2 which is incompatible. still it seems tow work with fvcore above 0.1.2 After installing the above libraries (fvcore1.1.3 detectron2 03) there are only warnings: ** fvcore version of PathManager will be deprecated soon. ** ** Please migrate to the version in iopath repo. ** https://github.com/facebookresearch/iopath

/home/user/anaconda3/lib/python3.8/site-packages/detectron2/modeling/roi_heads/fast_rcnn.py:124: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.) filter_inds = filter_mask.nonzero()

CrisChir avatar Apr 09 '21 09:04 CrisChir

Hello @lolipopshock, I believe we have the same issue with @CrisChir. When i installed Layour-parser with "pip install layoutparser", it's downloads fvcore==0.1.1.post20200623. when I installed detectron2 with "pip install git+https://github.com/facebookresearch/detectron2.git#egg=detectron2", it's downloads fvcore-0.1.5.post20210402. And the error occur as below:

ERROR: layoutparser 0.1.3 has requirement fvcore==0.1.1.post20200623, but you'll have fvcore 0.1.5.post20210402 which is incompatible. ERROR: layoutparser 0.1.3 has requirement pycocotools==2.0.1, but you'll have pycocotools 2.0.2 which is incompatible.

Also while I am trying to download the model with "model = lp.Detectron2LayoutModel('lp://PrimaLayout/mask_rcnn_R_50_FPN_3x/config')" I got the error as below

Traceback (most recent call last): File "", line 1, in File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 53, in new cls._import_module() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 39, in _import_module if importlib.util.find_spec(m["module_path"]): File "/usr/lib/python3.8/importlib/util.py", line 94, in find_spec parent = import(parent_name, fromlist=['path']) File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/init.py", line 5, in setup_environment() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 108, in setup_environment _configure_libraries() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 88, in _configure_libraries assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2" AssertionError: Requires fvcore>=0.1.2

could you suggest something to handle this error, please?

serdarakyol avatar Apr 09 '21 11:04 serdarakyol

Hello @lolipopshock, I believe we have the same issue with @CrisChir. When i installed Layour-parser with "pip install layoutparser", it's downloads fvcore==0.1.1.post20200623. when I installed detectron2 with "pip install git+https://github.com/facebookresearch/detectron2.git#egg=detectron2", it's downloads fvcore-0.1.5.post20210402. And the error occur as below:

ERROR: layoutparser 0.1.3 has requirement fvcore==0.1.1.post20200623, but you'll have fvcore 0.1.5.post20210402 which is incompatible. ERROR: layoutparser 0.1.3 has requirement pycocotools==2.0.1, but you'll have pycocotools 2.0.2 which is incompatible.

Also while I am trying to download the model with "model = lp.Detectron2LayoutModel('lp://PrimaLayout/mask_rcnn_R_50_FPN_3x/config')" I got the error as below

Traceback (most recent call last): File "", line 1, in File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 53, in new cls._import_module() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 39, in _import_module if importlib.util.find_spec(m["module_path"]): File "/usr/lib/python3.8/importlib/util.py", line 94, in find_spec parent = import(parent_name, fromlist=['path']) File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/init.py", line 5, in setup_environment() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 108, in setup_environment _configure_libraries() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 88, in _configure_libraries assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2" AssertionError: Requires fvcore>=0.1.2

could you suggest something to handle this error, please?

The workaround was in my case: pip install detectron2==0.3 ( see pypi FOR syntax) since detection 03 does not work with torch 1.8 downgrade torch to 1.7 Also pip install fvcore==1.1.3xyz (also pypi) It will work with warnings( restart kernel after pip :)

CrisChir avatar Apr 09 '21 19:04 CrisChir

Hi @CrisChir , I tried to use your workaround solution but couldn't downgrade detectron2. Command I use and output are:

pip install detectron2==0.3 ERROR: Could not find a version that satisfies the requirement detectron2==0.3 (from versions: none) ERROR: No matching distribution found for detectron2==0.3

How did you downgrade detectron2?

ermissa avatar Apr 09 '21 20:04 ermissa

This wroked for me to install detrectron v03: git clone --branch v0.3 https://github.com/facebookresearch/detectron2.git external/detectron2 pip install external/detectron2

maxml7331 avatar Apr 09 '21 23:04 maxml7331

Hello @lolipopshock, I believe we have the same issue with @CrisChir. When i installed Layour-parser with "pip install layoutparser", it's downloads fvcore==0.1.1.post20200623. when I installed detectron2 with "pip install git+https://github.com/facebookresearch/detectron2.git#egg=detectron2", it's downloads fvcore-0.1.5.post20210402. And the error occur as below: ERROR: layoutparser 0.1.3 has requirement fvcore==0.1.1.post20200623, but you'll have fvcore 0.1.5.post20210402 which is incompatible. ERROR: layoutparser 0.1.3 has requirement pycocotools==2.0.1, but you'll have pycocotools 2.0.2 which is incompatible. Also while I am trying to download the model with "model = lp.Detectron2LayoutModel('lp://PrimaLayout/mask_rcnn_R_50_FPN_3x/config')" I got the error as below Traceback (most recent call last): File "", line 1, in File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 53, in new cls._import_module() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 39, in _import_module if importlib.util.find_spec(m["module_path"]): File "/usr/lib/python3.8/importlib/util.py", line 94, in find_spec parent = import(parent_name, fromlist=['path']) File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/init.py", line 5, in setup_environment() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 108, in setup_environment _configure_libraries() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 88, in _configure_libraries assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2" AssertionError: Requires fvcore>=0.1.2 could you suggest something to handle this error, please?

The workaround was in my case: pip install detectron2==0.3 ( see pypi FOR syntax) since detection 03 does not work with torch 1.8 downgrade torch to 1.7 Also pip install fvcore==1.1.3xyz (also pypi) It will work with warnings( restart kernel after pip :)

@CrisChir: I still get an error even with exact the versions you mentioned ("DLL load failed while importing _C: The specified procedure could not be found.)" when doing model = lp.Detectron2.... . Could you double-check the versions? I still cannot make it work :(

maxml7331 avatar Apr 10 '21 00:04 maxml7331

Hello @lolipopshock, I believe we have the same issue with @CrisChir. When i installed Layour-parser with "pip install layoutparser", it's downloads fvcore==0.1.1.post20200623. when I installed detectron2 with "pip install git+https://github.com/facebookresearch/detectron2.git#egg=detectron2", it's downloads fvcore-0.1.5.post20210402. And the error occur as below: ERROR: layoutparser 0.1.3 has requirement fvcore==0.1.1.post20200623, but you'll have fvcore 0.1.5.post20210402 which is incompatible. ERROR: layoutparser 0.1.3 has requirement pycocotools==2.0.1, but you'll have pycocotools 2.0.2 which is incompatible. Also while I am trying to download the model with "model = lp.Detectron2LayoutModel('lp://PrimaLayout/mask_rcnn_R_50_FPN_3x/config')" I got the error as below Traceback (most recent call last): File "", line 1, in File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 53, in new cls._import_module() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/layoutparser/models/layoutmodel.py", line 39, in _import_module if importlib.util.find_spec(m["module_path"]): File "/usr/lib/python3.8/importlib/util.py", line 94, in find_spec parent = import(parent_name, fromlist=['path']) File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/init.py", line 5, in setup_environment() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 108, in setup_environment _configure_libraries() File "/home/ak/Desktop/parser/env_parser/lib/python3.8/site-packages/detectron2/utils/env.py", line 88, in _configure_libraries assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2" AssertionError: Requires fvcore>=0.1.2 could you suggest something to handle this error, please?

The workaround was in my case: pip install detectron2==0.3 ( see pypi FOR syntax) since detection 03 does not work with torch 1.8 downgrade torch to 1.7 Also pip install fvcore==1.1.3xyz (also pypi) It will work with warnings( restart kernel after pip :)

@CrisChir: I still get an error even with exact the versions you mentioned ("DLL load failed while importing _C: The specified procedure could not be found.)" when doing model = lp.Detectron2.... . Could you double-check the versions? I still cannot make it work :( I will check later today. still : YOU are using Windows or Linux ? Conda?

CrisChir avatar Apr 10 '21 04:04 CrisChir

I'm using Windows and conda, thanks!

maxml7331 avatar Apr 10 '21 04:04 maxml7331

I'm using Windows and conda, thanks! my error is something like this: ModuleNotFoundError: No module named 'fvcore.nn.distributed'

In jupyter notebook (use separate environment things can get messy:)

Workaround:

It is possible to have issues with windows but you can get a try: So jupyter notebook import sys print(sys.version)

Install a pip package in the current Jupyter kernel

import sys !{sys.executable} -m pip install fvcore==0.1.3.post20210310

Install a pip package in the current Jupyter kernel

import sys !{sys.executable} -m pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html import sys !{sys.executable} -m pip install detectron2==0.3 -f
https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/torch1.7/index.html Check the installation after restarting the kernel import fvcore print(fvcore.version) #0.1.3.post20210310 import detectron2 print(detectron2.version) #0.3 import torch print(torch.version) #1.7.1+cpu ##using pip simple is installing the packages in all environments #the alternative way is to install packages via conda in a dedicated environment but I have some issue with that in Linux Mint:)

CrisChir avatar Apr 10 '21 07:04 CrisChir

When trying to install the latest version of detectron2 (0.4) from this URL (https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.8/detectron2-0.4%2Bcu111-cp39-cp39-linux_x86_64.whl) I get this error:

  SolverProblemError

  Because detectron2 (0.4+cu111) depends on pycocotools (>=2.0.2)
   and layoutparser (0.1.3) depends on pycocotools (2.0.1), detectron2 (0.4+cu111) is incompatible with layoutparser (0.1.3).
  And because no versions of layoutparser match >0.1.3,<0.2.0, detectron2 (0.4+cu111) is incompatible with layoutparser (>=0.1.3,<0.2.0).
  So, because german-tools depends on both layoutparser (^0.1.3) and detectron2 (0.4+cu111), version solving failed.

The message looks a bit differently than the other reports here because I'm using Poetry, but the problem is the same.

When installing from master, this is the error I get:

Because detectron2 (0.4+cu111) depends on fvcore (>=0.1.3,<0.1.4)
   and layoutparser (rev master) depends on fvcore (0.1.1.post20200623), detectron2 (0.4+cu111) is incompatible with layoutparser (rev master).
  So, because german-tools depends on both layoutparser (branch master) and detectron2 (0.4+cu111), version solving failed.

I'm trying to install it on Manjaro Linux with Python 3.9.

edisongustavo avatar Apr 10 '21 10:04 edisongustavo

In #18, the dependency on fvcore has been removed. And I plan to fix the Detectron2 version as v0.3 in the installation as well. Could you try again with the current following installation scripts and see if it can fix your issues? @edisongustavo @CrisChir @maxml7331 @ermissa

pip install -U 'git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser'
pip install -U 'git+https://github.com/facebookresearch/[email protected]#egg=detectron2' 

Also Kudos to @edisongustavo for the help in #18.

lolipopshock avatar Apr 10 '21 20:04 lolipopshock

FYI, I am now able to install the library with:

$ poetry add git+https://github.com/Layout-Parser/layout-parser.git https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.8/detectron2-0.4%2Bcu111-cp39-cp39-linux_x86_64.whl

Looking forward for the new release :)

edisongustavo avatar Apr 11 '21 08:04 edisongustavo