ao icon indicating copy to clipboard operation
ao copied to clipboard

Use `importlib.util.find_spec` to check if `lm_eval` is installed instead of trying to import it

Open ringohoffman opened this issue 1 year ago • 2 comments

There is a circular dependency when trying to import lm_eval inside torchao. The chain is like this:

torchao -> lm_eval -> transformers.pipelines -> torchao

And results in the following error:

RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback): cannot import name 'quantize_' from partially initialized module 'torchao.quantization'

which

  1. causes _lm_eval_available to be erroneously set to False, even if lm_eval is available
  2. interrupts lm_eval's initialization, leaving it partially initialized

you can observe this with:

>>> import torchao
>>> import lm_eval.__main__
>>> import lm_eval.api.registry
>> lm_eval.api.registry
AttributeError: module 'lm_eval' has no attribute 'api'

Having a bare except clause here was suppressing this circular import error, which from glancing around seems kind of like a general pattern in this code base. It might be worth reconsidering this pattern.

ringohoffman avatar Oct 07 '24 00:10 ringohoffman

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1023

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: No Failures

As of commit 198ac0d41cbe24dd55f4a67faf5c60c8dfdefc69 with merge base c187f876d2f15c02b350f5e1c50855b485132a55 (image): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorch-bot[bot] avatar Oct 07 '24 00:10 pytorch-bot[bot]

Yeah not a huge fan of naked excepts either, if you wanna submit a few patches, would be happy to merge them

msaroufim avatar Oct 07 '24 01:10 msaroufim