coremltools icon indicating copy to clipboard operation
coremltools copied to clipboard

Error: resources.bin missing at file

Open rustui opened this issue 7 months ago • 15 comments

python[1656:43980] Error: resources.bin missing at file:///Users/xx/Library/Caches/python/com.apple.e5rt.e5bundlecache/24E263/8985626FC71D08A4A0ED36AAB769024A5F51CEA9866F1A20AF5627636E69C6D1/BBDE20036A52BB546C66A5D3F31990D7AC209EC8FB653D319F2EA463CA80D5B1.bundle/H13G.bundle/main_default_function/main_default_function_mps_graph/main_default_function_mps_graph.mpsgraphpackage/resources.bin

python[1656:44033] Error: resources.bin missing at file:///Users/xx/Library/Caches/python/com.apple.e5rt.e5bundlecache/24E263/8985626FC71D08A4A0ED36AAB769024A5F51CEA9866F1A20AF5627636E69C6D1/BBDE20036A52BB546C66A5D3F31990D7AC209EC8FB653D319F2EA463CA80D5B1.bundle/H13G.bundle/main_default_function/main_default_function_mps_graph/main_default_function_mps_graph.mpsgraphpackage/resources.bin

During a model conversion process, the disk became full. Since then, I’ve been consistently encountering the same error when trying to convert models again, even though I’ve freed up plenty of space on the disk, cleared all caches, and reinstalled Conda.

rustui avatar May 09 '25 15:05 rustui

I am still encountering the same error message after reinstalling the operating system.

coremltools: 8.3 macOS: 15.4.1

rustui avatar May 10 '25 06:05 rustui

I would probably try deleting everything under /Users/xx/Library/Caches/python/com.apple.e5rt.e5bundlecache/ and then restart your machine. It's not clear to me if you've already done that.

Do you know at what point in the conversion it failed? Was it during the actual conversion process or at the end when we load the converted model to be returned by ct.convert?

I think this is unlikely to be an issue with the Coremltools Python Package. This seems like an issue with the Operating System. So this probably isn't the best forum to you to get a solution.

I suggest you file a bug report here: https://developer.apple.com/bug-reporting/ And create a post in the forum: https://forums.developer.apple.com/forums/tags/core-ml

TobyRoseman avatar May 12 '25 19:05 TobyRoseman

Yes, I did delete that folder, but I didn't restart the system then. Later, I erased the entire hard drive and reinstalled the OS, but I still encountered the same error.

The error occurs after the conversion completes, specifically after the usual three progress stages printed by coremltools finish. I was only performing the conversion, not loading the model afterwards (ct.convert only).

Despite the error, the model file was successfully generated. I could even load it and run inference in Xcode, although I haven't verified if the inference results are completely correct.

rustui avatar May 13 '25 00:05 rustui

Unless you pass skip_model_load=True to ct.convert, the saved model is loaded at the end of the ct.convert call and returned.

Do you also get this error when you load your saved model?

TobyRoseman avatar May 13 '25 17:05 TobyRoseman

Thanks for the insight! After testing, I can confirm the error occurs when the model is loaded, rather than during the conversion (use skip_model_load=True).

rustui avatar May 17 '25 04:05 rustui

Also seeing this with macOS 15.5, coremltools 8.3, and Xcode 16.4 RC1.

And as well as seeing this with running the inference in python with coremltools, it also pops up if I take the model from coremltools and try to run it with a minimal piece of ObjC for inference.

RubeGoldbergJr avatar May 18 '25 23:05 RubeGoldbergJr

@RubeGoldbergJr - did you also run out of disk space right before this issue started happening?

Can anyone give us steps to reproduce the issue?

TobyRoseman avatar May 19 '25 20:05 TobyRoseman

I didn't run out of disk space (5TB free).

A few other things I've noticed:

  • when running the model through the Xcode performance analysis thing, it runs fine (no espresso errors in the logs), but when run with either coremltools or CoreML framework from obj-C, it has the weird missing resource.bin on load and the espresso errors, and then runs at only ~half the speed vs. the Xcode performance analysis thing (not sure if it is even running correctly though);
  • it is model specific. @rustui are you able to share your model? I can't share mine directly, but will look at figuring out a minimal reproduction.

RubeGoldbergJr avatar May 20 '25 17:05 RubeGoldbergJr

(deleted this message -- I realized I was looking at an older .mil file from previous Xcode, and coremlc versions in all paths are all in fact the same so that is nothing to do with it)

RubeGoldbergJr avatar May 20 '25 17:05 RubeGoldbergJr

@RubeGoldbergJr - did you also run out of disk space right before this issue started happening?

Can anyone give us steps to reproduce the issue?

Steps to reproduce

  1. Download the compiled coreml-sdv2.1 unet weights.

  2. Load unet.

    from time import perf_counter
    import coremltools as ct
    
    tick = perf_counter()
    mlmodel = ct.models.CompiledMLModel("/path/to/Unet.mlmodelc")
    print("time taken to load using ct.models.MLModel: {:.1f} secs".format(perf_counter() - tick))
    

System info

  • MacOS 15.5
  • Xcode 16.4, Build version 16F6
  • coremltools 8.3.0

I also noticed that after the first load, the subsequent loads did become faster(133.6 secs -> 44.1 secs), which means the cache mechanism does work.

zhangmxxx avatar Jun 13 '25 07:06 zhangmxxx

Just encountered this myself, using a training script we've been using for a few months, most recent training run failed during export with this error (not much changed about the training/export process, just changed the training data really, and plenty of disk space). Investigating and will report back if I find anything.

jameswilddev avatar Jul 01 '25 08:07 jameswilddev

Deleting the Python cache seems to have fixed it my end.

jameswilddev avatar Jul 01 '25 08:07 jameswilddev

I've also been encountering this. @jameswilddev oddly I only encounter this when I run in XCode and not when I load in Python with ct.models.MLModel. Do you only encounter this when trying to run the compiled version?

ajayarora1235 avatar Jul 01 '25 15:07 ajayarora1235

The issue has come back again and corrupted a dozen-hour training run. Definitely something wrong here.

jameswilddev avatar Jul 02 '25 07:07 jameswilddev

Same here:

2025-07-07 18:32:13.746 Python[2788:13763] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph1/main_mps_graph1.mpsgraphpackage/resources.bin
2025-07-07 18:32:13.747 Python[2788:13765] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph9/main_mps_graph9.mpsgraphpackage/resources.bin
2025-07-07 18:32:13.748 Python[2788:13766] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph10/main_mps_graph10.mpsgraphpackage/resources.bin
2025-07-07 18:32:13.749 Python[2788:13762] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph11/main_mps_graph11.mpsgraphpackage/resources.bin
2025-07-07 18:32:13.749 Python[2788:13764] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph12/main_mps_graph12.mpsgraphpackage/resources.bin
2025-07-07 18:32:13.750 Python[2788:12837] Error: resources.bin missing at file:///Users/devuser/Library/Caches/org.python.python/com.apple.e5rt.e5bundlecache/24F74/716004CF443CEFD5ECBB316800742769F7F1DD45964D3F1704ED4F07E42C59B3/6835E6A2E2DA18231F502603187E13E8BF92AD1B1734DC1FA7A4E1BD3EE6AFB3.bundle/H14S.bundle/main/main_mps_graph13/main_mps_graph13.mpsgraphpackage/resources.bin

daniil-lyakhov avatar Jul 07 '25 16:07 daniil-lyakhov