Error: No such file or directory while moving

[fabiendostie]

Problem

Hi everyone, Doing a complete re-import of my whole music collection. Been retagging, renaming modifying it for years now 6+ I feel like now config is good, and all those good things are happening almost automaticagically....

My problem is that, when I do a "test folder" run I get these No such file or directory while moving errors. This has been a recurring issue on my system. I then go on the NAS and look for the faulty files or folder... and most of the time they are there. I then proceed to just move them elsewhere in a "to check" folder... or simply delete the damn files... then redo my beet -vv import /Volumes/music/CleanBeet

so it runs for a while until it hits another...

can someone take a look at this please?

Running this command in verbose (-vv) mode:

...
Sending event: database_change
/Volumes/music/CleanBeet/E/electronic trance hip jaz/Late of the Pier/The Bears Are Coming - [ALBUM] (MP3)(2008) - (3_23)
Sending event: import_task_created
Replacing item 12044: /Volumes/music/CleanBeet/E/electronic trance hip jaz/Late of the Pier/The Bears Are Coming - [ALBUM] (MP3)(2008) - (3_23)/04 The Bears Are Coming [217kbps].mp3
Sending event: database_change
Sending event: database_change
Sending event: album_removed
Sending event: item_removed
1 of 1 items replaced
Sending event: database_change
Sending event: import_task_created
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Reimported album: added 1662737721.927134, flexible attributes [] from album 4082 for /Volumes/music/CleanBeet/E/electronic trance hip jaz/Late of the Pier/The Bears Are Coming - [ALBUM] (MP3)(2008) - (3_23)
Reimported item added 1662737721.950891 from item 12044 for /Volumes/music/CleanBeet/E/electronic trance hip jaz/Late of the Pier/The Bears Are Coming - [ALBUM] (MP3)(2008) - (3_23)/04 The Bears Are Coming [217kbps].mp3
Reimported item flexible attributes [] from item 12044 for /Volumes/music/CleanBeet/E/electronic trance hip jaz/Late of the Pier/The Bears Are Coming - [ALBUM] (MP3)(2008) - (3_23)/04 The Bears Are Coming [217kbps].mp3
Sending event: database_change
Traceback (most recent call last):
  File "/Users/lefab/beets/beets/util/__init__.py", line 494, in move
    os.replace(syspath(path), syspath(dest))
OSError: [Errno 18] Cross-device link: b'/Volumes/music/CleanBeet/E/electronic trance hip jaz/Compilations/Richard Dorfmeister Presents_ A Different Drummer Selection/01 Imagination.1.mp3' -> b'/Volumes/Music_Collection/Sorted_Music/Richard Dorfmeister/Richard Dorfmeister Presents_ A Different Drummer Selection [4211]/01 Imagination.mp3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/lefab/beets/beets/util/__init__.py", line 515, in move
    os.remove(syspath(path))
FileNotFoundError: [Errno 2] No such file or directory: b'/Volumes/music/CleanBeet/E/electronic trance hip jaz/Compilations/Richard Dorfmeister Presents_ A Different Drummer Selection/01 Imagination.1.mp3'

Error: No such file or directory while moving /Volumes/music/CleanBeet/E/electronic trance hip jaz/Compilations/Richard Dorfmeister Presents_ A Different Drummer Selection/01 Imagination.1.mp3 to /Volumes/Music_Collection/Sorted_Music/Richard Dorfmeister/Richard Dorfmeister Presents_ A Different Drummer Selection [4211]/01 Imagination.mp3

Setup

• OS: MacOS 10.15.7

• beets version: 1.6.1

• Python version: 3.10.6

• no plugins loaded

• Turning off plugins made problem go away (yes/no): no (no plugins loaded

My configuration (output of beet config) is:

directory: /Volumes/Music_Collection/Sorted_Music
library: ~/.config/beets/musiclibrary.db

import:
    move: yes
    copy: no
    write: no
    resume: yes
    incremental: yes
    quiet_fallback: skip
    timid: no
    default_action: apply
    autotag: no
    duplicate_action: merge

I don't understand, I am using the most simplest way to do this and I still get this error.

I need help, thanks

[sampsyo]

Wow; this is quite strange! Diving into the code, what's happening here is that, since you have import.move enabled, beets first tries just moving the file: https://github.com/beetbox/beets/blob/50bd693057de472470ab5175fae0cdb5b75811c6/beets/util/init.py#L494

That doesn't work, obviously, because we're going from one volume to another. So it switches to the copy-and-delete method of "moving." The first step is to copy the data from the original file, path, to a temporary: https://github.com/beetbox/beets/blob/50bd693057de472470ab5175fae0cdb5b75811c6/beets/util/init.py#L506-L507

That seems to be successful here! Then, beets just needs to move the temporary file and delete the original: https://github.com/beetbox/beets/blob/50bd693057de472470ab5175fae0cdb5b75811c6/beets/util/init.py#L513-L515

The move (os.replace) is again successful; the only thing that fails is the last step, removing the original file.

First of all, I think we should probably catch and suppress FileNotFoundError on this step. If the file is already gone when we try to delete it, it's probably benign—even if it's mysterious.

However, it remains very weird that the original copy works—indicating that path definitely exists—and then the deletion doesn't, saying that path no longer exists. Is there any chance that this is something like a "re-import" (importing something that is already in the beets library) or a "replacement" (i.e., the beets importer flagged it as a duplicate)?

[wisp3rwind]

Sounds spot-on — I'm also stumped what could be going on here, other than a race with some other application or another beets thread.

First of all, I think we should probably catch and suppress FileNotFoundError on this step. If the file is already gone when we try to delete it, it's probably benign—even if it's mysterious.

:+1:

[wisp3rwind]

Is it possible that open(syspath(path), 'rb') behaves differently under MacOS, and doesn't fail but reads an empty file?

@fabiendostie: Any chance that you could post the entire output from this command (e.g. as a Github gist if it's otherwise too long)? Might be useful to follow the entire import flow leading up to the error, if this is indeed an issue with the re-import process.

[fabiendostie]

Yes of course I'll put it on later today.

• It's a re-import of only a small part of my whole collection I'm attempting here with the end goal obviously being the whole collection.
• It's not supposed to read from an old database. New state.pickle & .db I'm pretty sure...

I think anyways

[fabiendostie]

here you go @wisp3rwind & @sampsyo : https://gist.github.com/fabiendostie/a74feda190acb5225458788903538d51

Thank you so much for your dedication and time. been a big fan of yours for years!

[sampsyo]

Got it; thanks! Given these messages in the log:

Reimported album: added 1662757836.884819, flexible attributes [] from album 5690 for /Volumes/music/CleanBeet/E/electronic_ambient_hip_jaz/Jurassic 5/Jurassic 5 EP [ep] - [EP] (MP3)(1999) - (4_08)
Reimported item added 1662757836.904449 from item 14624 for /Volumes/music/CleanBeet/E/electronic_ambient_hip_jaz/Jurassic 5/Jurassic 5 EP [ep] - [EP] (MP3)(1999) - (4_08)/05 Concrete Schoolyard.mp3
Reimported item flexible attributes ['ffmpeg', 'ffmpeg'] from item 14624 for /Volumes/music/CleanBeet/E/electronic_ambient_hip_jaz/Jurassic 5/Jurassic 5 EP [ep] - [EP] (MP3)(1999) - (4_08)/05 Concrete Schoolyard.mp3

It looks like this is indeed a re-import, as you said—that is, these files are already in the library database, and you're importing them again. I'm guessing that's the source of the bug, just because the logic around this stuff gets super complicated (more complicated than importing files "fresh") and is more likely to be incorrect. I still don't have an unambiguous diagnosis, but I suppose it would be helpful to confirm that the problem is with the re-import…

Any chance you could try importing one of these "problem" files with a completely fresh database and see if the same thing happens?

[fabiendostie]

I will try to do that, when I'm.out of the hospital. Just to make sure: By that, you mean removing the state.pickle file and the .db file and putting them somewhere safe ;) and start a new import from scratch right?

That will take forever but I guess it might be faster that what I have to do right now.

I'll give it a try

Thanks for the help