Network errors abort import process

Open 9999years opened this issue 5 months ago • 1 comments

Problem

If beets encounters a (transient) network error, the entire import process is aborted. It would be nice if beets instead skipped the particular files that failed and continued with the import.

When running beet import ~/Music/Music/ --quiet --quiet-fallback=skip --incremental:

2025-07-29 00:35:14 musicbrainzngs.musicbrainz DEBUG: GET request for https://musicbrainz.org/ws/2/release/f11c115e-afe8-45ab-a94f-91cc92656fe4?inc=isrcs+release-groups+artist-credits+release-rels+artist-rels+tags+aliases+work-level-rels+recording-level-rels+artists+work-rels+url-rels+media+labels+recordings
2025-07-29 00:50:44 beets ERROR: Error: MusicBrainz not reachable in get release by ID with query 'f11c115e-afe8-45ab-a94f-91cc92656fe4'
2025-07-29 00:50:44 beets DEBUG: Traceback (most recent call last):
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 501, in _safe_read
    f = opener.open(req, timeout=timeout)
  ...
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/lfv9ns20hz2bg6d44js378vcxjfm9261-beets-2.3.1/lib/python3.13/site-packages/beetsplug/musicbrainz.py", line 845, in album_for_id
    res = musicbrainzngs.get_release_by_id(albumid, RELEASE_INCLUDES)
  ...
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 536, in _safe_read
    raise NetworkError(cause=exc)
musicbrainzngs.musicbrainz.NetworkError: caused by: [Errno 54] Connection reset by peer

Debug logs / stack trace

2025-07-29 00:35:14 musicbrainzngs.musicbrainz DEBUG: GET request for https://musicbrainz.org/ws/2/release/f11c115e-afe8-45ab-a94f-91cc92656fe4?inc=isrcs+release-groups+artist-credits+release-rels+artist-rels+tags+aliases+work-level-rels+recording-level-rels+artists+work-rels+url-rels+media+labels+recordings
2025-07-29 00:35:14 musicbrainzngs.musicbrainz DEBUG: requesting with UA beets/2.3.1 python-musicbrainzngs/0.7.1 ( https://beets.io/ )
2025-07-29 00:50:44 beets DEBUG: Traceback (most recent call last):
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 501, in _safe_read
    f = opener.open(req, timeout=timeout)
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/urllib/request.py", line 489, in open
    response = self._open(req, data)
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/urllib/request.py", line 506, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/urllib/request.py", line 1367, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        context=self._context)
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/urllib/request.py", line 1323, in do_open
    r = h.getresponse()
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/http/client.py", line 1430, in getresponse
    response.begin()
    ~~~~~~~~~~~~~~^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ~~~~~~~~~~~~~~~~~^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/socket.py", line 719, in readinto
    return self._sock.recv_into(b)
           ~~~~~~~~~~~~~~~~~~~~^^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/ssl.py", line 1304, in recv_into
    return self.read(nbytes, buffer)
           ~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/nix/store/riv3i1wrig5kaf89wgpx46581f71gfqb-python3-3.13.5/lib/python3.13/ssl.py", line 1138, in read
    return self._sslobj.read(len, buffer)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/lfv9ns20hz2bg6d44js378vcxjfm9261-beets-2.3.1/lib/python3.13/site-packages/beetsplug/musicbrainz.py", line 845, in album_for_id
    res = musicbrainzngs.get_release_by_id(albumid, RELEASE_INCLUDES)
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 885, in get_release_by_id
    return _do_mb_query("release", id, includes, params)
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 734, in _do_mb_query
    return _mb_request(path, 'GET', auth_required, args=args)
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 421, in __call__
    return self.fun(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 694, in _mb_request
    resp = _safe_read(opener, req, body)
  File "/nix/store/f1jvr1vx9mwnj5sp7azx9mf0y52jcws5-python3.13-musicbrainzngs-0.7.1/lib/python3.13/site-packages/musicbrainzngs/musicbrainz.py", line 536, in _safe_read
    raise NetworkError(cause=exc)
musicbrainzngs.musicbrainz.NetworkError: caused by: [Errno 54] Connection reset by peer

2025-07-29 00:50:44 beets ERROR: Error: MusicBrainz not reachable in get release by ID with query 'f11c115e-afe8-45ab-a94f-91cc92656fe4'
2025-07-29 00:50:44 httpcore.connection DEBUG: close.started
2025-07-29 00:50:44 httpcore.connection DEBUG: close.complete
2025-07-29 00:50:44 httpcore.connection DEBUG: close.started
2025-07-29 00:50:44 httpcore.connection DEBUG: close.complete

#5163
#5156
#3986
#5105
#2330
#1926
#306

Setup

OS: macOS Sequoia 15.5 (24F74)
Python version: 3.13.5
beets version: master as of 0fec858a13854dd16cba5fcd6e698da148f4672d (2.3.1), plus some patches:
- https://github.com/beetbox/beets/pull/5893
- https://github.com/beetbox/beets/pull/5898
- https://github.com/beetbox/beets/pull/5897
Turning off plugins made problem go away (yes/no): N/A, probably yes if beets doesn't make network requests unless plugins are enabled? This particular stack trace is for the MusicBrainz integration, but I believe the other API bindings are vulnerable to this issue as well.

My configuration (output of beet config) is:

directory: ~/Music/beets

# --------------- Plugins ---------------

plugins: bandcamp chroma convert discogs duplicates embedart fetchart fromfilename lastgenre lyrics spotify

# --------------- UI ---------------

verbose: yes
logging:
    format: '{asctime} {name} {levelname}: {message}'
    levels:
        beets.plugins: INFO
        musicbrainzngs.mbxml: WARNING
lyrics:
    auto: yes
    sources: lrclib genius tekstowo
    synced: yes
    translate:
        api_key: REDACTED
        from_languages: []
        to_language:
    dist_thresh: 0.11
    google_API_key: REDACTED
    google_engine_ID: REDACTED
    genius_api_key: REDACTED
    fallback:
    force: no
    local: no
    print: no

import:
    move: yes
    copy: no
    duplicate_action: skip
    log: ~/.cache/beets/beets.log
    languages: [en]
    group_albums: yes
musicbrainz:
    external_ids:
        discogs: yes
        spotify: yes
        bandcamp: yes
        deezer: no
        tidal: no
    source_weight: 0.5
    host: musicbrainz.org
    https: no
    ratelimit: 1
    ratelimit_interval: 1
    searchlimit: 5
    genres: no
    extra_tags: []
chroma:
    auto: yes
    source_weight: 0.5
convert:
    delete_originals: yes
    formats:
        mp3: ffmpeg -y -i $source -c:a libmp3lame -q:a 0 -map_metadata 0 -map_metadata 0:s:0 $dest
        aac:
            command: ffmpeg -i $source -y -vn -acodec aac -aq 1 $dest
            extension: m4a
        alac:
            command: ffmpeg -i $source -y -vn -acodec alac $dest
            extension: m4a
        flac: ffmpeg -i $source -y -vn -acodec flac $dest
        opus: ffmpeg -i $source -y -vn -acodec libopus -ab 96k $dest
        ogg: ffmpeg -i $source -y -vn -acodec libvorbis -aq 3 $dest
        wma: ffmpeg -i $source -y -vn -acodec wmav2 -vn $dest
    dest:
    pretend: no
    link: no
    hardlink: no
    threads: 12
    format: mp3
    id3v23: inherit
    max_bitrate:
    auto: no
    auto_keep: no
    tmpdir:
    quiet: no
    embed: yes
    paths: {}
    no_convert: ''
    never_convert_lossy_files: no
    copy_album_art: no
    album_art_maxwidth: 0
    playlist:
bandcamp:
    source_weight: 0.5
    art: yes
    comments_separator: "\n\u2014\u2014\u2014\n"
    genre:
        capitalize: yes
        mode: progressive
        maximum: 0
        always_include: []
    include_digital_only_tracks: yes
    search_max: 2
    exclude_extra_fields: []
    truncate_comments: no
fetchart:
    cautious: yes
    sources: itunes *
    auto: yes
    minwidth: 0
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    store_source: no
    high_resolution: no
    deinterlace: no
    cover_format:
    google_key: REDACTED
    google_engine: REDACTED
    lastfm_key: REDACTED
    fanarttv_key: REDACTED
duplicates:
    album: yes
    merge: yes
    checksum: ''
    copy: ''
    count: no
    delete: no
    format: ''
    full: no
    keys: []
    move: ''
    path: no
    tiebreak: {}
    strict: no
    tag: ''
    remove: no
lastgenre:
    canonical: yes
    count: 5
    force: yes
    min_weight: 10
    separator: '; '
    whitelist: yes
    fallback:
    source: album
    keep_existing: no
    auto: yes
    prefer_specific: no
    title_case: yes
    extended_debug: no
embedart:
    maxwidth: 0
    auto: yes
    compare_threshold: 0
    ifempty: no
    remove_art_file: no
    quality: 0
spotify:
    source_weight: 0.5
    mode: list
    tiebreak: popularity
    show_failures: no
    artist_field: albumartist
    album_field: album
    track_field: title
    region_filter:
    regex: []
    client_id: REDACTED
    client_secret: REDACTED
    tokenfile: spotify_token.json
    search_query_ascii: no
discogs:
    source_weight: 0.5
    apikey: REDACTED
    apisecret: REDACTED
    tokenfile: discogs_token.json
    user_token: REDACTED
    separator: ', '
    index_tracks: no
    append_style_genre: no
    search_limit: 5

Jul 29 '25 15:07 9999years

This crash lost 15 hours of progress importing music :( I'm surprised the import process, even with --incremental, is so prone to losing data.

Jul 29 '25 16:07 9999years