Expand zero plugin to read any given tag

[2V3EvG4LMJFdRe]

Problem

The zero plugin, per documentation, should erase every tag that isn't listed under keep_fields:. However, on my end, it keeps a bunch of tags regardless.

-vv doesn't really tell me anything beyond listing a bunch of fields like this:

$ beet -vv zero mew
user configuration: /Users/redacted/.config/beets/config.yaml
data directory: /Users/redacted/.config/beets
plugin paths: 
Sending event: pluginload
artresizer: method is (2, (7, 0, 10), False)
library database: /Users/redacted/.config/beets/library.db
library directory: /Volumes/Expansion-1/Media/Audio/Sound
Sending event: library_opened
zero: genres:  -> None
zero: genre:  -> None
zero: lyricist:  -> None
zero: composer_sort:  -> None
zero: arranger:  -> None
zero: grouping:  -> None
zero: comments:  -> None
zero: bpm: 0 -> None
zero: comp: 0 -> None
zero: artist_sort:  -> None
zero: albumartist_sort:  -> None
zero: asin:  -> None
zero: catalognum:  -> None
zero: barcode:  -> None
zero: disctitle:  -> None
zero: encoder:  -> None
zero: script:  -> None
zero: language:  -> None
zero: country:  -> None
zero: albumstatus:  -> None
zero: media:  -> None
zero: albumdisambig:  -> None
zero: date:  -> None
zero: month: 0 -> None
zero: day: 0 -> None
zero: original_date:  -> None
zero: original_year: 0 -> None
zero: original_month: 0 -> None
zero: original_day: 0 -> None
zero: artist_credit:  -> None
zero: albumartist_credit:  -> None
zero: art:  -> None
zero: mb_trackid:  -> None
zero: mb_releasetrackid:  -> None
zero: mb_workid:  -> None
zero: mb_albumid:  -> None
zero: mb_artistid:  -> None
zero: mb_albumartistid:  -> None
zero: mb_releasegroupid:  -> None
zero: rg_track_gain: None -> None
zero: rg_album_gain: None -> None
zero: rg_track_peak: None -> None
zero: rg_album_peak: None -> None
zero: r128_track_gain: None -> None
zero: r128_album_gain: None -> None
zero: initial_key: None -> None
Sending event: write
zero: genres: None -> None
zero: genre: None -> None
zero: lyricist: None -> None
zero: composer_sort: None -> None
zero: arranger: None -> None
zero: grouping: None -> None
zero: comments: None -> None
zero: bpm: None -> None
zero: comp: None -> None
zero: artist_sort: None -> None
zero: albumartist_sort: None -> None
zero: asin: None -> None
zero: catalognum: None -> None
zero: barcode: None -> None
zero: disctitle: None -> None
zero: encoder: None -> None
zero: script: None -> None
zero: language: None -> None
zero: country: None -> None
zero: albumstatus: None -> None
zero: media: None -> None
zero: albumdisambig: None -> None
zero: date: None -> None
zero: month: None -> None
zero: day: None -> None
zero: original_date: None -> None
zero: original_year: None -> None
zero: original_month: None -> None
zero: original_day: None -> None
zero: artist_credit: None -> None
zero: albumartist_credit: None -> None
zero: art: None -> None
zero: mb_trackid: None -> None
zero: mb_releasetrackid: None -> None
zero: mb_workid: None -> None
zero: mb_albumid: None -> None
zero: mb_artistid: None -> None
zero: mb_albumartistid: None -> None
zero: mb_releasegroupid: None -> None
zero: rg_track_gain: None -> None
zero: rg_album_gain: None -> None
zero: rg_track_peak: None -> None
zero: rg_album_peak: None -> None
zero: r128_track_gain: None -> None
zero: r128_album_gain: None -> None
zero: initial_key: None -> None
Sending event: after_write
Sending event: database_change

I noticed this because my imported collection has all sorts of garbage tags that I didn't list on keep_fields:. I've tested importing an album many times with different zero settings each time, and in all cases there was a tool tag I created to see what would happen to it, and it's still there.

Maybe the way zero works doesn't really take care of these custom tags that many programs can create and instead goes through a list obvious things like genre or comment where it indeed works and removes them.

Setup

• OS: macOS High Sierra
• Python version: 2.7.16
• beets version: 1.4.9 and 1.5.0 work the same

My configuration (output of beet config) is:

directory: /Volumes/Expansion-1/Media/Audio/Sound/

plugins: convert discogs duplicates fetchart embedart lyrics missing zero

autotag: yes

art_filename: cover

clutter: .pdf .bmp .png .jpg .jpef .tif .gif .sfv .nfo .m3u .log .DS_Store Thumbs.DB Thumbs.db

original_date: yes

languages: en

va_name: Various

ignored_media: .vob .mp4 .mkv .avi .ts .pdf .cbz .cbr

import:
    copy: no
    write: yes
    move: yes

paths:
    default: $albumartist/$album%aunique{albumartist album, tracktotal albumtype year label,()}/$track. $title
    comp: Various/$album%aunique{albumartist album, tracktotal albumtype year label,()}/$track. $title
    albumartist:various artists: Various/$album%aunique{albumartist album, tracktotal albumtype year label,()}/$track. $title
replace:
    '[\\/]': _
    '^\.': _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    '\.$': _
    '\s+$': ''
    '^\s+': ''
    '^-': _
convert:
    tmpdir: /Volumes/Expansion-1/Media/Audio/Sound/
    auto: yes
    delete_originals: yes
    never_convert_lossy_files: yes
    format: mp3
    mp3:
      command: ffmpeg -i $source -q:a 0 -map_metadata 0 $dest
      extension: mp3
discogs:
    user_token: redacted
duplicates:
    album: yes
    move: /Users/redacted/Downloads/
fetchart:
    auto: yes
    sources: fanarttv itunes amazon coverart albumart google
    fanarttv_key: redacted
    google_API_key: redacted
embedart: 
    auto: yes
    maxwidth: 1000
lyrics:
    auto: no
    force: yes
    sources: genius musixmatch lyricwiki google
    google_API_key: redacted
    fallback: ''
    import.write: yes
replaygain:
    auto: no
    backend: command
chroma:
    auto: no
zero:
    auto: yes
    fields: ''
    keep_fields: 
        - albumtype
        - artist
        - title
        - album
        - year
        - composer
        - albumartist
        - track
        - tracktotal
        - disc
        - disctotal
        - label
        - lyrics
        - rating
        - acoustid_fingerprint
        - acoustid_id
        - first played
        - last played
        - play count
        - images
    update_database: true

I have also tried:

removing fields: '' using fields: [] using keep_fields: albumtype artist title album year composer albumartist track tracktotal disc disctotal label lyrics rating acoustid_fingerprint acoustid_id first_played last_played play_count images

It is also unclear to me how to list fields without an underscore such as first played (not first_played) with this plugin. Because the listing is one string, it would recognized as first and played separately. My solution is includes these - listing, which I must've took from somewhere, and I don't know if they're right in the first place.

[jackwilsdon]

The zero plugin will only remove fields which beets already knows about (via mediafile, see here for a list).

You could try using the scrub plugin alongside zero, which should remove all fields, after which beets will only put back the tags it understands.

To answer your question about fields with spaces - none of the fields supported by beets actually have spaces in them, so zero won't touch them.

There is also the beet fields command which lists all of the field names that beets supports.

Related discussion

[2V3EvG4LMJFdRe]

Thanks, I understand it better now. I think the documentation should mention that it only targets fields from a hard-coded list, this wasn't clear to me. scrub won't work for my case, because there are a handful of not-supported tags that I like to keep.

If possible, I'd like the status of this issue to change to feature request. I've changed the title to reflect this request.

[jackwilsdon]

Sounds reasonable enough! I'm not sure how user-friendly it would be to support listing unsupported fields in keep_fields, as their names can differ from format-to-format (unlike the fields we support, which we alias together when managing them), but I'm sure something can be worked out.

The documentation does mention that a list of supported fields can be retrieved with beet fields, although this could possibly be made clearer.

[sampsyo]

Improving the docs would be great! Listing unsupported tags to remove or preserve would be pretty tricky for the reasons @jackwilsdon mentioned above… it could possibly work for free-form tag formats like Vorbis Comments, but is not really feasible for ID3 or MPEG-4 metadata, so it would end up being a pretty format-specific thing (unlike the rest of the plugin). If we do explore it, maybe it would be better to do it in the scrub plugin instead of zero, which currently does not interact with low-level tag metadata at all.

[2V3EvG4LMJFdRe]

As an user, I really hope it doesn't end up being format specific, at least not if it excludes ID3 which is by far the most popular format still. Just for reference, foobar2000 and mp3tag can remove them from ID3.