beets icon indicating copy to clipboard operation
beets copied to clipboard

fix: enable tracebacks for "user"/custom sqlite functions

Open ThinkChaos opened this issue 1 year ago • 2 comments

Description

A bit niche but I tried setting my bareasc prefix to an empty string, and was getting an obtuse error. This should help make clearer what is happening when queries fail.

The exception is not properly raised up the stack in the first place because it happens across 2 FFI boundaries: the DB query (Python -> SQLite), and the custom DB function (SQLite -> Python). Thus Python cannot forwarded it back to itself through SQLite, and it's treated as an "unraisable" exception.

We could override sys.unraisablehook to not print anything for the original exception, and store it in a global for the outer Python interpreter to fetch and raise properly, but that's pretty hacky, limited to a single DB instance and query at once, and risks swallowing other "unraisable" exceptions. Instead we just tell the user to look above for what Python prints.

Sample output:

Exception ignored in: <function unidecode_expect_ascii at 0x7f7fa20bb060>
Traceback (most recent call last):
  File "site-packages/unidecode/__init__.py", line 60, in unidecode_expect_ascii
    bytestring = string.encode('ASCII')
                 ^^^^^^^^^^^^^
AttributeError: 'bytes' object has no attribute 'encode'
Traceback (most recent call last):
  File "site-packages/beets/dbcore/db.py", line 988, in query
    cursor = self.db._connection().execute(statement, subvals)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: user-defined function raised exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "site-packages/beets/__main__.py", line 9, in <module>
    sys.exit(main())
             ^^^^^^
  File "site-packages/beets/ui/__init__.py", line 1865, in main
    _raw_main(args)
  File "site-packages/beets/ui/__init__.py", line 1852, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "site-packages/beets/ui/commands.py", line 1599, in list_func
    list_items(lib, decargs(args), opts.album)
  File "site-packages/beets/ui/commands.py", line 1594, in list_items
    for item in lib.items(query):
                ^^^^^^^^^^^^^^^^
  File "site-packages/beets/library.py", line 1695, in items
    return self._fetch(Item, query, sort or self.get_default_item_sort())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/beets/library.py", line 1673, in _fetch
    return super()._fetch(model_cls, query, sort)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/beets/dbcore/db.py", line 1301, in _fetch
    rows = tx.query(sql, subvals)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "site-packages/beets/dbcore/db.py", line 991, in query
    raise DBCustomFunctionError()
beets.dbcore.db.DBCustomFunctionError: beets defined custom SQLite function failed; see the other errors above for details

To Do

  • [x] ~Documentation. (If you've added a new command-line flag, for example, find the appropriate page under docs/ to describe it.)~
  • [x] ~Changelog. (Add an entry to docs/changelog.rst to the bottom of one of the lists near the top of the document.)~
  • [ ] Tests. (Very much encouraged but not strictly required.)

ThinkChaos avatar Aug 10 '24 00:08 ThinkChaos

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

github-actions[bot] avatar Aug 10 '24 00:08 github-actions[bot]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Apr 26 '25 01:04 stale[bot]

Codecov Report

:x: Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 65.84%. Comparing base (e3574a7) to head (0be4e3e). :warning: Report is 20 commits behind head on master.

Files with missing lines Patch % Lines
beets/dbcore/db.py 87.50% 0 Missing and 2 partials :warning:
:rocket: New features to boost your workflow:
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Aug 09 '25 00:08 codecov[bot]

Sorry it took me a while to get back after the review. Forgot about it and just got to cleaning up enough of my notifications to see it again.

I retested it to make sure it still works, and found the motivation to add an automated test.

ThinkChaos avatar Aug 09 '25 00:08 ThinkChaos

Nice, thanky you! What would you think about fixing the original issue as well?

snejus avatar Aug 09 '25 08:08 snejus

What would you think about fixing the original issue as well?

I can't reproduce the error anymore with empty bareasc prefix... :sweat_smile:

ThinkChaos avatar Aug 09 '25 23:08 ThinkChaos

Just rebase from master and we should be good to go!

snejus avatar Aug 10 '25 09:08 snejus

I rebased it.

In case you haven't seen it before, a Musicbrainz test failed on first try due to network flakiness: https://github.com/beetbox/beets/actions/runs/16862383319/job/47764495282?pr=5383 Retrying worked.

ThinkChaos avatar Aug 10 '25 15:08 ThinkChaos

Thank you!

ThinkChaos avatar Sep 21 '25 16:09 ThinkChaos