openlibrary icon indicating copy to clipboard operation
openlibrary copied to clipboard

Trending API error

Open seabelis opened this issue 7 months ago • 4 comments

Problem

https://openlibrary.org/trending/daily.json throws the following error: A Problem Occurred We're sorry, a problem occurred while responding to your request:

    <class 'TypeError'>
    Object of type set is not JSON serializable

The reference code 2025-05-25/112433965649 has been created to track this error and we will investigate as we're able.

Reproducing the bug

  1. Go to ... https://openlibrary.org/trending/daily.json
  2. Do ...
  • Expected behavior: no error
  • Actual behavior: error message

Context

  • Browser (Chrome, Safari, Firefox, etc):
  • OS (Windows, Mac, etc):
  • Logged in (Y/N):
  • Environment (prod, dev, local): prod

Breakdown

Requirements Checklist

  • [ ]

Related files

Stakeholders


Instructions for Contributors

  • Please run these commands to ensure your repository is up to date before creating a new branch to work on this issue and each time after pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.

seabelis avatar May 25 '25 11:05 seabelis

@mekarpeles @seabelis I got curious and have been looking into this issue this morning. I have a few ideas as to what's going on. My current guess is that the "json.dumps(result)" in the trending_books_api class in api.py isn't able to return what it is meant to, and so no json with the information is being returned. Possibly an issue with the input parameters, but again, I'm still investigating.

If no one else is currently working on this, I can submit a pull request later today with an idea on how to fix this.

emggeorge avatar May 31 '25 16:05 emggeorge

This is still failing, could this be fixed please?

Ashkan-san avatar Jun 07 '25 15:06 Ashkan-san

Sentry reports: https://sentry.archive.org/organizations/ia-ux/issues/550406/?project=7&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D&referrer=issue-stream&stream_index=21

The https://openlibrary.org/trending/daily.json weekly.json and monthly.json all fail with the same error:

TypeError: Object of type set is not JSON serializable

on

openlibrary/plugins/openlibrary/api.py in GET at line 102
In App

        result = {
            'query': f"/trending/{period}",
            'works': [dict(work) for work in works],
            'days': days,
            'hours': i.hours,
        }
        return delegate.RawText(json.dumps(result), content_type="application/json")

With a stack trace of:

SINCE_DAYS = {'daily': 1, 'forever': None, 'monthly': 30, 'now': 0, 'weekly': 7, 'yearly': 365}
result = {'days': 1, 'hours': 0, 'query': '/trending/daily', 'works': [{...}, ...]

This is likely because either works contains a set field that is not serializable -- this seems to match this PR https://github.com/internetarchive/openlibrary/pull/10882/files by @emggeorge 🎉

mekarpeles avatar Jun 09 '25 00:06 mekarpeles

The fact that this went undetected probably indicates that test coverage needs to be improved in addition to fixing the bug.

tfmorris avatar Jun 09 '25 16:06 tfmorris

We may want to replace this API with the trending feed from solr rather than a db query powered by reading log counts. Also, tests probably useful

mekarpeles avatar Jul 14 '25 13:07 mekarpeles

Fixed by #10882

cdrini avatar Jul 14 '25 16:07 cdrini