jelu icon indicating copy to clipboard operation
jelu copied to clipboard

Feature request: How to add custom metadata provider?

Open dodog opened this issue 1 year ago • 14 comments

This book library looks very good. It would like to add my whole library, but I found out it that the default import search does not find the books I own (mostly Slovak and Czech). Would it be possible to add search for ISBN or title in Czech National library database?

https://aleph.nkp.cz/F/?func=file&file_name=find-b&local_base=NKC&CON_LNG=ENG

dodog avatar Dec 20 '24 16:12 dodog

Which default import did you use ? Have you activated inventaire in the settings ? If not maybe you could give it a try.

bayang avatar Dec 20 '24 18:12 bayang

First I just used the automatic default import. Then I created the application.yml and enabled google and inventaire plugins. But it could not find the randomly selected Slovak and Czech books. I tried to search directly in inventaire.io and no search results. English books are found instantly.

How can I add custom metadata provider?

dodog avatar Dec 20 '24 19:12 dodog

short answer is you can't.

Long answer is : the only way is to develop a new metadata provider for each source.

bayang avatar Jan 14 '25 08:01 bayang

Thank you for your answer. Is it difficult to develop a new metadata provider? Could you please direct me how to create a new provider?

dodog avatar Jan 14 '25 18:01 dodog

By the way have you tried to activate only the google api provider or to put it first in the priorities ? It is quite powerful as google has a massive database. I think it could work quite well.

Concerning the development it is not complicated if, and only if, you have an API to query. You can have a look at the google API provider, it was developed by a third party dev. First all providers have to implement a common interface : https://github.com/bayang/jelu/blob/main/src/main/kotlin/io/github/bayang/jelu/service/metadata/providers/IMetaDataProvider.kt

Which means your code receives an isbn, and/or a title and/or an authors list, it should query your target API and then return the most relevant result.

The google implementtion is here : https://github.com/bayang/jelu/blob/main/src/main/kotlin/io/github/bayang/jelu/service/metadata/providers/GoogleBooksIMetaDataProvider.kt

bayang avatar Jan 15 '25 08:01 bayang

I tried, but could not find anything - it is just loading and loading, but no result. Same for calibre. screenshot-2025-01-15_12-24-42 When I try inventaire.io, I get this screen, but when I click to "Import", nothing is filled out. screenshot-2025-01-15_12-24-15 Maybe some error messages would be useful like "book not found" or something else.

When I tried to search for this particular book in google books, it is found, so maybe there is something wrong with my setup.

This is my application.yml

dodog avatar Jan 15 '25 11:01 dodog

I just remembered the google API provider only fetches by isbn. With isbn your book is found but I also discovered a few bugs while testing. I'll try to add support for search by title or authors with the google plugin.

But as soon as I'll release a fix for the bug you'll be able to search again by isbn at least for now.

Your config is fine by the way. Providers are called by decreasing order of priority, ie provider with highest priority is called first.

bayang avatar Jan 15 '25 12:01 bayang

ok, but when I use ISBN only in Jelu via Google API, the book is not found. If I go manually to books.google.com and I enter the ISBN, the book is found. So, maybe there is some other issue.

dodog avatar Jan 15 '25 12:01 dodog

I just found a bug with some results in the google provider. The example book you pasted above triggers this bug. I'll release a fix and then you can try again.

bayang avatar Jan 15 '25 13:01 bayang

oh, ok. Thank you.

dodog avatar Jan 15 '25 13:01 dodog

The google provider should be fixed now. You can try again with isbn, title or author search. The results should be better suited to your researches.

bayang avatar Jan 17 '25 21:01 bayang

I tried the google provider with ISBN: 9788088912767, but it does not seems to work. In Jelu, the book is not found. In books.google.com using the same ISBN 9788088912767 - the book is found.

I checked the logs, but it shows some errors only for the previous search of book (ISBN 9788073910594), maybe it is not related to this jelu.log. Can you please have a look?

dodog avatar Jan 18 '25 13:01 dodog

The google books API does not return a book for this ISBN, you can check by yourself by calling it directly :

https://www.googleapis.com/books/v1/volumes?q=isbn%3A9788088912767

{
  "kind": "books#volumes",
  "totalItems": 0
}

I can't do much about that sorry. maybe the google books site has access to more data or queries the API in a different way.

bayang avatar Jan 19 '25 16:01 bayang

Thank you @bayang for looking into it. I checked through the API link you have provided - and it is very strange. I used two books with ISBN13. Here is what I have found:

Book One ISBN: 9788024653372 FOUND https://www.googleapis.com/books/v1/volumes?q=9788024653372 https://www.googleapis.com/books/v1/volumes?q=isbn9788024653372 https://www.googleapis.com/books/v1/volumes?q=isbn:%209788024653372 https://www.googleapis.com/books/v1/volumes?q=isbn%3A%209788024653372

NOT FOUND https://www.googleapis.com/books/v1/volumes?q=isbn:9788024653372 https://www.googleapis.com/books/v1/volumes?q=isbn%3A9788024653372

Book Two ISBN: 9780857202505 FOUND https://www.googleapis.com/books/v1/volumes?q=isbn:9780857202505 https://www.googleapis.com/books/v1/volumes?q=isbn%3A9780857202505

NOT FOUND https://www.googleapis.com/books/v1/volumes?q=9780857202505 https://www.googleapis.com/books/v1/volumes?q=isbn9780857202505 https://www.googleapis.com/books/v1/volumes?q=isbn%209780857202505 https://www.googleapis.com/books/v1/volumes?q=isbn%3A%209780857202505

Maybe it is some character encoding issue, but unfortunately they does not overlap. Some books are found with the space after colon, some without. I tried with different books.

dodog avatar Jan 19 '25 22:01 dodog