calibre-web icon indicating copy to clipboard operation
calibre-web copied to clipboard

[Feature Request] Support for split Database/Library locations

Open jfeeser opened this issue 5 years ago • 19 comments

Hi there, As my "application server" uses SSDs and has limited space, and my file server has much more space, i am using the "CALIBRE_OVERRIDE_DATABASE_PATH" option.

My database resides local to the application server and my books are stored on the file server. Unfortunately, since calibre-web appears to look for the books as subfolders of where the database is located, it can pull the book list from calibre's databse, but then can't find the books.

Is there any way to introduce an option to specify "database location" as well as "books location" if it's not the default?

jfeeser avatar Jun 02 '20 17:06 jfeeser

I wasn't aware of a setting like this. Yes, there is a way of implementing it (later)

OzzieIsaacs avatar Jun 02 '20 18:06 OzzieIsaacs

I too would love this feature!

zerodayz1 avatar Jun 03 '20 19:06 zerodayz1

+1 to this! I'm in exactly the same situation

dansnipe avatar Jun 15 '20 08:06 dansnipe

+1 to this! I need this very much.

torus13 avatar Jun 18 '20 12:06 torus13

+1 :) I move metadata.db in tmpfs and make symlink to library folder. It's make my library faster.

AleksdemSA avatar Jun 23 '20 06:06 AleksdemSA

I got this working by just overlaying the metadata.db file in my database location in docker and it worked fine.

artbird309 avatar Nov 11 '20 02:11 artbird309

@zanhecht

OzzieIsaacs avatar Dec 21 '20 18:12 OzzieIsaacs

+1 for this feature.

alex-phillips avatar Aug 22 '21 21:08 alex-phillips

@jfeeser I feel your pain.

A workaround could be using TrueNas Scale and container. You could keep your books (with metadata) on ZFS on HDD and config folder on ZFS on SSD.

@AleksdemSA / @artbird309 I also tried to mount books as NFS share and then put links to metadata.db stored elsewhere, though not working. Would you mind sharing how you have made it to work?

konradbjk avatar Nov 15 '21 10:11 konradbjk

@konradbjk I am not using a link I am mounting that file directly in docker, below is my volume section of my compose:

volumes:
      - $DOCKERDIR/calibre-web:/config
      - $MEDIA/Books:/media/Media/Books #whole Books directory from NAS
      - $DOCKERDIR/calibre/metadata.db:/media/Media/Books/Calibre/metadata.db #overlay the metadata.db on the library location on the nas but loaded from local file that lets it write it in the calibre container storage

artbird309 avatar Nov 26 '21 19:11 artbird309

Interesting @artbird309 the only difference I see that you have Calibre subfolder inside your books folder.

Do all your books are inside Calibre subfolder? Could you please share your folder structure?

konradbjk avatar Nov 27 '21 08:11 konradbjk

My folder structure: Physical: NAS SMB mount\Books\calibre #libray location where calibre books are located LazyLibrarian #lazyLibrarian folder where audiobooks are stored and files are downloaded to Local docker store\calibre/metadata.db #DB file to make it writable

Logical how the container sees the files: NAS SMB mount\Books\calibre #libray location where calibre books are located NAS SMB mount\Books\calibre\metadata.db #DB looks like its inside the folder but stored locally

artbird309 avatar Nov 27 '21 10:11 artbird309

Thanks @artbird309 I tried to avoid this approach (can't work on windows by the way) but it seems to be the only way because SQLlite locking is not possible on NFS

here my compose file if it can helps:

version: "3"
services:
  calibre:
    image: ghcr.io/linuxserver/calibre-web
    container_name: calibre-web
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
      - DOCKER_MODS=linuxserver/calibre-web:calibre #optional
    volumes:
      - ./config:/config
      - nfs-ebook:/books
      - ./config/metadata.db:/books/metadata.db
    ports:
      - 8083:8083
    restart: unless-stopped

delcroip avatar Dec 25 '21 15:12 delcroip

Is there any update regarding this issue?

chnoeli avatar Aug 12 '23 20:08 chnoeli

A work around that appears to be working - nfs_or_smb_mount:/config # yes /config not /books - ./config/metadata.db:/config/metadata.db - ./config/app.db:/config/app.db - ./config/gdrive.db:/config/gdrive.db - ./config/server-users.sqlite:/config/server-users.sqlite

gc4g40u6 avatar Dec 28 '23 12:12 gc4g40u6

I‘ve implemented the feature in the development branch, but it needs testing before I can merge it

OzzieIsaacs avatar Dec 28 '23 12:12 OzzieIsaacs

@OzzieIsaacs Is that available with any existing docker tags? Happy to test, I'm already on nightly.

undaunt avatar Jan 13 '24 01:01 undaunt

Highly Experimental:

This appeared 3 days ago on Calibre-Web's master branch!

  • "Implement split library and books" https://github.com/iiab/calibre-web/commit/d68e57c4fc5525b32c2312a251434368d9211251
  • "Mark separate library as highly experimental" https://github.com/iiab/calibre-web/commit/d8f5c175187c12d53bf059693c0dc61f4a6070a4

Any testing & configuring experiences so far?

holta avatar Feb 16 '24 01:02 holta

I've been tearing my hear out (see here https://github.com/janeczku/calibre-web/issues/3040)

But have now just pulled the master branch, and made use of the https://github.com/janeczku/calibre-web/issues/3040 and the first sync (only two books in my libary at the minute) worked a breeze.

Outsmart2383 avatar May 01 '24 20:05 Outsmart2383