Add db support for library

[mkb79]

Working on it on this branch

[mkb79]

@devvythelopper @rmcrackan and all other GitHub users

The code in the feature/db-ibrary branch is most ready for now. Before I refactor and format it, I need your help. Can you test the code and report back when running the following commands:

1. audible db library sync --init How many items do you have and how long does it take to sync your library.

2. audible db library remove Does it remove the file?

3. audible db library restore --fresh

4. Adding some free books to your library and remove some else and run audible db library sync This should run fast also if you have thousands of items. The client requests a delta update only for the changed items.

If you are ready you can remove the database by running audible db library remove.

If the work is done, I will rework the download command to use the database instead of fetching the whole library every time on execution. Maybe I will add another table to track downloaded items instead of searching in the download directory, if the files already exist.

[devvythelopper]

Cool feature. I'll see when I'll find the time though, the coming week is a bit busy, hope you don't have any time constraints.

You'd need commands to check for non-existing files then too.

[mkb79]

@devvythelopper

You'd need commands to check for non-existing files then too.

Do you mean a command like check-integrity which can verify that a db file exist and the schema is fine?

[rmcrackan]

@mkb79 I just got back in town and unfortunately I'm leaving again soon. I cloned the branch but I'm not familiar with running python. For windows, how can I help?

[mkb79]

@rmcrackan

As long as you have at least Python 3.10 and uv installed on your machine, you have two options:

---

Option A – with clone

1. Clone the correct branch:

   git clone --branch feature/db-library --single-branch https://github.com/mkb79/audible-cli.git
   

2. Change into the cloned directory.
3. Prefix all commands with uv run (for example:

   uv run audible db library sync --init
   

   ).

---

Option B – without clone

1. Prefix all commands with:

   uvx --fresh --from git+https://github.com/mkb79/audible-cli@feature/db-library
   

   Example:

   uvx --fresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync --init
   

   The --fresh flag ensures that the repository is cloned from GitHub every time before executing the command.
   If you are confident that there haven’t been any changes, you can omit --fresh and uvx will use the cached version instead.

---

💡 Almost forgot: if you use uv and uvx, you don’t need to have a Python interpreter installed.
uv will automatically handle that for you on the first run.

---

Manual setup with virtual environment

If you prefer to set up the project manually using a virtual environment instead of uv, follow these steps:

1. Navigate into the cloned directory.
2. Create a virtual environment:

   python3 -m venv venv
   

3. Activate the virtual environment:
   • On Windows with CMD:

     .\venv\Scripts\activate.bat
     

   • On Windows with PowerShell:

     .\venv\Scripts\activate.ps1
     

4. Run the initialization command:

   audible db library --init
   

[devvythelopper]

@devvythelopper

You'd need commands to check for non-existing files then too.

Do you mean a command like check-integrity which can verify that a db file exist and the schema is fine?

No, I mean that there needs to be a way to check the actual .aax, -annotations.json, etc. files for whether they exist instead of just checking whether the .aax file (etc) has been marked as downloaded in the database. And that I like the current default which is namely that file-existance is being checked for.

[devvythelopper]

@mkb79 I just got back in town and unfortunately I'm leaving again soon. I cloned the branch but I'm not familiar with running python. For windows, how can I help?

@rmcrackan I usually use git worktree to check out different branches. That way you have only one git. Then I cd and use python3 -m venv .venv and then from fish shell source ./.venv/bin/activate.fish or source ./.venv/bin/activate from bash. I frankly don't know how to activate the venv on windows. If you make changes to the code, running can be done via PYTHONPATH=src python3 pyi_entrypoint.py db library --init. Well, in a unix shell; if you use windows, you'd have to set the PYTHONPATH environment variable in some windows specific way.

[mkb79]

@mkb79 I just got back in town and unfortunately I'm leaving again soon. I cloned the branch but I'm not familiar with running python. For windows, how can I help?

@rmcrackan I usually use git worktree to check out different branches. That way you have only one git. Then I cd and use python3 -m venv .venv and then from fish shell source ./.venv/bin/activate.fish or source ./.venv/bin/activate from bash. I frankly don't know how to activate the venv on windows. If you make changes to the code, running can be done via PYTHONPATH=src python3 pyi_entrypoint.py db library --init. Well, in a unix shell; if you use windows, you'd have to set the PYTHONPATH environment variable in some windows specific way.

@rmcrackan

Alternatively, if you don’t plan to modify the code and therefore don’t need to clone the repository, you can simply use the uvx command. This will automatically provide the uv tool in an isolated environment, without requiring a system-wide Python installation or a local checkout of the repository.

Edit: One important note I forgot to mention: if you haven’t used audible-cli before, you’ll need to run audible quickstart once to complete the initial setup.

[rmcrackan]

I don't understand python's habits. From each of these folders:

• C:\Dev\audible-cli
• C:\Dev\audible-cli\src
• C:\Dev\audible-cli\src\audible_cli

I have tried each of these commands:

> audible quickstart
'audible' is not recognized as an internal or external command,
operable program or batch file.

> py audible quickstart
C:\Users\rmcrackan\AppData\Local\Programs\Python\Python313\python.exe:
can't open file 'C:\\Dev\\AUDIBLE_CLI\\audible-cli\\src\\audible_cli\\audible':
[Errno 2] No such file or directory

> py -m audible quickstart
C:\Users\rmcrackan\AppData\Local\Programs\Python\Python313\python.exe: No module named audible

Just to confirm, from those same folders, this works:

> py -V
Python 3.13.5

Can you use uv to run the package. It's very easy with uv to work with Python commands.

[rmcrackan]

Can you use uv to run the package. It's very easy with uv to work with Python commands.

Trying that in each of these directories:

C:\Dev\audible-cli>uv audible quickstart
error: unrecognized subcommand 'audible'

  tip: a similar subcommand exists: 'build'

Usage: uv [OPTIONS] <COMMAND>

For more information, try '--help'.

C:\Dev\audible-cli\src>uv audible quickstart
error: unrecognized subcommand 'audible'

  tip: a similar subcommand exists: 'build'

Usage: uv [OPTIONS] <COMMAND>

For more information, try '--help'.

C:\Dev\audible-cli\src\audible_cli>uv audible quickstart
error: unrecognized subcommand 'audible'

  tip: a similar subcommand exists: 'build'

Usage: uv [OPTIONS] <COMMAND>

For more information, try '--help'.

[mkb79]

Take a look at Option B here.

Simple run uvx --fresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible -h

[rmcrackan]

[mkb79]

@rmcrackan Maybe my mistake. Try --refresh instead of --fresh! Thank you!

[rmcrackan]

Progress:

[mkb79]

@rmcrackan You have to provide uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library before the audible command. Or you can install audible with uv tool install.

[rmcrackan]

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible quickstart

success

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync --init [sync] mode=full Upserted=1582, Soft-deleted=0, new state_token=1757132213333

<30 seconds

audible db library remove Does it remove the file?

Where is the file? I'd like to locate it before this test.

[rmcrackan]

Where is the file? I'd like to locate it before this test.

Ignore. I found it.

2:

audible db library remove Does it remove the file?

Yes. db file deleted. toml and json are unchanged

3:

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library restore --fresh
Installed 29 packages in 416ms
Usage: audible db library restore [OPTIONS]
Try 'audible db library restore -h' for help.

Error: Missing option '--payload'.

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library restore --refresh
Installed 29 packages in 413ms
Usage: audible db library restore [OPTIONS]
Try 'audible db library restore -h' for help.

Error: No such option: --refresh (Possible options: --fresh, --merge)

[mkb79]

Do you tried sync --init? How long does it take to load your lib?

[rmcrackan]

sync init is above -- https://github.com/mkb79/audible-cli/issues/240#issuecomment-3266396492

it took just under 30 seconds

[mkb79]

Ah okay. Thanks. This is much for ~1.500 titles.

Do you have exported your db before you have used the restore command?

[rmcrackan]

I just now deleted my files so I can start fresh. Each of the following is a separate command which seems to do some kind of environment initialization each time. That is: I seem to be creating a new session and incurring that overhead each time. I'm not familiar with uvx so I don't know if it's possible to do all of these in the same session instead. These are the only commands I have run:

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible -h

result: prints the help "usage" options and commands

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible quickstart

result: 2 files created:
C:\Users\rmcrackan\AppData\Local\Audible\config.toml
C:\Users\rmcrackan\AppData\Local\Audible\cupidneedsglasses_gmail.com.json

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync --init

result:
[sync] mode=full Upserted=1582, Soft-deleted=0, new state_token=1757343637893
Ran for about 29 seconds
created: C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite

uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library remove

result:
deleted: C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite
unchanged: C:\Users\rmcrackan\AppData\Local\Audible\config.toml
unchanged: C:\Users\rmcrackan\AppData\Local\Audible\cupidneedsglasses_gmail.com.json

[rmcrackan]

After the above, I was not able to run: audible db library restore --fresh

Error: Missing option '--payload'

[mkb79]

@rmcrackan

Sorry my mistake. You have to run audible db library export to get your library.json. Then you can restore the db with this file.

[rmcrackan]

I think this is a complete workflow of what you wanted tested:

1. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible -h result: prints the help "usage" options and commands

2. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible quickstart result: 2 files created:
   C:\Users\rmcrackan\AppData\Local\Audible\config.toml
   C:\Users\rmcrackan\AppData\Local\Audible\cupidneedsglasses_gmail.com.json

3. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync --init result:
   [sync] mode=full Upserted=1582, Soft-deleted=0, new state_token=1757343637893
   Ran for about 29 seconds
   created: C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite

4. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library export created: \library.json

5. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library remove result:
   deleted: C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite
   unchanged: C:\Users\rmcrackan\AppData\Local\Audible\config.toml
   unchanged: C:\Users\rmcrackan\AppData\Local\Audible\cupidneedsglasses_gmail.com.json

6. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library restore --fresh --payload library.json [restore] Upserted 1581 ? C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite (merge mode)
   created: C:\Users\rmcrackan\AppData\Local\Audible\library_5f1f4017cbf7b590.sqlite

7. Add to library: https://www.audible.com/pd/Headcase-Audiobook/B0F668ZJ4J

8. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync [sync] mode=delta Upserted=1, Soft-deleted=0, new state_token=1757431347606
   3-4 sec. Reminder: each step is running in its own session with init overhead

9. Remove from library: https://www.audible.com/pd/Headcase-Audiobook/B0F668ZJ4J

10. > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library sync [sync] mode=delta Upserted=0, Soft-deleted=0, new state_token=1757431347606
    NOTE: 'Headcase' not removed
    > uvx --refresh --from git+https://github.com/mkb79/audible-cli@feature/db-library audible db library export [export] Wrote 1582 items to library.json
    audible db library export: 'Headcase' is still present in export

[mkb79]

@rmcrackan Thank you for testing. Step 10 is strange. I've tested it myself with 5 - 6 different asins. All without any sync issues when adding or removing.

[damajor]

I tested the following library commands:

• sync --init
• export
• remove
• restore --fresh --payload
• sync (with new book, and also after removing one book)
• count
• search-fts
• search
• list-deleted
• inspect (with various parameters)

All good for me.

I wonder what can be the use of full as it looks like restore ?

[damajor]

@mkb79 Do you have basic usage of audible db assets ?

[mkb79]

I tested the following library commands:

• sync --init
• export
• remove
• restore --fresh --payload
• sync (with new book, and also after removing one book)
• count
• search-fts
• search
• list-deleted
• inspect (with various parameters)

All good for me.

I wonder what can be the use of full as it looks like restore ?

Hi. How long did the initial synchronization take? And how many titles do you have in your library? I want to know how performant the process is.

Do you have basic usage of audible db assets ?

The db assets command should manage the assets of the library. So storage location, asset type, etc. The db should later make it possible to quickly check which assets are still missing and downloadable. However, the status is not yet far advanced. Or was your question meant differently?

[damajor]

Hi. How long did the initial synchronization take? And how many titles do you have in your library? I want to know how performant the process is.

time audible db library sync --init
[sync] mode=full Upserted=590, Soft-deleted=0, new state_token=***

real    0m10.846s
user    0m0.629s
sys     0m0.330s

The db assets command should manage the assets of the library. So storage location, asset type, etc. The db should later make it possible to quickly check which assets are still missing and downloadable. However, the status is not yet far advanced. Or was your question meant differently?

I wonder the use of this command to leverage the maximum of audible-cli on my personal audible workflow (https://github.com/damajor/BALD). Library db was one thing I wanted to have for some time.

[mkb79]

@damajor The only catch with my current implementation is that the initial loading of the library runs synchronously over a continuation-token. Therefore, especially with large libraries it can take longer than before. However, the syncs run very fast, because only the diff is loaded.

But when everything is ready, you can of course use the database for your project. There are a lot of interesting things in it. You can take a look at the file that is created by db export command. Maybe you can use this to tag your converted files!