Tag generation when importing

[bland328]

I just discovered linkding, and think it may be what I've been hoping for, except for one issue.

Is there a way to get it to turn bookmark folder names into tags when importing?

That is, if I have a bookmark like this stored three folders deep...

Cars > Restoration > Paint > http://example.com/howtopaintcars

...I'd like to have it automatically tagged Cars, Restoration and Paint.

I have a couple thousand bookmarks to import, so bringing them in without tags is unworkable, as is manual tagging.

This approach wouldn't be perfect in all cases, but would retain much of the organization I've already established.

[sissbruecker]

@bland328 I understand the problem, but it's not something that I would want support in the application itself unless there is a large demand for it. Also while your proposed solution might work very well for you specifically I can image this leading to and unexpected tag explosion for others 😄.

For now the best suggestion I can give you is to pre-process your bookmark file using some 3rd party tool before importing it, or writing such a tool yourself. I know that there are some libraries and scripts out there that work with the Netscape HTML file format which might be repurposed to support your case. For example from a quick search, this PHP script seems to do something similar: https://github.com/kafene/netscape-bookmark-parser/blob/master/NetscapeBookmarkParser.php#L126

[sunrisepi]

Hi @sissbruecker , I just discovered LINKDING and like it a lot! Very fast, efficient, clean interface. However, I have the same challenge as @bland328... I imported over 1,900 links from an exported Firefox bookmark file, and now I just have a huge mass of links and no cohesive framework to search them by tags.

I had tried to use Shaarli, but it is not able to handle all links efficiently. It requires 15+ seconds to switch pages (my links cause about 96 pages of links, each about 20 links per page...). Shaarli handles the import like I would like, in that it converts Firefox folder names into tags. This allows me to easily search by tag groups in Shaarli. However, Shaarli is very slow!

I was able to import all the bookmarks into LINKDING (took a while, about 4 minutes for my Raspberry Pi 4B) and the switching between pages is fast. Search is also great! If LINKDING could support converting folders to tags during the import of my Firefox bookmark file, then I would have the solution I need. Alas, as it does not, I have four options: (1) Write my own importer/format the file myself, (2) Search and see if someone else already has a tool I can use, (3) Wait to see if you could support this feature, or (4) Find some other application.

I think this would be a great feature to add to LINKDING. You could just have a checkbox on the import feature of the UI, so people could choose to have folders converted to tags if they have a checkbox selected. And if not selected, then LINKDING just goes the import like it does today, ignoring the folder names. I think adding this feature would expand the appeal of LINKDING even further and encourage those with huge bookmarks libraries to give it a try. Anyway, just a suggestion.

I really like what I see with LINKDING. It is so close to having everything I need! Anyway, thanks very much for creating it. I look forward to future updates. :)

[sunrisepi]

OK, @bland328, I figured out how to get my exported bookmarks formatted with tags! Sharing this in case it helps someone else. @sissbruecker, what do you think about adding this to the documentation/README.md, so that others who have this use case would know about this great utility for formatting tags in bookmark files?

Use buku (``Bookmark manager like a text-based mini-web'') to format the bookmarks for import. Github: Buku. Here are the steps I followed:

• Install on Raspberry Pi via sudo apt-get install buku
• Should also work for other Uni*x OS's with package managers like apt, or using pip3... check the Github link above to learn more about Buku and how to install it on your specific system. Once installed, continue.
• Export bookmarks from Firefox.. calling the file bookmarksfromFF.html
• Edit bookmarks in text editor if you want to clean things up: Convert spaces to dash ("-") in folder names; also change folder names if you want better tags. Save changes. I converted spaces to dashes as in my testing, spaces in tags didn't work when searching in LINKDING.
• Next, we will use buku to edit the bookmarks file (automatically converts folder names to tags when imported into buku database for editing; use --tacit parameter to prevent buku from adding timestamp tag for current day). Overview of steps: First import into buku database (for conversion of folder names->tags), then export to html, then delete the bookmarks from buku database.
• Import bookmarks into buku: buku -i bookmarksfromFF.html --tacit [answer no to first question, "Append tags when bookmark exist? (y/n)"; answer yes to question, "Add parent folder names as tags? (y/n)"]
• Next, export bookmarks from buku: buku -e bookmarksdone.html
• Finally, delete the imported bookmarks from buku's database as we're done processing them: buku -d
• Feel free to review all the other features buku offers; it seems very feature-rich and awesome! I plan to explore it further later. For now, my immediate goal is to get my bookmarks into LINKDING with folders converted to tags.

Then, the bookmarks may be imported into LINKDING, as we've (a) converted spaces to dashes so all tags work (mentioned space issue in #86); (b) converted folder names to tags automatically using buku and exported from buku into an HTML file.

So, the first time I tried to import my massive list of bookmarks (~1900), LINKDING crashed. From docker logs linkding:

Mon Feb 22 23:43:10 2021 - *** HARAKIRI ON WORKER 1 (pid: 12, try: 1) ***
Mon Feb 22 23:43:10 2021 - HARAKIRI !!! worker 1 status !!!
Mon Feb 22 23:43:10 2021 - HARAKIRI [core 0] [IP ADDRESS OBFUSCATED] - POST /settings/import since 1614036889
Mon Feb 22 23:43:10 2021 - HARAKIRI !!! end of worker 1 status !!!
DAMN ! worker 1 (pid: 12) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 27)

It crashed even though I extended the timeout by including -e LD_REQUEST_TIMEOUT=500 on my docker run command. It crashed before the timeout elapsed. So, I stopped & deleted the container, deleted the data directory, and tried again. The second time the import finished, creating a sql lite DB 672K in size. The webpage timed out, but I kept watching the data directory and noticed the size of the sqlite file kept increasing, so I knew LINKDING was still processing the import. Once the file stopped growing, I checked the docker logs and confirmed it was done with the import. Then I went to the web UI and confirmed that all the links appeared to be present. All the tags I tested worked great too.

@sissbruecker, if I wish to edit the tags in this database would I be able to just load this DB file into sql lite and search/replace to update the tags? Wondering how to do this if I want to change the tag names later. Or, I guess I could update the tags in the HTML and re-import it again.

Thanks very much for LINKDING! I was able to find a workaround for my need (import Firefox bookmarks, converting folder names to tags) by using buku. LINKDING is fast, allowing me to search the database quickly (clicking on tags returns results in under 5 seconds, often 2 or 3 seconds), and I can change to any page in under 5 seconds nearly every time. I have an outstanding question on if spaces are allowed in tags (#86), but other than that it seems to work as expected. Thank you @sissbruecker! ;)

[sunrisepi]

Answering my own questions! :)

Yes, seems you can edit the database file using sqlitebrowser. And, I think one can't use spaces in tags, as spaces are the separator for multiple tags. I updated #86 with this as well.

I was able to edit the LINKDING database by doing the following (I did see a few tags I forgot to update; had to replace spaces with - characters, so e.g. instead of big island had to update to big-island.

• Download sqlite browser & install it
• Stop LINKDING docker container, make backup of database in case of issues
• In sqlitebrowser, run statement select * from bookmarks_tag to see which tags had spaces & needed to be fixed
• Then update bookmarks as needed. Found a reference.

One example:

UPDATE bookmarks_tag
SET name=REPLACE(name,'diskpart and efi','diskpart-and-efi');

Then write all changes to database, close database. After doing that update the permissions so they are correct for the database (on my system should be owned by www-data:www-data and permissions rw-r--r--).

Next started LINKDING docker container, verified the updated tags now work!! And also confirmed old tags work, and that I could add new links, update tags, etc. So looks like the change worked OK.

One other thing I noticed, is that on my mobile phone, the LINKDING site doesn't show the "Tags" section on the right side of the screen. I can't even scroll over to find it, and rotating to landscape mode also doesn't cause it to show up. Any idea how to get that to show up, @sissbruecker?

Everything else is looking good! :) Thanks.

[sissbruecker]

Thanks for providing workaround @sunrisepi .

I'll leave this open for now if someone wants to work on providing an option to convert folders to tags.

[ulixxe]

I add a link to a python script I did to convert folders into tags: https://github.com/ulixxe/Netscape-Bookmark-File-Parser You can modify folders2tags.py script to suit your needs.

[StrangePeanut]

@ulixxe script worked brilliantly, cheers!