zk

zk copied to clipboard

#+title: zk.el - A dead-simple, feature-rich Zettelkasten implementation for Emacs #+author: Grant Rosson #+language: en

[[https://melpa.org/#/zk][file:https://melpa.org/packages/zk-badge.svg]]

• [[#introduction][Introduction]]
• [[#setup][Setup]]
• [[#features][Features]]
  • [[#follow-links][Follow Links]]
    • [[#using-org-mode-and-zk-links][Using Org-Mode and zk-links]]
    • [[using-org-links-with-zk-org-link.el][Using Org-links with zk-org-link.el]]
    • [[#link-hint.el][Support for link-hint.el]]
  • [[#list-backlinks][List Backlinks]]
  • [[#smart-new-note-creation][Smart New-Note Creation]]
  • [[#insert-links][Insert Links]]
  • [[#search][Search]]
    • [[#alternative-search-functions-using-consult-grep][Alternative Search Functions, using Consult-Grep]]
  • [[#embark-integration][Embark Integration]]
  • [[#list-current-notes][List Current Notes]]
• [[#zk-index-and-zk-desktop][ZK-Index and ZK-Desktop]]
  • [[#setup-1][Setup]]
  • [[zk-index][ZK-Index]]
  • [[zk-desktop][ZK-Desktop]]
  • [[#embark-integration-for-zk-index-and-zk-desktop][Embark integration for ZK-Index and ZK-Desktop]]
• [[#comparable-zettelkasten-like-implementations][Comparable Zettelkasten(-like) Implementations]]

• Introduction

This set of functions aims to implement many (but not all) of the features of the package [[https://github.com/EFLS/zetteldeft/][Zetteldeft]], while circumventing and eliminating any dependency on [[https://github.com/jrblevin/deft][Deft]], or any other external packages for that matter. It does not use any backend cache or database, preferring instead to query a directory of plaintext notes directly, treating and utilizing that directory as a sufficient database unto itself.

To that end, these functions rely, at the lowest level, on simple calls to =grep=, which returns lists of files, links, and tags to the Emacs completion function =completing-read=, from which files can be opened and links and tags can be inserted into an open buffer.

Out of the box, links are clickable buttons made with the built-in =button.el=. This means that links will work the same way in (almost) any major mode: fundamental-mode, text-mode, outline-mode, markdown-mode, etc. The key exception is Org-Mode, where a minor change is necessary to enable clickable zk-links. (See below for details.)

The structural simplicity of this set of functions is---one hopes, at least---in line with the structural simplicity of the so-called "Zettelkasten method," of which much can be read in many places, including at https://www.zettelkasten.de. Ultimately, this package aims to be a lean, understandable, and eminently forkable Zettelkasten implementation for Emacs. Fork away, and make it your own.

Video demonstrations:

• [[https://www.youtube.com/watch?v=BixlUK4QTNk][zk - Setup and Main Features]]
• [[https://www.youtube.com/watch?v=oEgdJlojlU8][zk - Getting Started - Hydra and Inbox Note]]
• [[https://www.youtube.com/watch?v=7qNT87dphiA][zk-index and zk-desktop]]
• [[https://www.youtube.com/watch?v=O6iSV4pQQ5g][zk-luhmann]]

** File Structure

Notes are all kept in a single directory, set to the variable =zk-directory=, with no subdirectories.

Each note is a separate file, named as follows: a unique ID number followed by the title of the note followed by the file extension (set in the variable =zk-extension=), e.g. "202012091130 On the origin of species.txt".

** IDs and Links

The primary connector between notes is the simple link, which takes the form of an ID number enclosed in double-brackets, eg, =[[202012091130]]=. A note's ID number, by default, is a twelve-digit string corresponding to the date and time the note was originally created. For example, a note created on December 9th, 2020 at 11:30 will have the ID "202012091130". Linking to such a note involves nothing more than placing the string =[[202012091130]]= into another note in the directory.

A key consequence of this ID and file-naming scheme is that a note's title can change without any existing links to the note being broken, wherever they might be in the directory.

** Completion

For the best experience completing filenames and links, it is highly recommended to use the =orderless= completion style, from [[https://github.com/oantolin/orderless][the package of the same name]]. Another high recommendation is [[https://github.com/minad/vertico][Vertico]], for completion in the minibuffer.

• Setup

The easiest way to install is from [[https://melpa.org/#/zk][MELPA]].

Or, manually add =zk.el= to your loadpath and include =(require 'zk)= in your init.el file.

At a minimum, you must set the variables =zk-directory= and =zk-file-extension=:

#+begin_src emacs-lisp (setq zk-directory "~/path/to/your/zk-directory") (setq zk-file-extension "md") ;; any plaintext file extension, eg, "org" or "txt" #+end_src

Once =zk= is loaded, call =M-x zk-new-note= to create a note or =M-x zk-find-file= to open an existing note.

*** Additional Setup:

• To enable automatic link-creation when opening a zk-file, include the function =(zk-setup-auto-link-buttons)= in your init config. This ensures that =zk-enable-link-buttons= is set to =t= and adds =zk-make-link-buttons= to Emacs's =find-file-hook=.

• To enable Embark integration, include the function =(zk-setup-embark)= in your init config.

*** Sample setup with =use-package=

#+begin_src emacs-lisp (use-package zk :custom (zk-directory "~/path/to/zk-directory") (zk-file-extension "md") :config (zk-setup-auto-link-buttons) (zk-setup-embark)) #+end_src

*** Sample setup with =straight.el= to include optional =zk-consult.el= functions

See [[#alternative-search-functions-using-consult-grep][Alternative Search Functions, using Consult-Grep]]

#+begin_src emacs-lisp (use-package zk :straight (zk :files (:defaults "zk-consult.el")) :custom (zk-directory "~/path/to/zk-directory") (zk-file-extension "md") :config (require 'zk-consult) (zk-setup-auto-link-buttons) (zk-setup-embark) (setq zk-tag-grep-function #'zk-consult-grep-tag-search zk-grep-function #'zk-consult-grep)) #+end_src

• Features

** Follow Links

Links are buttons made with the built-in package =button.el=: they are clickable text that work the same way in any major mode. Whether in fundamental-mode, text-mode, outline-mode, or markdown-mode, etc., clicking or pressing =RET= on a zk-link opens the corresponding note. The only exception is Org-Mode. (See below.) configuring clickable links in Org-Mode, see below.)

It is also possible to call the command =zk-follow-link-at-point= when a link is at point, or call the command =zk-links-in-note= to be presented with a =completing-read= list of all links in the current note.

*** Using Org-Mode and zk-links

In Org-Mode, links in the default format =zk-link-format= (an ID in double-brackets) will be treated as internal links. This means that when they are clicked, Org will, by default, look for an in-buffer heading or target that is named, or contains, the given ID. To make Org treat zk-links /as/ zk-links and open the corresponding note, it is only necessary to advise the function =org-open-at-point= as follows:

#+begin_src emacs-lisp (defun zk-org-try-to-follow-link (fn &optional arg) "When 'org-open-at-point' FN fails, try 'zk-follow-link-at-point'. Optional ARG." (let ((org-link-search-must-match-exact-headline t)) (condition-case nil (apply fn arg) (error (zk-follow-link-at-point)))))

(advice-add 'org-open-at-point :around #'zk-org-try-to-follow-link) #+end_src

Briefly, this function instructs =org-open-at-point= to try calling =zk-follow-link-at-point= when a link is not an internal link.

An alternative solution for using Org-Mode would be to change =zk-link-format= to use, for example, single brackets instead of double brackets. With this change, the default link buttons will work as expected.

Note that using Org links makes the creation of link buttons, via =zk-make-link-buttons=, redundant. This link button aspects of the package can be disabled by setting =zk-enable-link-buttons= to nil.

*** Using Org-links with zk-org-link.el

The companion package =zk-org-link.el= provides a custom Org-link type called =zk=, such that links will be styled =[[zk:201812101245]]= instead of =[[201812101245]]=. Using Org-links allows notes to be followed as expected, as well as exported to various formats via =org-export=, stored via =org-store-link=, and completed via =org-insert-link=.

The link styles cannot be combined --- they are not mutually compatible. Use one style or the other. That is, either use =zk-org-link.el= or don't. (I do not, but here it is anyway.)

To use org-links, include the following in your init.el:

#+begin_src emacs-lisp (with-eval-after-load 'org (with-eval-after-load 'zk (require 'zk-org-link))) #+end_src

This will set create the =zk= Org-link type and set necessary values for several variables. Be sure to load =zk-org-link.el= /after/ zk, as the above code snippet does.

NOTE: =zk-completion-at-point= functionality is not available when using =zk-org-link.el=.

*** link-hint.el

To allow link-hint.el to find zk-links, it is necessary to add a new link type, as follows:

#+begin_src emacs-lisp (defun zk-link-hint--zk-link-at-point-p () "Return the id of the zk-link at point or nil." (thing-at-point-looking-at (zk-link-regexp)))

(defun zk-link-hint--next-zk-link (&optional bound) "Find the next zk-link. Only search the range between just after the point and BOUND." (link-hint--next-regexp zk-id-regexp bound))

(eval-when-compile (link-hint-define-type 'zk-link :next #'zk-link-hint--next-zk-link :at-point-p #'zk-link-hint--zk-link-at-point-p :open #'zk-follow-link-at-point :copy #'kill-new))

(push 'link-hint-zk-link link-hint-types) #+end_src

** List Backlinks

Calling =zk-backlinks= in any note presents a list, with completion, of all notes that contain at least one link to the current note.

** Smart New-Note Creation

The function =zk-new-note= prompts for a title and generates a unique ID number for the new note based on the current date and time. A new file with that ID and title will be created in the =zk-directory=.

*** New-Note Header and Backlink

The header of the new note is inserted by means of a function, the name of which must be set to the variable =zk-new-note-header-function=.

The default header function, =zk-new-note-header=, behaves differently depending on the context in which =zk-new-note= is initiated. If =zk-new-note= is called within an existing note, from within the =zk-directory=, the new note's header will contain a backlink to that note. If =zk-new-note= is called from outside of the =zk-directory=, there are two possible behaviors, depending on the setting of the variable =zk-default-backlink=. If this variable is set to nil, the header of the new note will contain no backlink. If this variable is set to an ID (as a string), the header will contain a link and title corresponding with that ID. This can be useful if the directory contains a something like a "home" note or an "inbox" note.

*** Insert New-Note Link at Point of Creation

By default, a link to the new note, along with the new note's title, will be placed at point wherever =zk-new-note= was called. This behavior can be configured with the variable =zk-new-note-link-insert=: when set to =t=, a link is always inserted; when set to =zk=, a link is inserted only when =zk-new-note= is initiated inside an existing note in =zk-directory=; when set to =ask=, the user is asked whether or not a link should be inserted; when set to =nil=, a link is not inserted. Calling =zk-new-note= with a prefix-argument will insert a link regardless of setting of =zk-new-note-link-insert=.

*** ID Format

By default, the date/time of a generated ID only goes to the minute, though this can be configured with the variable =zk-id-time-string-format=. In the default case, however, if more than one note is created in the same minute, the ID will be incremented by 1 until it is unique, allowing for rapid note creation.

*** New-Note from Region

Finally, a new note can be created from a selected region of text. The convention for this feature is that the first line of the region will be used as the new note's title, while the subsequent lines will be used as the body, with the exception of a single separator line between title and body. To clarify, consider the following as the region selected swhen =zk-new-note= is called:

#+begin_src emacs-lisp On the origin of species

It is not knowledge we lack. What is missing is the courage to understand what we know and to draw conclusions. #+end_src

The title of the new note in this case will be "On the origin of species." The body will be the two sentences that follow it. The empty line separating title from body is necessary and should not be excluded.

Note: This behavior is derived from the behavior of an earlier, long-used Zettelkasten implementation and it persists here by custom only. It would be trivial to alter this function to behave perhaps more sensibly, for example by using the selected region in its entirety as the body and prompting for a title. For now, though, custom prevails.

** Insert Links

*** Insert Links via Function

Calling =zk-insert-link= presents a list, with completion, of all notes in the =zk-directory=. By default this function inserts only the link itself, like so: =[[202012091130]]=.

To insert both a link and title, either use a prefix-argument before calling =zk-insert-link= or set the variable =zk-link-insert-title= to =t=, to always insert link and title together. Note that when =zk-link-insert-title= is set to =t=, calling =zk-insert-link= with a prefix-argument temporarily restores the default behavior and inserts the link without a title.

To be prompted with a yes-or-no query, asking whether to insert a title with the link or insert only a link by itself, set =zk-link-insert-title= to =ask=. With this setting, a prefix-argument also restores the default behavior of inserting only a link.

The format in which link and title are inserted can be configured with the variable =zk-link-and-title-format=.

*** Completion-at-Point

This package includes a completion-at-point-function, =zk-completion-at-point=, for inserting links. Completion candidates are formatted as links followed by a title, i.e., =[[202012091130]] On the origin of species=, such that typing =[[= will initiate completion. To enable this functionality, add =zk-completion-at-point= function to =completion-at-point-functions=, by evaluating the following:

=(add-hook 'completion-at-point-functions #'zk-completion-at-point 'append)=

Consider using [[https://github.com/minad/corfu][Corfu]] or [[https://github.com/company-mode/company-mode][Company]] as a convenient interface for such completions.

** Search

*** Note Search

The default search behavior of =zk-search= calls the built-in function =lgrep= to search for a regexp in all files in =zk-directory=. Results are presented in a =grep= buffer.

The function =zk-find-file-by-full-text-search= presents, via =completing-read=, a list of all files containing at least a single instance of a give search string somewhere in the body of the note. Compare this to =zk-file-file= which returns matches only from the filename.

*** Tag Search (and Insert)

There are two functions that query all notes in the =zk-directory= for tags in following form: =#tag=. One of the functions, =zk-tag-search=, opens a grep buffer listing all notes that contain the selected tag. The other function, =zk-tag-insert=, inserts the selected tag into the current buffer.

*** Alternative Search Functions, using Consult-Grep

The file =zk-consult.el= includes two alternative functions, for use with the [[https://github.com/minad/consult][Consult]] package, that display the results using =completing-read=.

To use, make sure =Consult= is loaded, then load =zk-consult.el=, and set the following variables accordingly:

#+begin_src emacs-lisp (setq zk-grep-function 'zk-consult-grep) (setq zk-tag-grep-function 'zk-consult-grep-tag-search) #+end_src

** Embark Integration

This package includes support for [[https://github.com/oantolin/embark][Embark]], both on links-at-point and in the minibuffer.

To enable Embark integration, evaluate the function =zk-setup-embark=. Include this function in your config file to setup Embark integration on startup.

When Embark is loaded, calling =embark-act= on a zk-id at point makes available the functions in the keymap =zk-id-map=. This is a convenient way to follow links or to search for instances of the ID in all notes using =zk-search=.

Calling =embark-act= in the minibuffer makes available the functions in =zk-file-map=. This is a convenient way to open notes or insert links.

Additionally, note that because the function =zk-current-notes= uses =read-buffer= by default, all Embark buffer actions are automatically available through =embark-act=. This makes killing open notes a snap!

Last note: adding =zk-search= to other Embark keymaps is a convenient way to search all notes for a given Embark target. Consider adding it to the =embark-region-map=, for example, with a memorable keybinding --- like "z"!

** List Current Notes

The function =zk-current-notes= presents a list of all currently open notes. Selecting a note opens it in the current frame.

The command can be set to use custom function, however, by setting the variable =zk-current-note-function= to the name of a function.

One such function is available in =zk-consult.el=: =zk-consult-current-notes= presents the list of current notes as a narrowed =consult-buffer-source=. Note that this source can also be included in the primary =consult-buffer= interface by adding =zk-consult-source= to list =consult-buffer-sources=. (This is not done by default.)

• ZK-Index and ZK-Desktop

The package =zk-index.el= is a companion to =zk= that offers two buffer-based interfaces for working with notes in your zk-directory.

For a video demonstration, see: https://youtu.be/7qNT87dphiA

** Setup

This package is available on [[https://melpa.org/#/zk-index][MELPA]].

Sample setup with =use-package=:

#+begin_src emacs-lisp (use-package zk-index :after zk :config (zk-index-setup-embark) :custom (zk-index-desktop-directory zk-directory)) #+end_src

** ZK-Index

The function =zk-index= pops up a buffer listing of all note titles, each of which is a clickable button. Clicking a title will pop the note into the above window.

The ZK-Index buffer is in a major mode with a dedicated keymap:

#+begin_src emacs-lisp (defvar zk-index-mode-map (let ((map (make-sparse-keymap))) (define-key map (kbd "n") #'zk-index-next-line) (define-key map (kbd "p") #'zk-index-previous-line) (define-key map (kbd "v") #'zk-index-view-note) (define-key map (kbd "o") #'other-window) (define-key map (kbd "f") #'zk-index-focus) (define-key map (kbd "s") #'zk-index-search) (define-key map (kbd "d") #'zk-index-send-to-desktop) (define-key map (kbd "D") #'zk-index-switch-to-desktop) (define-key map (kbd "c") #'zk-index-current-notes) (define-key map (kbd "i") #'zk-index-refresh) (define-key map (kbd "S") #'zk-index-sort-size) (define-key map (kbd "M") #'zk-index-sort-modified) (define-key map (kbd "C") #'zk-index-sort-created) (define-key map (kbd "RET") #'zk-index-open-note) (define-key map (kbd "q") #'delete-window) (make-composed-keymap map tabulated-list-mode-map)) "Keymap for ZK-Index buffer.") #+end_src

*** Navigation

The keys =n= and =p= move the point to the next/previous index item, previewing the note at point in the above window. (This previewing behavior can be disabled by setting =zk-index-auto-scroll= to nil.) In contrast, using =C-n= and =C-p= will move the point up and down the list without previewing notes.

Pressing =v= (short for for 'view') on an index item will open the corresponding note in =read-only-mode=, such that pressing =q= will quit the buffer and return the point to the index. Pressing =RET= on an index item will open the corresponding note the expected major mode.

*** Narrowing and Filtering

The key =f= (for 'focus') filters notes by matching a string in the note's TITLE. For example, pressing =f= and entering the string "nature" will produce an index of all notes with the word "nature" in their titles.

The focus feature is cumulative, so pressing =f= again and entering another string, say, "climate," will narrow down the index down further, to notes with the words "nature" and "climate" in the title.

The key 's' (for 'search') for filters notes by matching a string in their full text. So, pressing =s= and entering the string "nature" will produce an index of all notes that contain the word "nature" anywhere in the note itself.

The search feature is also cumulative.

Moreover, focus and search can be combined: you can focus by title and then search by content, or the other way around.

The key 'i' refreshes the index, canceling any filtering/narrowing, returning all notes to the list.

*** Sorting

By default the index is sorted by time of last modification, with most recently modified notes being sorted to the top of the index. The key =M= (for 'modified') enacts this sorting method.

The key =C= (for 'created') sorts the index by time of creation, with the most recently created notes sorted to the top.

The key =S= (for 'size') sorts the index by size of note, with largest notes sorted to the top.

** ZK-Desktop

The feature =zk-desktop= allows users to select and organize groups of notes relevant to specific projects. The only necessary setup is setting a directory for saved desktops. A convenient and unobtrusive option is to simply use the =zk-directory= itself:

#+begin_src emacs-lisp (setq zk-index-desktop-directory zk-directory) #+end_src

Think of =zk-desktop= as allowing you to achieve something like pulling project-specific note cards from a physical file cabinet and laying them out on a desktop in front of you, to be grouped and rearranged any way you like. In this case, however, the "desktop" is a simple plaintext file saved in the =zk-directory= and the "note cards" are just note titles, each a clickable button, just like in =zk-index=.

In contrast to =zk-index=, all notes on a given desktop are selected and placed there individually by the user, note-by-note, rather than en masse and programmatically. Additionally, the notes placed on the desktop can be rearranged, grouped, and commented on in-line.

It is possible to have several desktops at once, each an individual file, and each corresponding to a different project. Use the function =zk-index-desktop-select= to switch from working with one desktop to working with another.

*** Working with notes on a desktop

The notes listed on in the zk-desktop buffer can be rearranged, a single note can appear more than once, and the user can type on the desktop just like in a normal buffer --- for example, to create headings or simply to type notes.

A zk-desktop buffers open in =fundamental-mode= by default, but this can be changed by setting the variable =zk-index-desktop-major-mode= to the symbol for a major mode. Consider setting this to =text-mode=, =outline-mode=, or =org-mode=.

#+begin_src emacs-lisp (setq zk-index-desktop-major-mode 'outline-mode) #+end_src

*** Adding notes to a desktop

Each method of adding notes to the currently active desktop is accomplished via the same function: =zk-index-send-to-desktop=.

When this function is called in the =zk-index= buffer itself, the note at point is sent to the desktop. If several notes are selected in the index, all notes in the active region are sent to the current deskop. This selection feature is usefully combined with the focus/search feature of =zk-index=, to allow for sending a lot of relevant notes to a desktop at once.

** Embark Integration for ZK-Index and ZK-Desktop

To enable integration with Embark, include =(zk-index-setup-embark)= in your init config.

This setup allows all index and desktop items to be recognized as zk-id Embark targets, making available all Embark actions in the =zk-id-map=.

This also adds =zk-index-send-to-deskop= to =zk-id-map= and =zk-file-map=, to facilitate sending files to desktop from the minibuffer or via =embark-act= in the zk-index buffer.

Finally, calling =embark-export= on zk-files in the minibuffer will open a new ZK-Index buffer listing those files.

• Comparable Zettelkasten(-like) Implementations

• Emacs-based

  • [[https://github.com/EFLS/zetteldeft][Zetteldeft]]
  • [[https://github.com/org-roam/org-roam][Org-Roam]]
  • [[https://git.sr.ht/~protesilaos/denote][Denote]]

• Non-Emacs

  • [[https://zettelkasten.de/the-archive/][The Archive]]
  • [[https://zettlr.com][Zettlr]]
  • [[https://roamresearch.com][Roam]]
  • [[https://obsidian.md][Obsidian]]

** Why not use one of these?

/You should/! They are great. I used each one of them for a least some time, some for longer than others. At a certain point with each, however, I found that I couldn't make them do exactly what I wanted. My sense, eventually, was that the best implementation of a Zettelkasten is the one in which a user has as much control as possible over its structure, over its behavior, and, frankly, over its future viability. At first, this primarily meant using only plaintext files --- no proprietary formats, no opaque databases. Eventually, however, it also meant seeking out malleability and extensibility in the means of dealing with those plaintext files, ie, in the software.

My best experiences in this regard were with "The Archive" and, after I discovered Emacs, with "Zetteldeft." The former is highly extensible, largely by virtue (at least at this point) of the macro editor "KeyboardMaestro," through which one can do nearly anything with a directory of text files, in terms of editing, querying, inserting tags and links, etc. If I hadn't fallen into Emacs, I would definitely still be using "The Archive" in combination with "KeyboardMaestro." Little about my note-taking practices and preferences has changed since I used "The Archive." As for "Zetteldeft," the notable differences between it and the present package are only to be found under-the-hood, so to speak. The only reason I'm not still using it is that, over time, it became this.