transclusion-minor-mode

transclusion-minor-mode copied to clipboard

• transclusion minor-minor mode

work in progress in progress.

made to work with org-mode files

** what it's supposed to do

[[https://i.imgur.com/Zgsfx79.gif]]

• quickstart

after cloning this repo

#+BEGIN_SRC sh :eval never git submodule init # pulls ext--json-rpc-request git submodule update #+END_SRC

set up and run the [[https://github.com/whacked/xcl][transclusion server]] somewhere

start emacs and load =xcl-transclude.el=

open this file, =M-x xcl-transclude-mode=, cursor over a transclusion directive, and =C-c E=

(functions and key chords are subject to change)

• development setup

  assuming you have the [[https://nixos.org/nix/][nix package manager]], clone this repository and run =nix-shell=.

** emacs

from the =nix-shell=, running =emacs= should give you a development-ready emacs instance

** webserver

the transclusion engine is a nodejs webserver; see [[https://github.com/whacked/xcl][transclusion server]] for install + setup; launch it from another terminal

#+BEGIN_SRC sh :eval never node build/webserver.js #+END_SRC

• using org-mode MACRO syntax instead of INCLUDE

  with [[http://orgmode.org/manual/Macro-replacement.html][org macros]] this seems to be a better option.

  see https://github.com/fniessen/org-macros/blob/master/org-macros.setup

  To test this, run the org-mode export process (e.g. =C-c C-e= then =h H= or =t A=), then find the text from =LICENSE= exported as well.

  TODO: figure out how to make this org-compatible. Right now it is trivially easy to cause export-time conflicts due to =eval=

** org-mode export interop test

using org's native macro facility in normal org export

#+MACRO: transclude (eval (with-temp-buffer (insert-file-contents "$1" nil $2 $3) (buffer-string)))

partial transclusion

{{{transclude(LICENSE::60,500)}}}

the whole thing

{{{transclude(LICENSE)}}}

• transclusion addressing syntax

  primary consideration is edit-time user friendliness;

  for accuracy, [[https://github.com/whacked/SPRI/blob/experimental/lib/resolution/text.sibilant#make-anchor][fuzzy text anchors]] should be used

  non-resolution should lead to an empty resource, i.e. an empty string at render time

** point vs range resource resolution

There is 1 goal of the transclusion target notation: to establish a /range/. However, conventional address methods do not distinguish between /point/ and /range/ resources. For transclusion, we need to ensure that we operate, code-wise, in range space. Hence, when given a point resource, we must convert it into a range resource.

In addition to the straightforward range methods covered in sections below, some examples of org internal links:

• textual link ([[https://orgmode.org/manual/Internal-links.html][see documentation]]) like =[[My Target]]= can resolve, in this order, to:
  1. a text label =<<My Target>>= aka /dedicated target/. This defines a point resource.
  2. an /element/, =#+NAME: My Target=. This needs to be interpreted as the range encompassing that /element/.
  3. a headline, =* My Target=. This is the range encompassing the /section/.
• =CUSTOM_ID= a la =[[#my-custom-id]]=, resolves exactly to an entry with =CUSTOM_ID: my-custom-id=. This needs to be interpreted as the range encompassing the /section/ bearing that id.

the textual link method logic may involve back-tracking, so it may be sensible to simply disallow providing a single textual link for transclusion.

** non-semantic, non-structural ranges

*** full file

{{{transclude(file:tiny.org)}}}

*** lines

{{{transclude(file:tiny.org::5)}}}

**** line range

{{{transclude(file:tiny.org::2-3)}}}

**** from start

{{{transclude(file:tiny.org::-2)}}}

**** until end

{{{transclude(file:tiny.org::7-)}}}

*** character index range

**** character range

 {{{transclude(file:tiny.org::0,100)}}}

**** from start

 {{{transclude(file:tiny.org::,100)}}}

**** until end

 {{{transclude(file:tiny.org::50,)}}}

*** percent range

convert to line first

{{{transclude(file:moozorgjs.org::1%-3%)}}}

** structural ranges

*** contextual target

**** native org links

 {{{transclude(file:tiny.org::*decoy 1)}}}

**** exact match with range

 {{{transclude(file:tiny.org::in 2101...for great justice)}}}

*** exact match with range

these should all resolve to
 
[[file:tiny.org::*huh][huh]]

*** glob file name

{{{transclude(file:t*ny.org::*huh)}}}

**** fuzzy find file by content

{{{transclude(grep:gg+you::*huh)}}}

{{{transclude(grep:gg you::*huh)}}}

{{{transclude(grep:gg%20you::*huh)}}}

** TODO reconcile org internal link syntax with structural ranges

• constriction syntax / semantic range

  a better term than /constriction/ / /constrictor/ probably exists

  the =$constrictor= should fallback to resolution logic of the colon-based content address when no constriction method is found

** content based range constriction

address like:

• file:$RESOLVER#$constrictor
• https://$RESOLVER#constrictor
• =http://example.com/#dummy target#ignore again#This domain...=

where $constrictor ...

** anchor based constriction

not yet a user friendly way to create anchors in edit phase

** examples using moozorgjs.org

*** nearest token set (fallback)

$RESOLVER#handling todo

resolve to [[file:moozorgjs.org::*Handling%20TODO][Handling TODO]]

via first + best sequential match

*** token set range (fallback)

$RESOLVER#in 2101...for great justice

- [X] js poc
- [ ] emacs poc

*** =line=: nearest statement / line

possibly just =L=

=$RESOLVER#line/support+scheduled=

resolve to the line,

"Support attributes like ~SCHEDULED:~."

[[file:moozorgjs.org::*Implement%20todo%20attributes][Implement todo attributes]]

- [X] js poc
- [ ] emacs poc

*** =para=: nearest paragraph

possibly just =P=

=$RESOLVER#para/org.js parser converter=

intro paragraph

[[file:moozorgjs.org][intro paragraph]]

- [X] js poc
- [ ] emacs poc

*** =section= nearest hierarchical section

need org context

=$RESOLVER#section/web browser=

Web Browser

[[file:moozorgjs.org::*Web%20Browser][Web Browser]]

*** ??? nearest conceptual unit

*hardest* -- need broader semantic analysis

=$RESOLVER#unit/require org=

require("org")

the src block with "require" within

[[file:moozorgjs.org::*Node.js][Node.js]]
 

• post-transclusion processor syntax

  this only makes sense in read-only transclusion

  the pipe is an obvious choice of "filter" designation, but since it conflicts with org-mode table syntax, the post-processor parsing should also recognize unicode =BROKEN BAR= =¦= as an alternative

  • file:$RESOLVER#$constrictor|post-processor
  • file:$RESOLVER#$constrictor¦post-processor

** collapse newlines

this currently makes sense for something transcluded into an org doc where only the text is wanted, but the transclusion target's (host document) location has significant whitespace, such as in an org table. Linebreaks in the source document cause the table to break.

** strip heading

when the transclusion target is defined by an org headline section, one may want to keep the text only. The address may look like

=file:somefile.org::*infamous muse of famous mouse¦no-headline=

• testing decoy headers

** real thing

*** decoy 1

text in the real

** fake thing

*** decoy 1

text in the fake

** what does org do?

(org picks the first occurrence)

[[*decoy 1][decoy 1 (real)]]

[[*decoy 1][decoy 1 (fake)]]

• testing org-mode compatibility

  • [[file:transcluding-org-elements.org?]] # opens dired
  • [[file:transcluding-org-elements.org?s=1]] # opens dired

• transclusion macro syntax interaction in emacs

  this relies on running the webserver from [[file:xcl/src/xcl/node_webserver.cljs]]

  see [[file:xcl/README.org]] for details

** playground

{{{transclude(LICENSE::2,10)}}}

{{{transclude(file:transcluding-org-elements.org)}}}

{{{transclude(file:./transcluding-org-elements.org::2-10)}}}

{{{transclude(file:./transcluding-org-elements.org::20)}}}

{{{transclude(xcl:./public/tracemonkey.pdf?p=3&s=Monkey observes that...so TraceMonkey attempts)}}}

{{{transclude(calibre:quick start?s=The real advantage...on your computer.)}}}

{{{transclude(zotero:just-in-time?s=Monkey observes...TraceMonkey attempts)}}}

{{{transclude(xcl:./public/alice.epub?p=2&s=Would you tell me, please...walk long enough)}}}

{{{transclude(zotero:faulty reward functions?s=We assumed...measure the performance)}}}

• also see

** using org dynamic blocks

http://stackoverflow.com/questions/15328515/iso-transclusion-in-emacs-org-mode

http://stackoverflow.com/a/15352203

#+BEGIN_SRC emacs-lisp :results none (defun org-dblock-write:transclusion (params) (progn (with-temp-buffer (insert-file-contents (plist-get params :filename)) (let ((range-start (or (plist-get params :min) (line-number-at-pos (point-min)))) (range-end (or (plist-get params :max) (line-number-at-pos (point-max))))) (copy-region-as-kill (line-beginning-position range-start) (line-end-position range-end)))) (yank))) #+END_SRC

Then to include a line range from a given file, you can create a dynamic block like so:

#+BEGIN: transclusion :filename "transclude.el" :min 2 :max 4 #+END:

=C-c C-x C-u= (=org-dblock-update=) will on the =BEGIN/END= block updates that block; =org-update-all-dblocks= does all in the buffer.

This is an unsatisfactory solution because it is actually just unidirectional /quasi-dynamic inclusion/; /quasi/, because the update step has no knowledge of the target /until/ the update is invoked. It is possible to add action hooks to run updates automatically, but that is no different from export-time inclusion, which by nature assumes a unidirectional /flow/ of information. Org already has a number of ways to achieve this, such as a =sh= src block that runs =cat=.

Transclusion is different in practice: it is the /actual content, appearing/ at the location of transclusion.

** John Kitchin's version (requires org-mode 9)

see [[file:transcluding-org-elements.org]]