NIP-23: long-form content

[fiatjaf]

This one is much larger than I thought it would be. I lost my patience in the middle of it so the quality might have degraded.

This PR also modifies NIP-33 and NIP-19 to introduce a new canonical way of referring to parameterized replaceable events.

[cmdruid]

Looks great. Is this the place to discuss both nips, or are there dedicated discussions for each NIP?

[fiatjaf]

I've created this nref thing on NIP-19, but I dislike the name and I don't know if it is a good idea. Someone please fix that.

Also I am not sure of how one of these articles can reference another using the nref stuff. That part is broken in the NIP because I got confused and distracted. Must be fixed.

[cmdruid]

These articles are meant to be editable, so they should make use of the replace-ability feature of NIP-33 and include a "d" tag with an identifier for the article.

Should this be optional? Assuming relays honor the replaceable part, I don't see a simple way to track an edit history, which is a nice feature to have for a blog, especially with news articles.

Edit: I guess you can control this by the kind while keeping the 'd' tag for reference, which let's the author control how the document is tracked / stored on the relay.

This NIP defines only title as the metadata.

Have you thought about using the "subject" field instead since it already exists in the spec?

I think the idea of standardizing a markdown-aware note for long-form content is fantastic, and supporting YAML front-matter is even more fantastic.

One obvious addition for meta-data I can see is to have a payment address for the article. This could default to an lud16 if one exists for the author in their profile, but having custom payment links per article could help with tracking engagement.

[cmdruid]

Also I am not sure of how one of these articles can reference another using the nref stuff. That part is broken in the NIP because I got confused and distracted. Must be fixed.

The NIP seems to make sense to me. Each nref is parsed out of the content field, and replaced with an tag reference plus a positional reference in the document.

I suppose you can't use 'e' since the current document can change, and 'd' is used for pinning events to a static identifier. It seems like there needs to be a tag similar to 'e', but for referencing events identified by a 'd' tag correct?

[eskema]

I still need some time to process this NIP, my brain doesn't get it yet... i find all this nref things a bit confusing and anti pattern for nostr. I think the document should be able to referenced by note1d and it's hex protocol id, and we should be able to see the edits, so a list of versions.. so I think we need at least two files for this to work. having only one creates a lot of undesirable trade-offs that ends up beeing worse. (or I simply need to process this NIP better..) anyway.. here's what I'm thinking:

n1 = kind 11 n2 = kind 30011 with [d, n1.id], we can use the contents here for the event metadata, title, description, whatever and every time we want to edit the text, what happens is we create a new copy of the n1 and add this new id to the tags array of n2.

when sharing the event, you share the original id of n1, the client when it fetches it, sees it's a kind 11 so it searches for it's corresponding 30011 file. there we can grab the last [e] tag as the last version. if the array is empty then we have the only and last version (i.e. no edits). this lets us see what was actually written before edits (so we have accountability) instead of "permanently" deleting it. Also, we can keep updating the "metadata" of the kind 30011 content without altering the event itself if having the id, i think we can also search directly for n2 and then get latest from that (instead of fetching n1 first)

P.S. Also, each kind 11 edits would reference the original id so we can fetch n2 from any edit

[wires]

Very nice, this complements my KaTeX proposal nicely in that I've been thinking about this in a long form "plain text" format, ie. markdown or this NIP.

I wrote some comments (in my own markdown editor :-P), maybe these are helpful.

[Section Format] ... Markdown syntax

Which specific version of Markdown? There are various flavors.

I suggest to learn from past experience and decide "once and for all" on a good Markdown-variant. To decide which one is subtle, I could write to write down an argumentation and suggestion... (based on my experiences from working heavily with md, in my daily job I we build a markdown based literate programming language and editor).

... YAML front-matter

We used YAML initially but its complex and has unwanted conveniences such as "yes" === true. Would suggest TOML for the "event format" instead and if clients insist on having YAML as input language they could convert TOML but present YAML to the user.

Would also propose to extract the TOML frontmatter into a separate property. .frontmatter or .metadata for instance.

This avoids having an extra parsing step to seperate these two languages (YAML/MD or TOML/MD) and in turn avoids complex parsers and highlighters that must support "multi modal" editing, because now you can use a separate editor for each string.

Again if a client insists in having frontmatter in the same editor, they could concatenate both into one string and split again when building the event.

[Section References] ... (for example [click here][0]

Why not use existing markdown / NIP-08 behaviour for this, as in [click here](#[0]). This would require no special support for detecting when the user is entering a markdown link in the client, I can just use existing autocomplete for n* refs/mentions and replace with #[..]. As opposed to having a special case for article links.

Or is the goal to have article hyperlinking explicitly distinguishable from regular markdown hyperlinks that happen to refer to nrefs?

In that case I would suggest considering support for deeper linking, such as

• referencing sub headings (#### = eid§1.1.1.3),
• or even specific formulas (see katex proposal#216, section [Alternative for KaTeX support (discuss?)] aludes at this, using a NIP-08 style referencing, but would work equally well with other proposal, simply enumerating each $..$ expression)
• or images/figures,
• and maybe even (character) ranges. (think: medium highlights, random quotes)

Last section, example "title":

shouldn't this be "title" and "content" ?

[wires]

As for the front matter, I would suggesting putting "special" / "semantically meaningful" fields under a uncommon subheading with a name such as "$$meta" or NOSTR.

In TOML:

This way the user is free to pick their own use for this front matter and this is very powerful (your markdown editor is now a flexible database, each document being a row).

Going further there are really two use cases for this "front matter" business:

• Allow the user to enter stuff like title, layout, enable extension, ISBN, etc.. ; the semantically meaningful.
• Allow the user to attach their own data to their notes (notion, air table style)

We never got around to migrating away from multi modal editing, but we were planning to use something like this:

type Metadata = {
  title: string,
  theme: string,
  pubData: string,
  /* URLs, DOI, Arxiv, ... */
}
type LongForm = {
   userdata: any,
   metadata: Metadata,
   markdown: string
}

This has the advantage of simplifying the parsing, typing, etc. and being clearing about what is each type of data for. The "front matter" business is after all just a convenient way of entering that data and I rather have the client do the parsing that this being essential part of the protocol in order to just show the title, date.

[wires]

Additional comment on the front-matter semantically meaning/not distinction and separating these out into their own properties.

Imagine tagging a document, this can be done in YAML:

tags:
- flep
- koek

or TOML

tags = ["flep","koek"]

or even

[tags]
flep=123
lang="nl"

(and assuming these now mean something to nostr clients that understand tags)

however, this isn't the nicest UX and so you end up wanting to build a specialized tagging element with autocomplete, but now you need to synchronized the front matter text with the data coming from that UX element state. This kinda sucks because in order to preserve white-space while en/decoding to/from text you need to use a special parser and these are rare.

So having a "complicated" language like "YAMLFrontMatterMarkdown" is just not as nice and much more complex than just having an object sub-property that you can pass to an UI element.

[wires]

I've created this nref thing on NIP-19, but I dislike the name and I don't know if it is a good idea. Someone please fix that.

Another long comment on this, sorry for the spam, hope this is useful somehow. I tried to implement persistent/strongly typed referencing/querying of arbitrary (sub) content on our (nostr/matrix like) system and I came to the following insights.

Initially linking is easy because there are only 1 or two types of events in the system. But since we allow extensions it quickly becomes opaque to 'contents' of unknown kinds of events. In our case we deal with different nested/complex structures but we can provide a 'content tree structure' for most of them and map nodes in that tree to character ranges or other "lenses" into their "source of truth data" structure.

This way the client can still browse references and links even if it doesn't understand the content. I will illustrate below at npath section what I mean.

Because I suspect something similar is going on here:

nref itself is Not A Bad Idea, altho a bit "specific", as in

• they "only" are coordinates to query for Parameterized Replaceable Events of any kind
• (but not generic "(deeper) references"),

so maybe its better to pick a more specific/technical name such as

• npre
• nprec (PRE coodinate), or
• npreq (PRE query)

?

On NIP-19 for note it doesn't state clearly if this requires the event to be of kind:1, I assume not? Initially I was thinking note is for kind:1 and nevent is the "any kind" event coordinate and in that line of thinking:

The PRE coordinate is (pubkey, d-value) and the relays are "just" hints.

Maybe instead of nref/nprec, its nice to have nlong or nlog instead, which would refer only to long/blog type notes (and maybe their content/tags). So more specific to long form content, not more general like npath, more like what I thought note meant. It would have the same spec as nref, with d-value+pubkey+ relay hints, but with added requirement of kind:30023 events only?

However, I can imagine more and more needs for specialized references arising.

So Instead maybe an npath approach would be interesting, borrowing ideas from XPath/CSS references?

These paths are 'drill down expression' into content on the nostr network and the different dimensions available at each 'drill down step' depend on the event kind at the current "location". So you start at some coordinate (nevent, nprofile,...) and then like a CSS selector step to a next coordinate. Examples of stepping dimensions could be

• following tag-indices #[0],
• diving into metadata properties .name or other event properties,
• filtering by typestamp :npub > notes > 2022
• filtering by tag (value), etc.

For instance for long form we require the client implementing the npath nip to parse the markdown and produce a table of contents. This would be similar to outputting a parser AST and with character ranges on each tree node. This is similar to how errors work in an IDE, you click on the error/reference and you find the exact lines/char-range etc that it refers to.

To keep this simple the 'table of contents' should be a tree of (nested/non-overlapping/valid..?) character ranges into the .content string, and an optional name (for each branch). Then a npath reference could be a "note coordinate" (nprec/note/nlong) followed by a list of numbers, indicating each branch of the content tree. That would be a kind:30023 table of contents dimension. But a similar construction could be used for chapter annotations in lectures or indices into a mixtape, source maps, stuff like that (altho this may changes the type of the character range label on the tree nodes).

If you think of the "space" of Nostr state, the discrete "points" in this space are the event id's. Then there is a partitioning on (pubkey, kind) and with NIP-33 sub partition d-values is added. Referencing only a d-value would should not be defined as a valid coordinate (as you could have d-value collisions from different pubkeys/kind) and d-values should always be seen as a sub partitioning of pubkey/kind space. So (pubkey,kind,d-value) is a much better coordinate.

It would require some puzzling but if we define the right dimensions this is would be both simple, powerful and generic way to reference any (custom) (sub) content on the nostr network.

Also I am not sure of how one of these articles can reference another using the nref stuff. That part is broken in the NIP because I got confused and distracted. Must be fixed.

Would it not suffice to replace in the .content references to npre/note/nlong/npath by #[..], and place a tag ["r", :refType, :hex] where :refType is nref,note, ... and :hex the decoded bech32 content?

[wires]

NB: I explained this npath not to say this NIP should include that, rather that it makes sense to think seriously about linking/referencing and take inspiration from semantic web (failures) in a different NIP maybe and for now focus on something long form specialized and maybe using the fact that markdown has ToC structure.

[fiatjaf]

This GitHub thing is terrible for discussing. I think Nostr is mature enough for discussions to be done inside nested threads there.

[dskvr]

This GitHub thing is terrible for discussing. I think Nostr is mature enough for discussions to be done inside nested threads there.

Is there a canonical event ID for this discussion?

[wires]

Is there a canonical event ID for this discussion?

I scratched an itch and made nips.be/pr/220 to redirect to PRs (see nips.be for redirect URLs).

Which client do you recommend for such threaded discussion?

Right now NIPs in various states are spread across PRs, issues and the main branch.

Would be nice if there one canonical place where NIPs are kept and a CONTRIBUTING.md file that outlines the NIPs lifecycle. Maybe a table (sequenceNr, eventId/nref, title, short slug, title) (on github/nostr). I could parse that and generate redirects /:slug and /:nr. Right now nips.be redirects to github, but I wouldn't mind serving a static page that tries to pull the longform NIP from some relays and renders it. Could even try to do this with a CDN edge function to have SSR and good indexing by search engines. I can then add nostr:nref deep link so you can use your preferred client to comment/edit. Or link to various suggested clients.

[fiatjaf]

Let's try to have this discussion over at note1t7ekflxtvg7hsfh7f9zm6e6eykv3e0qyw690spw4rfzm0dw5e29qw54p32.

[fiatjaf]

Which client do you recommend for such threaded discussion?

I recommend https://github.com/mikedilger/gossip

[fiatjaf]

Answered @eskema and @wires (partially) at note1t7ekflxtvg7hsfh7f9zm6e6eykv3e0qyw690spw4rfzm0dw5e29qw54p32.

[fiatjaf]

A prototype implementation of a writer client here by @talvasconcelos: https://write.nostr.com/

[mplorentz]

Ok so I made a bunch of comments on the source before reading the comments about moving conversation to Nostr 😳 will post future comments there. (although the Nostr clients I use today feel like a bad place to have long important discussion, since not all people necessarily see all the same comments. Maybe we need a NIPS-discussion relay 😛)

[fiatjaf]

A prototype implementation of a reader client here by @talvasconcelos: https://read.nostr.com