nips icon indicating copy to clipboard operation
nips copied to clipboard

Relational Databases

Open vitorpamplona opened this issue 1 year ago • 16 comments
trafficstars

This NIP offers a plug-and-play method to export/sync information from existing systems using relational databases into Nostr. It describes a generic ROW event type to be used by all tables and all database types.

Read here

vitorpamplona avatar Apr 09 '24 20:04 vitorpamplona

Interesting. How about handling datatypes, sizes, null values, etc.

Is this just intended as an export for import? If so, why JSON instead of either a) a binary blob, or b) existing tools for data export as text

vicariousdrama avatar Apr 09 '24 20:04 vicariousdrama

How about handling datatypes, sizes, null values, etc.

Everything has to become a string :(

Is this just intended as an export for import?

Yes, I have a need to expose information from a relational system into the Nostr ecosystem. So, it would just export into these rows as of now.

vitorpamplona avatar Apr 09 '24 21:04 vitorpamplona

Right now, I am just doing a simple DB Trigger to push changes from 5 tables into Nostr. Then using a tool I developed in nostr to generate a report based on that data. It's obviously not the most performant thing ever, but it seems to work well for small to medium data sets.

vitorpamplona avatar Apr 09 '24 21:04 vitorpamplona

I don’t know if this nip is a good idea but I would use the concatenation of "database name" + "table name" + "row id" for the “d” tag to ensure uniqueness anyway.

itawa-panema avatar Apr 09 '24 21:04 itawa-panema

"database name" + "table name" + "row id"

oh yeah! good catch!

vitorpamplona avatar Apr 09 '24 21:04 vitorpamplona

I also agree that the d tag should be a composite key. In many databases, primary keys end up as being an incrementing number if not a GUID.

vicariousdrama avatar Apr 09 '24 21:04 vicariousdrama

Yeah, my case was using UUIDs for PKs, so I didn't even notice that issue. :)

vitorpamplona avatar Apr 09 '24 21:04 vitorpamplona

Is this export row by row? Or multiple rows in a table? Or multiple tables in a database?

vicariousdrama avatar Apr 09 '24 21:04 vicariousdrama

Is this export row by row?

Right now it is a trigger in Postgres that runs individually for every insert, update or delete of each row.

For those wondering, I did consider creating one event kind for each table, document those tables as a new NIP and turn all foreign keys into a-Tags to make it more "Nostr native". But it is hard to justify using new event types if there is no expectation of interoperability with anything else. The tables were designed for that application alone.

vitorpamplona avatar Apr 09 '24 21:04 vitorpamplona

But why?

staab avatar Apr 09 '24 23:04 staab

But why?

Well, I need to bring an existing system into Nostr without breaking the production code. So, this is my first step. The next part is to replace the db with Nostr events entirely. Then phase the server out completely and keep only the JS client connecting directly into Nostr. Maybe much later make it interoperable with other Nostr events.

vitorpamplona avatar Apr 09 '24 23:04 vitorpamplona

This is just a way to use nostr relays as a database. The schema is a proprietary format, so I don't see what defining how to fit the schema into nostr accomplishes. You're not going to get any kind of interoperability from this, which is the point of NIPs.

staab avatar Apr 09 '24 23:04 staab

You're not going to get any kind of interoperability from this

Yep, this is just like NIP-78, but for generic DBs instead. The NIP is just documenting what is going on in case this event kind is found in the wild.

vitorpamplona avatar Apr 09 '24 23:04 vitorpamplona

So just use NIP 78 for your proprietary schema events

staab avatar Apr 09 '24 23:04 staab

So just use NIP 78 for your proprietary schema events

I could use the same kind for sure, but since events from this PR can be massive and quite insert/update intensive since the entire app is using only one kind for everything, I thought a new kind could allow relays to block these objects without affecting kind 30078 applications, which seem to be much leaner/slower moving than the ones from this PR.

vitorpamplona avatar Apr 10 '24 00:04 vitorpamplona

My first instinct is in favor of this, of a new kind, and of such a mapping.

mikedilger avatar Apr 10 '24 00:04 mikedilger