More about storing state in Elixir, please!

[nathanl]

Hi! I enjoyed "Episode 104 - Storing State in Elixir Apps" and would love to hear more on the subject.

I've used GenServer-based state in a couple of apps, but never as a real "source of truth". Most recently I used it to store an XML sitemap which would be served to search engines. But I could recalculate that at any time based on the routes and the database.

For storing data reliably, I agreed with the objections Desmond made, and would add a lot more. I want transactions, foreign keys, types, and uniqueness and exclusion constraints to make my data consistent and prevent race conditions - I've written about this here. I want indexes and joins and database views and GROUP BY and fulltext search to power my queries.

"You might not need a database" sounds to me like "you might want to roll your own NoSQL database". 😆

Admittedly I have a very SQL-based mindset, and PostgreSQL is a favorite tool. I'm afraid I'm missing something.

Wouldn't it be interesting to bring Joe Armstrong on the show (or someone else who says "you might not need a database") and get some real-world use cases from them?

(I'd happily volunteer to come on and bring questions, if you like.)

[nathanl]

Relevant tweet: https://twitter.com/joeerl/status/940856646095593472

[michalmuskala]

I listened to the episode recently as well, and there are definitely examples where you don’t need a database. Two practical examples of applications I’ve built:

• a business application powered by an event sourced system, that stores events in a persistent store, but then the derived state is fully in memory (in processes and ets tables). We can then easily rehydrate the in-memory state from the event log.

• https://github.com/michalmuskala/hex_view which is going to be a source code explorer for hex packages. The registry is stored in ets and rehydrated from the main hex API and the code is just stored in the file syststem.

[nathanl]

a business application powered by an event sourced system, that stores events in a persistent store, but then the derived state is fully in memory (in processes and ets tables). We can then easily rehydrate the in-memory state from the event log.

I'm curious about this. What's the persistent store - a db, flat files, or what? Is the hydrated data relational in shape? Can new events be rejected based on the current state of data - the kind of thing you'd use constraints for in a db? Does the app take longer to boot when there are more historical events, and do you reboot on deploy?