Archiving streams, snapshots and async projections

[elexisvenator]

More of a discussion than an issue.

I've been putting together ideas for how to remodel one of our apps that uses marten events. Currently the main business entity - a versioned document - has its own stream. I see this as a problem as these documents tend to live forever and can have ongoing activity. The new design would split out parts into their own streams - most importantly each "document version" is its own stream. This along with some other changes I think is a good potential design, but it led to me thinking about archiving. So here is what I am considering:

• Event streams track changes to a document over time.
• 90% of the time, you only care about the latest shape of the document.
• Once a document is "complete", you would only care about the latest shape 99.9% of the time. The other 0.1% is auditing.
• If a document is to be archived, this would involve an event being written. There may be projections that need to consume this event.

So my questions are:

• If I want to archive a stream after this last event, how can I reliably ensure that async projections have processed this event first?
• If I want to make a "snapshot" of an archived stream, is the current projection system the right way to do this? Is there a better or future way instead?
• If I build a new projection that needs to be aware of archived streams, can it catch up these streams (suspect this is the answer)

I think the first question is the most important, as I don't like writing code that is essentially "wait until highwater mark is equal to this event sequence id".