tg-archive icon indicating copy to clipboard operation
tg-archive copied to clipboard

Todo: Incremental builds

Open knadh opened this issue 3 years ago • 5 comments

For large groups, re-publishing and re-uploading every page isn't ideal. There should be a mechanism to build incrementally, maybe with an optional --incremental flag.

  1. The dynamic date index on the left sidebar is present in every single page. This should become a standalone page and be iframed in all other pages so that it can change independently without modifying older pages.
  2. What about pagination? As the month progresses, new page numbers appear at the bottom of every page for that month. Make this also an iframe? Or let the current month be rebuilt every time anyway as a trade-off?

knadh avatar Dec 02 '21 15:12 knadh

This is a great idea! Rebuilding the left sidebar on each run and iframing it into every page looks straightforward. We can also assume that per_page will not be changed for an existing site and iframe the page number section, so every month gets its own iframe.

Incrementally building pages seems complex, the last page could have less messages than the config[”per_page”]. We can work around this by rebuilding this last page, which requires figuring out its filename and the message ID of the first message on this page. Linking message replies between old and new pages will also need a few steps. To get a page’s first message or the page number a message belongs to:

  • We can make calculations based on per_page and row number of a message (assuming the per_page didn’t change) to get the required page numbers.
  • We could store the message ID range for each page like: (start_msg_id, end_msg_id, filename). This would be simpler and cleaner. It also tells us up to which ID the messages have been published.

faraazb avatar Dec 23 '21 19:12 faraazb

Thanks @faraazb. Yep, it's probably best to store the data in a new table, all behind a new --incremental=true|false flag.

knadh avatar Dec 24 '21 03:12 knadh

image This feature is very useful for me and I would like to know if there is any development progress so far? Can I contribute to it?

Seele0oO avatar Feb 18 '22 10:02 Seele0oO

Also wondering How I could contribute, this is the missing piece of this great piece of code!

benborges avatar May 26 '22 17:05 benborges

@faraazb's comment lays out a good starting point if you would like to attempt this. Thanks.

knadh avatar May 27 '22 05:05 knadh