wiki
wiki copied to clipboard
feat: save data into pageTree table after processing a batch of pages
While re-building a page tree, all the existing page details are fetched from the database and a tree
array is created at the application side. Once the tree
is created, the pageTree
table is truncated & the objects in the tree
array are saved to the pageTree
table in chunks. When there are a lot of records present in the page
table, it takes a lot of time to build the tree
array at the application side causing the database connections to timeout and it is causing the rebuild-tree job to fail.
In order to avoid idle connections, data should be saved right after enough number of records are added to the tree
array.
The downside of this approach is that the pageTree table will be empty or incomplete during the rebuild. Users will experience empty or broken navigation if the rebuild takes a long time to complete.
The worker maintains a pool of DB connections and this isn't a transaction. So whether the connection times out shouldn't matter in this case. A new one will be established automatically if needed.