dgraph
dgraph copied to clipboard
fix(snapshot): Enable snapshot streaming after bulkload
Description: Enable snapshot streaming after bulkload
Summary: Previously, the bulkloaded p
directory couldn't stream to a new alpha due to a commitTs of zero. In this PR, the commitTs is sourced from the p
directory, allowing the alpha to create a snapshot and subsequently stream it to another alpha.
Tests:
-
TestBulkLoaderSnapshotPDirinAlpha0: Load a
p
directory using bulkload. Then, initiate one alpha using the bulkloadedp
directory and start a second alpha without anyp
directory. Query both alphas to ensure the snapshot has been successfully generated and that the data is accessible from both instances. -
TestBulkLoaderSnapshotPDirinAll: Load a
p
directory using bulkload. Then, copy bulkloadedp
directory in all alphas and start the cluster. Query all alphas to ensure that the data is accessible from all instances. -
TestBulkLoaderDataLoss: Move the
zero
timestamp and load ap
directory using bulkload. Then use thisp
directory on a fresh cluster (both zero and alpha are new). Validate that the query doesn't work without moving the timestamp of the new zero.
Closes: https://dgraph.atlassian.net/browse/DGRAPHCORE-214
Docs: NA
We need more tests as we discussed, a few questions and minor comments are dgraphtest package
I have added more tests and extended the dgraphtest package to support some of the functions required for the new tests. This PR is ready for another round of reviews.
This PR has been stale for 60 days and will be closed automatically in 7 days. Comment to keep it open.