dgraph
dgraph copied to clipboard
fix(snapshot): Enable snapshot streaming after bulkload
Description: Enable snapshot streaming after bulkload
Summary: Previously, the bulkloaded p directory couldn't stream to a new alpha due to a commitTs of zero. In this PR, the commitTs is sourced from the p directory, allowing the alpha to create a snapshot and subsequently stream it to another alpha.
Tests:
- TestBulkLoaderSnapshotPDirinAlpha0: Load a
pdirectory using bulkload. Then, initiate one alpha using the bulkloadedpdirectory and start a second alpha without anypdirectory. Query both alphas to ensure the snapshot has been successfully generated and that the data is accessible from both instances. - TestBulkLoaderSnapshotPDirinAll: Load a
pdirectory using bulkload. Then, copy bulkloadedpdirectory in all alphas and start the cluster. Query all alphas to ensure that the data is accessible from all instances. - TestBulkLoaderDataLoss: Move the
zerotimestamp and load apdirectory using bulkload. Then use thispdirectory on a fresh cluster (both zero and alpha are new). Validate that the query doesn't work without moving the timestamp of the new zero.
Closes: https://dgraph.atlassian.net/browse/DGRAPHCORE-214
Docs: NA
We need more tests as we discussed, a few questions and minor comments are dgraphtest package
I have added more tests and extended the dgraphtest package to support some of the functions required for the new tests. This PR is ready for another round of reviews.
This PR has been stale for 60 days and will be closed automatically in 7 days. Comment to keep it open.