Replibyte icon indicating copy to clipboard operation
Replibyte copied to clipboard

Add Buffer Size Parameter to dump create

Open gugacavalieri opened this issue 2 years ago • 5 comments

Changelog

  • Add Argument to set buffer_size when dumping the database
  • Set the default size to 100 (This was the previous value)

Motivation

I was running into the same issue when dumping tables bigger than 100MB (https://github.com/Qovery/Replibyte/issues/208). The order of the Statements were wrong and I was not able to restore the dumps. I still don't know 100% what is cause the issue but it's probably related when the compressed chunks are being read from the datastore and are being restored in the wrong order.

So, this PR aims to make the arguments more flexible and serve as a workaround for the issue above.

PS. I am still learning Rust, so please let me know if this is a good way to move parameters around 😄

gugacavalieri avatar Nov 05 '22 19:11 gugacavalieri

Thank you for the contribution. I'm going to take a look this weekend

evoxmusic avatar Nov 05 '22 19:11 evoxmusic

Good idea! I will do some work on the fixes.

gugacavalieri avatar Nov 06 '22 20:11 gugacavalieri

@evoxmusic, added dump_chunk_size to the SourceOptions struct.

Also have to play a little bit with the test methods 😃

gugacavalieri avatar Nov 20 '22 20:11 gugacavalieri

@gugacavalieri can you check tests errors?

evoxmusic avatar Dec 12 '22 10:12 evoxmusic

Thanks for adding this, can you also update the README with an example of how to use this?

michaelrode avatar Oct 05 '23 13:10 michaelrode