AzureRmStorageTable icon indicating copy to clipboard operation
AzureRmStorageTable copied to clipboard

Published 20 hours ago • verified publisher icon paulomarquesc

Batch operation support: offer

Open davidroberts63 opened this issue 3 years ago • 4 comments

This is providing batch capabilities for adding records. I have not built out tests or docs, but I will do so if this PR is of interest, as an initial contribution offer.

If this is of interest I also have a question about the Save-AzTableBatch name. The 'Save' verb sounds about right, but I'm easily convinced to use something else.

davidroberts63 avatar Jul 19 '21 19:07 davidroberts63

Hi @davidroberts63 , this seems very interesting, have you noticed significant performance improvements on this one?

paulomarquesc avatar Aug 31 '21 17:08 paulomarquesc

Yes. In my situation I was processing about 1.9 million records. Doing them in batches of 100 (the limit for a TableBatch) took my processing from 10-20 records per second, for individual record operations, to 150-200 records per second for batching. It also reduces the number of transactions involved in the cost of the table storage account.

davidroberts63 avatar Aug 31 '21 18:08 davidroberts63

Cool, thanks for the info, I think we should move forward with this PR.

To move forward, a few comments:

  1. Regarding Save-AzTableBatch, the Invoke verb seems more appropriate in this case.
  2. Add the test case
  3. Let's add the cmdlet help at (https://github.com/paulomarquesc/AzureRmStorageTable/tree/master/docs)
  4. Update the main documentation (docs/README.md) to have an end-to-end example.

paulomarquesc avatar Sep 17 '21 18:09 paulomarquesc

Hi @davidroberts63, are you still going to continue with this PR? If, yes, can you please provide the changes I asked in my previous comment?

paulomarquesc avatar Feb 02 '22 00:02 paulomarquesc