pgdumplib icon indicating copy to clipboard operation
pgdumplib copied to clipboard

Performance Enhancement

Open SugaanthMohan opened this issue 5 years ago • 4 comments

I am using your package and the code works great. Thank you so much for your contribution.

And I was thinking that you could integrate your code with dataframe packages like pandas as well.

It could increase the code usage speed and by nature databases do tend to be huge, that could really bring your package to the next level.

A good Example would be :

Your current structure: (Dump.py)

self.entries = [] ( Existing List attribute )
self.entries.append( Entry(...) ) ( Existing Method)

Enhancement suggestion structure: (Dump.py)

import pandas as pd
self.dataset = pd.DataFrame() ( Similar Method )
self.dataset.append(Entry(...).__dict__) ( Similar Method )

Hope you get the idea. And thanks again for your contribution, you help developers like us make a living from coding. :)

SugaanthMohan avatar Jul 03 '20 22:07 SugaanthMohan

Thanks for this. I noticed you closed it, does that mean you changed your mind or?

gmr avatar Jul 06 '20 01:07 gmr

It was by mistake, since I left a review from mobile.

SugaanthMohan avatar Jul 06 '20 08:07 SugaanthMohan

So to integrate it with Pandas as a hard requirement is not something I'm keen to do, but I will keep it in mind from an optional perspective.

gmr avatar Oct 01 '20 23:10 gmr

Thanks GMR. My review was based on a real-time example that, when I used pgdumplib for a database with objects (not data) for a database around 600 MB.

It did take a good chunk of time for me to load them, and then again organise them so that i can perform my cherry picking and other operations. But none the less, this is an excellent tool for the future, Since I do not see such functionalities anywhere being provided for Postgresql.

Thanks for your consideration.

SugaanthMohan avatar Oct 02 '20 20:10 SugaanthMohan