unpywall icon indicating copy to clipboard operation
unpywall copied to clipboard

Test cache performance with larger amounts of data

Open bganglia opened this issue 6 years ago • 5 comments

This issue has been separated from #18

It may be necessary to revise the way the cache saves data to be higher-performance. It would still be necessary to append and delete entries

bganglia avatar Apr 16 '20 12:04 bganglia

An alternative interface for the console would also fit here. So without calling the python interpreter and importing the module manually. We can use argparse for that (https://docs.python.org/3.7/library/argparse.html).

And then something like

unpywall --doi 10.324/dsgsdg --backend cache

naustica avatar Apr 16 '20 15:04 naustica

ok my comment is maybe more related to issue #18

naustica avatar Apr 16 '20 15:04 naustica

@naustica That is good. One of the users who I was talking to actually said that a command-line interface would be useful for her.

What would the backend options be? remote, cache, data_dump...

bganglia avatar Apr 16 '20 18:04 bganglia

@bganglia ok, then I would work on a command-line interface if you dont mind.

I think the options would be enough for now. Maybe we should use the name snapshot instead of data_dump.

naustica avatar Apr 16 '20 19:04 naustica

@bganglia there exists a requests cache package: https://requests-cache.readthedocs.io/en/latest/user_guide.html. They use a sqlite database and also a filter for http responses. Maybe this helps.

naustica avatar Apr 29 '20 13:04 naustica