XRPLF
Mitigating huge requests (Data analysis)
Since Python is more focused on data related projects, wouldn't it be logical to have functions on xrpl-py that allows users to retrieve massive amounts of data efficiently without damaging a node's overall performance.
Possible solution(s):
- Mitigating huge requests in small organized bunches:
500 tx request
1 second cool down
500 tx request
1 second cool down
- Mitigating huge requests to configurable set of nodes (s1/s2/xrplf/cluster/default list) and using solution 1 to not effect the nodes' overall performance.
Use case:
- A massive amount of data needed to be retrieved on the XRPL (an automated trading account) E.g: 3-10 Offer transactions issued every few seconds nonstop (Account: rETx8GBiH6fxhTcfHM9fGeyShqxozyD3xe).
- When a large scale investigation is going on, software like Palantir could compute and organize massive amounts of data but it needs to retrieve this data efficiently.
- Studying a token issuer's transaction flow and overall behavior.
I have a client doing 9000 tx airdrops every week if you want to test "robust transaction processing"?
I can invite you to the private repo and you can see how we batch them together.
5 tx batchs (pause 1 second on each tx and 5 seconds on batch) 100 tx batch array (pause 5 seconds on batch)
To all the people that think I'm accusing @hammertoe of suppressing a token's price, I am not in any way, shape, or form.
I was simply researching the MM account as someone publicly disclosed on the XRPL devs discord server that this account was doing something 'suspicious' with a token, I was not aware Matt was the one who was operating this account nor am I accusing anyone of anything, I was simply researching independently.
I wanted to retrieve this MM account's transactions on a rippled API service but it was denied due to the request being large in size (which was understandable). This MM account is not damaging the XRPL's performance nor is it damaging any node, please stop.
