cortx
cortx copied to clipboard
Seagate
Cortx Request Rate Limit
Is there a default limit to Cortx's request rate?
I am testing out the H2O integration and I'm hitting the below error, which seems to indicate the request rate is too high. Is there any way to configure the request rate limit?
Reduce your request rate (Service: Amazon S3; Status Code: 503; Error Code: ServiceUnavailable;
Request ID: 00000000-0000-0000-0000-000000000000; S3 Extended Request ID: null; Proxy: null)
The step that failed is under section 7
# Load data into H2O from Seagate CORTX Server
df = h2o.import_file(path = "s3://testbucket/h2o.csv")
For the convenience of the Seagate development team, this issue has been mirrored in a private Seagate Jira Server: https://jts.seagate.com/browse/CORTX-29714. Note that community members will not be able to access that Jira server but that is not a problem since all activity in that Jira mirror will be copied into this GitHub issue.
Hi @hseow-seagate thank you for raising the issue, I haven't faced rate limit on CORTX before, I'll also try the H20 integration and reproduce the same.
Hi @hseow-seagate I have tested the integration on my end using k8s OVA, I'm getting the same error. However when you reduce the h2o.csv file from ~10k rows to 50 rows the error is not raised.
Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%
@johnbent you are one of the contributors of the notebook, could you please let us know if you tested the integration successfully in the past. Is there an explanation as to why when the file size is reduced the h20 import file function works.
@shailesh-vaidya Is there a default limit to Cortx's request rate?
Hi @r-wambui , I merged the pull request and now github mistakenly tags me as a contributor. I'm afraid that I did not test this myself.
This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.
Hi @hseow-seagate I have tested the integration on my end using k8s OVA, I'm getting the same error. However when you reduce the h2o.csv file from ~10k rows to 50 rows the error is not raised.
Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%
@johnbent you are one of the contributors of the notebook, could you please let us know if you tested the integration successfully in the past. Is there an explanation as to why when the file size is reduced the h20 import file function works.
@shailesh-vaidya Is there a default limit to Cortx's request rate?
Hi @r-wambui - Sorry for the delayed reply. I am not aware of any request rate limit on the CORTX side. @bkirunge7 Can you please help on this.
This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.
This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.
Hi @r-wambui, I just tested the H2O integration with the original h2o.cvs (9.5k rows), and did not encounter any error.
I'm using the latest OVA-2.0.0-713 (PI-6).
Suggest you try the latest OVA too.
This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.