mentorship-backend icon indicating copy to clipboard operation
mentorship-backend copied to clipboard

Feature: Setting a rate limit on the endpoints using flask limiter

Open epicadk opened this issue 4 years ago • 30 comments

Is your feature request related to a problem? Please describe.

Heroku Does offer DDoS mitigation however, according to this post they strongly recommend using a rate limiter library.

Describe the solution you'd like

Rate limit the endpoints of the api using flask limiter.

Additional context

The time and amount of requests that should be allowed should be discussed, possibly in the mentorship session.

epicadk avatar Feb 18 '21 03:02 epicadk

Marking this as status on hold because it needs to be approved by @isabelcosta .

epicadk avatar Feb 18 '21 06:02 epicadk

@epicadk this sounds amazing. This is the kind of thing I only learned this year, so I really appreciate you coming with these "out of the box" ideas 🤗 I will approve this. Could you link the library in the description of the issue and add a little more information if you have more.

isabelcosta avatar Feb 22 '21 23:02 isabelcosta

@isabelcosta 😅 Thankyou : ) . I have linked the library in the issue however I haven't used it much so I don't really have any more information about it. We also need to discuss the limit or the amount of times that a user should be allowed to query the backend.

epicadk avatar Feb 23 '21 20:02 epicadk

thank you so much @epicadk !

isabelcosta avatar Feb 23 '21 21:02 isabelcosta

I'd like to work on it I'll be in the next open session to discuss about this issue.

jalajcodes avatar Mar 02 '21 03:03 jalajcodes

I'd like to work on it I'll be in the next open session to discuss about this issue.

I think we already discussed it. Since Isabel has approved I'll assign you. Happy coding : ) .

epicadk avatar Mar 02 '21 04:03 epicadk

I'd like to work on it I'll be in the next open session to discuss about this issue.

I think we already discussed it. Since Isabel has approved I'll assign you. Happy coding : ) .

Ohk! Do you mind sharing the meeting docs link? I'd like to go through the discussion.

jalajcodes avatar Mar 02 '21 07:03 jalajcodes

I'd like to work on it I'll be in the next open session to discuss about this issue.

I think we already discussed it. Since Isabel has approved I'll assign you. Happy coding : ) .

Ohk! Do you mind sharing the meeting docs link? I'd like to go through the discussion.

I went through the docs and didn't find any thing related to this issue so maybe I was mistaken. But isabel has apporved it so you can work on it. What we need to discuss is the rates for each endpoint. You can still work on it and submit a pr. Although you might have to change only the rates later on so up to you.

epicadk avatar Mar 02 '21 09:03 epicadk

I'd like to work on it I'll be in the next open session to discuss about this issue.

I think we already discussed it. Since Isabel has approved I'll assign you. Happy coding : ) .

Ohk! Do you mind sharing the meeting docs link? I'd like to go through the discussion.

I went through the docs and didn't find any thing related to this issue so maybe I was mistaken. But isabel has apporved it so you can work on it. What we need to discuss is the rates for each endpoint. You can still work on it and submit a pr. Although you might have to change only the rates later on so up to you.

Ok cool I'll start working on it then ;)

jalajcodes avatar Mar 02 '21 09:03 jalajcodes

@jalajcodes hey any updates?

vj-codes avatar Mar 06 '21 07:03 vj-codes

@mtreacy002 I think this issue can cause problems for BIT. we'll have to exclude the BIT server from the rate limiting.

epicadk avatar Mar 06 '21 09:03 epicadk

@jalajcodes hey any updates?

I'll create PR tomorrow or if time permits, today

jalajcodes avatar Mar 06 '21 09:03 jalajcodes

@jalajcodes take your time, just update the progress after every 3 days if it's taking long

vj-codes avatar Mar 06 '21 09:03 vj-codes

@mtreacy002 I think this issue can cause problems for BIT. we'll have to exclude the BIT server from the rate limiting.

@epicadk , I agree, this could cause issue if the rate limiter sets the amount of requests BIT can do, since in BIT-MS integration the number of requests sent would be extensive 🤣🤣. If possible, @jalajcodes, please share the approach you're thinking of doing (e.g. what type of limiter you will apply here), so we can assess the impact to BIT better. Perhaps we should add CROSS-PORJECT ISSUE on the title so I can monitor the progress made here as it would impact BIT. cc @vj-codes and @isabelcosta

mtreacy002 avatar Mar 07 '21 02:03 mtreacy002

If possible, @jalajcodes, please share the approach you're thinking of doing (e.g. what type of limiter you will apply here), so we can assess the impact to BIT better.

As we haven't decided rates for specific endpoints yet, I am thinking of adding a generous rate limit globally for all endpoints and exempt it for those endpoints which don't need to be rate limited like login etc.

regarding the BIT-MS integration, I think flask-limiter allows us to customize rate limits to be based on characteristics of the incoming request. I am not sure but I guess we could add some kind of check to determine if the request is coming from BIT and increase the limit or maybe disable it completely?

jalajcodes avatar Mar 07 '21 05:03 jalajcodes

If possible, @jalajcodes, please share the approach you're thinking of doing (e.g. what type of limiter you will apply here), so we can assess the impact to BIT better.

As we haven't decided rates for specific endpoints yet, I am thinking of adding a generous rate limit globally for all endpoints and exempt it for those endpoints which don't need to be rate limited like login etc.

regarding the BIT-MS integration, I think flask-limiter allows us to customize rate limits to be based on characteristics of the incoming request. I am not sure but I guess we could add some kind of check to determine if the request is coming from BIT and increase the limit or maybe disable it completely?

Login is probably the endpoint we want to rate limit first. 😄 .

epicadk avatar Mar 07 '21 05:03 epicadk

@epicadk 😂😅 my bad, just assume any other endpoint that doesn't require limiting

jalajcodes avatar Mar 07 '21 06:03 jalajcodes

I would like to take up this issue if it's available.

Anmollenka avatar Mar 13 '21 03:03 Anmollenka

I'll create PR today

jalajcodes avatar Mar 14 '21 01:03 jalajcodes

Hi @epicadk and @isabelcosta. Please assign this issue to me if it is open :)

battuAshita avatar Jun 14 '21 07:06 battuAshita

@battuAshita are you still interested in the issue?

isabelcosta avatar Aug 22 '21 10:08 isabelcosta

Hi @isabelcosta. I am actually caught up in some work. It is free to be assigned to someone else. Thank you :)

battuAshita avatar Aug 22 '21 11:08 battuAshita

No worries @battuAshita ! thank you for responding so quickly 🤗 in this way we can assign to someone else who is interested!

isabelcosta avatar Aug 22 '21 11:08 isabelcosta

Hi @isabelcosta. Is this open to work on? If yes, can you please assign this to me? Thanks!

sakshivij avatar Oct 03 '21 21:10 sakshivij

Assigning you @sakshivij Happy coding!

vj-codes avatar Oct 03 '21 21:10 vj-codes

@vj-codes to decide on rate-limit was there any doc created as mentioned on the closed PR? Or I would need to create one?

sakshivij avatar Oct 06 '21 19:10 sakshivij

@epicadk can you answer that please? ⬆️

vj-codes avatar Oct 06 '21 19:10 vj-codes

@epicadk can you answer that please? ⬆️

That's something we need to discuss in the mentorship session. I think you could probably use a constant right now and then change it later? cc @isabelcosta

epicadk avatar Oct 06 '21 19:10 epicadk

@epicadk can you answer that please? ⬆️

That's something we need to discuss in the mentorship session. I think you could probably use a constant right now and then change it later? cc @isabelcosta

Makes sense. If you mean the sync up session, I'll try to join as well.

sakshivij avatar Oct 06 '21 19:10 sakshivij

Yes the sync up session, it is conducted on Saturdays biweekly at 6:30 pm IST The next one scheduled is this Saturday

vj-codes avatar Oct 06 '21 20:10 vj-codes