capgo

capgo copied to clipboard

For a period i did set a ip ban for google IP sadly it was also blocking some user with vpn. i need to find a better option

/bounty $100

💎 $100 bounty • Capgo

Steps to solve:

1. Start working: Comment /attempt #229 with your implementation plan
2. Submit work: Create a pull request including /claim #229 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Additional notes:

ℹ️ If something is not clear ask before working on it, otherwise your chance to rework it is high 🎥 To claim you need to provide in your PR a demo video of the change 👨‍👩‍👧‍👦 Join the Discord to get help 📏 Check all Bounty rules

Thank you for contributing to Cap-go/capgo!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🔴 @rishi-raj-jain	Aug 13, 2023, 6:21:48 PM	WIP
🔴 @ologbonowiwi	Oct 13, 2023, 7:19:17 PM	WIP
🔴 @WcaleNieWolny	Oct 13, 2023, 7:24:33 PM	WIP

/attempt #229

where do we want the ban to be applied? on the web? more details would help!

Options

• Cancel my attempt

this is for the update endpoint. when users have CI/CD to publish to google play store then google play is starting the app and request live update. who are counted in users billing plan. So the goal is to be able to know what is a google bot device and block the update, to not count in billing

On it!

This is partially fixed by https://github.com/Cap-go/capgo/pull/314 correct?

if redis does get merged can I just add a fn in the updates_cached that will call supabase, check if device exists in cache and if it does not then return the cached version (if possible). Here is how I see it:

1. Only if app exists in redis and the requested version is cached
2. Check if device exists in cache, if it does not create it as `standard` (not overwritten) and set some sort of value that indicates the requests (like for example `device_{ID}_usage` to 1
3. When this usage > 5 (or any other threshold) then call the update endpoint

This will fix this by forcing the google devices to never exceed this threshold thus the user will not be billed for them

The downside of this is that if a device does not send at least 5 requests then it cannot be overwritten in the dashboard. Also if app calls setChannel then we need to ensure that it will work even if there is not such device in the supabase db. setChannel is a gigantic pain when caching

@WcaleNieWolny isn't that the same as I've been suggesting in https://github.com/Cap-go/capgo/pull/267 to @riderx?

Perhaps, I was not sure what "forever blocked" ment. I proposed this as I submitted a PR with redis and it was just a thought

@riderx mentions in https://github.com/Cap-go/capgo/pull/267#issuecomment-1678265784 that redis isn't an option, but anyways he refers to storing lot of IPs as an expensive operation. In any case, of rate limiting by 5 attempts or rate limiting by 1 attempt (that is within the last time vs new time wherever the user should be able to update), we're storing IPs with unique param (say app_id) of each update request that comes. But if Redis becomes part of the stack, the cost of Upstash is like 0.2$ per 100K which I think would be fine for @riderx?

Perhaps, I was not sure what "forever blocked" ment. I proposed this as I submitted a PR with redis and it was just a thought

Hope you didn't take me the wrong way, I was just confirming

that redis isn't an option

A lot of things happened after that comment. Capgo got a gigantic client that caused a lot of downtime as capgo was not able to scale. Since then adding redis became an option.

I already implemented caching for /updates and I wanted to build upon that. This implementation works with device id and NOT the IP. I believe using this device_id is enough. Why would we store the ip if we can just respond with the cached response without ever calling the updates endpoint for new devices that have not yer made at least 5 requests? This would essentially create zero records in supabase and remove the unnecessary cost from the users bill.

Regardless, I am sorry if I appeared as if I was trying to steal your idea

Regardless, I am sorry if I appeared as if I was trying to steal your idea

Nah, that's why I clarified here at https://github.com/Cap-go/capgo/issues/229#issuecomment-1703913414

Go ahead, rock it!

Thanks guys for putting your 2 brain on it ! I value your opinion but I wonder if such practices ( the 5 first request idea ) Will not reduce the accuracy of stats about usage, and also bring weird behavior for set channel like happening too late when user expect instant update. For me the best idea for Google issue is more finding a way to identify and ban the right IP

Yeah, for that, @riderx we can tweak my solution of storing into redis as 1 or forever flag to storing the "hot" IPs by using redis keys with expiry. Shall I elaborate more?

So imagine only 10 hot IPs and 1M not so hot IPs,

We store all of them in the redis with set to expiry say X (an inbuilt functionality of redis). If the time crosses X and we see no re-occurence the key is dropped automatically from your database. If we do receive, it's kept in your database for an increased number of the count.

This takes care of the database maintaining itself.

Also, note, we have to decide what is considered as a hot IP. Let's say if any IP gets above Y updates refs in the period X, we'll blacklist it.

That's what I meant, let me know if this makes sense or needs a POC, I'm good at those :D

There is no way to do a PoC right now, redis is still not merged

You could expand on my redis branch but right now I am not sure when will it get merged

I mean we don't need the whole redis PR to be merged for this, this'll be super quick with just using the upstash package in the update endpoint.

Okay, go for the PoC 👍

Awesome! I'll be on it in a day.

What still needs to be done on this one?

I'm up to it @WcaleNieWolny @riderx.

/attempt #229

Options

• Cancel my attempt

Mainly a actual test run with a published play store app if I understand correctly

So we're already blocking it, we only need to check if the block is working?

I'm on it! Had a discussion with @riderx where I need to add an endpoint for this

On it after the CF thing we're doing today

Oh sorry for it, your pr was closed I thought you canceled your attempt.

Go for it mate!

wait what

omg I deleted the cap-go/capgo for a fork shit