instagram4j icon indicating copy to clipboard operation
instagram4j copied to clipboard

Unable to get all users followers - Instagram knows that it is a scrapping tool

Open spapapan opened this issue 3 years ago • 7 comments

I am using this code to get users followers info:

    public static List<Profile> getUsersFollowers(IGClient client, String username)
    {
        Print.e("Getting users followers. Please wait...");

        final List<Profile> result = client.actions().users().findByUsername(username)
                .thenApply(userAction -> userAction.followersFeed().stream()
                        .flatMap(feedUsersResponse -> feedUsersResponse.getUsers().stream()).collect(Collectors.toList())
                ).join();

        Print.e("Total followers:" + result.size());

        return result;
    }

If the user has many followers (>40k) I get this error:

java.util.concurrent.CompletionException: com.github.instagram4j.instagram4j.exceptions.IGResponseException: challenge_required

and then I need to login to my account from instagram official app to put a captcha.

I guess they know that this is a scrape tool because of the loading speed. Is there any way to slow down the code you use to get the list of followers or any other way to get all users followers without any problem?

spapapan avatar Apr 22 '21 16:04 spapapan

I am using this code to get users followers info:

    public static List<Profile> getUsersFollowers(IGClient client, String username)
    {
        Print.e("Getting users followers. Please wait...");

        final List<Profile> result = client.actions().users().findByUsername(username)
                .thenApply(userAction -> userAction.followersFeed().stream()
                        .flatMap(feedUsersResponse -> feedUsersResponse.getUsers().stream()).collect(Collectors.toList())
                ).join();

        Print.e("Total followers:" + result.size());

        return result;
    }

If the user has many followers (>40k) I get this error:

java.util.concurrent.CompletionException: com.github.instagram4j.instagram4j.exceptions.IGResponseException: challenge_required

and then I need to login to my account from instagram official app to put a captcha.

I guess they know that this is a scrape tool because of the loading speed. Is there any way to slow down the code you use to get the list of followers or any other way to get all users followers without any problem?

There are multiple ways to do this; The burden is on you to work that out unfortunately! The library does not have any methods that implement a cooldown or get around detection since this is not the goal of the library.

One option is that you can have a scheduled task execute after a random amount of time that gets the next page of followers.

Although I don't know the exact details for "detection" right now but I do know Instagram had certain daily limits on actions (like comment follow etc.) before they either ask for verification or temporarily ban you from any more requests, Even in the app, you may run into these limits since an average user wouldn't do a massive amount of requests on the app!

jvogit avatar Apr 22 '21 18:04 jvogit

@spapapan this is limit of instagram and you can not catch 40K request easy

lusicsmusics1 avatar Apr 29 '21 08:04 lusicsmusics1

@jvogit My quetion is how to perform those cooldowns using the provided code. Those CompletableFuture's are a real mindf*ck.

spapapan avatar May 02 '21 17:05 spapapan

I am using this code to get users followers info:

    public static List<Profile> getUsersFollowers(IGClient client, String username)
    {
        Print.e("Getting users followers. Please wait...");

        final List<Profile> result = client.actions().users().findByUsername(username)
                .thenApply(userAction -> userAction.followersFeed().stream()
                        .flatMap(feedUsersResponse -> feedUsersResponse.getUsers().stream()).collect(Collectors.toList())
                ).join();

        Print.e("Total followers:" + result.size());

        return result;
    }

If the user has many followers (>40k) I get this error:

java.util.concurrent.CompletionException: com.github.instagram4j.instagram4j.exceptions.IGResponseException: challenge_required

and then I need to login to my account from instagram official app to put a captcha.

I guess they know that this is a scrape tool because of the loading speed. Is there any way to slow down the code you use to get the list of followers or any other way to get all users followers without any problem?

Did you find any solution about this ?

arslantugrul66 avatar Jun 24 '21 21:06 arslantugrul66

You're basically sending too many requests, scraping that much isn't possible, except you take long sleeps

On Thu, 24 Jun 2021, 10:26 pm arslantugrul66, @.***> wrote:

I am using this code to get users followers info:

public static List<Profile> getUsersFollowers(IGClient client, String username)
{
    Print.e("Getting users followers. Please wait...");

    final List<Profile> result = client.actions().users().findByUsername(username)
            .thenApply(userAction -> userAction.followersFeed().stream()
                    .flatMap(feedUsersResponse -> feedUsersResponse.getUsers().stream()).collect(Collectors.toList())
            ).join();

    Print.e("Total followers:" + result.size());

    return result;
}

If the user has many followers (>40k) I get this error:

java.util.concurrent.CompletionException: com.github.instagram4j.instagram4j.exceptions.IGResponseException: challenge_required

and then I need to login to my account from instagram official app to put a captcha.

I guess they know that this is a scrape tool because of the loading speed. Is there any way to slow down the code you use to get the list of followers or any other way to get all users followers without any problem?

Did you find any solution about this ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/instagram4j/instagram4j/issues/581#issuecomment-867962823, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFB4NEVDXPBA7A3SNR7MYQ3TUOPKBANCNFSM43M3IQYA .

Oyetomi avatar Jun 24 '21 21:06 Oyetomi

I use the same code with @spapapan but I do not know how to solve this on code. I am also struggling to get all of the likers. For example, there are 140k likers but when I use the MediaGetLikersRequest() method, I get first 1k likers. How to get all of the likers any idea?

arslantugrul66 avatar Jun 24 '21 21:06 arslantugrul66

My understanding is that instagram only shows first 1000 of likers.

On Thu, 24 Jun 2021, 10:43 pm arslantugrul66, @.***> wrote:

I use the same code with @spapapan https://github.com/spapapan but I do not know how to solve this on code. I am also struggling to get all of the likers. For example, there are 140k likers but when I use the MediaGetLikersRequest() method, I get first 1k likers. How to get all of the likers any idea?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/instagram4j/instagram4j/issues/581#issuecomment-867970835, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFB4NEVJ6QDW4WJ7PB4SWODTUORG3ANCNFSM43M3IQYA .

Oyetomi avatar Jun 24 '21 21:06 Oyetomi