IP Blocked from Newegg

[headbasha]

Do you have time to update with a random interval to add items to the cart for purchase? It took only about an hour to get IP blocked from newegg for 24 hours.

[retrac752]

I also got ipblocked by newegg for 24 hours after running the bot for about 2 hours

[retrac752]

i mean to make the interval random, I'm replacing await page.waitForTimeout(config.refresh_time * 1000) with await page.waitForTimeout((config.refresh_time + (Math.random() * config.refresh_time * 2)) * 1000) this also makes the refresh up to 3x slower to make it seem more human like, but would this really be enough to trick newegg?

[headbasha]

I usually get IP banned within an hour but im also looking at about 15 different newegg products trying to buy a 3080. Im having to use my VPN to just keep refreshing IP's. Ill try that random number string and see how soon it IP bans.

[mateuszdrab]

I had to put that line somewhere else but it doesnt work for me totally. Est total field shows empty and the script doesn't know what to do.

[Codolophier]

this also makes the refresh up to 3x slower to make it seem more human like, but would this really be enough to trick newegg?

@retrac752 This is how I've been running. I placed a comment in Issue #12 about 10 days ago with the same pattern to it. I've been running my bot almost 24x7 since then , no ban so far :) I pushed the code, along with some other changes, to my fork earlier today. I'm not sure if @Ataraksia is watching this repo anymore, but I submitted a PR about 2 weeks ago that contained updates to accommodate for changes in the checkout flow by Newegg.

[retrac752]

@Codolophier I changed my code to randomize the time but also noticed and fixed the same bug you did where the bot would check the url for "ShoppingCart" instead of "cart" which would always fail, so we would never actually enter the check cart function to call the delay, so I'm unsure if it was the fact that this bot was refreshing without delay or if the randomization actually helped

I've also ran the bot for almost 24 hours with no problems since this fix. However, since I've never entered the checkout flow, I didn't realize there was an issue there. Have you successfully checked out of newegg with your code changes that looks for the secure checkout button?

Also, is chrome better than firefox for this?

[mateuszdrab]

I pretty much had to rewrite the whole thing. Nothing worked, it seems the UI changed or something. I had to make it go through each step of the checkout and then hit place order but at this point I think my credit card will kill it all with some verifications

[Codolophier]

@retrac752

Have you successfully checked out of newegg with your code changes that looks for the secure checkout button? Yes, I've specified an item number that is in-stock and the bot successfully completed the purchase.

Also, is chrome better than firefox for this? Either should be fine.

[Codolophier]

@mateuszdrab Are your account defaults all set for payment method as well as the billing and shipping address? When I first tried the bot, I was getting all kinds of prompts to select and confirm things. Once I set all of my defaults, the only input required was the CVV.

[mateuszdrab]

@Codolophier yes the defaults are set but since I've never purchased anything from them that might be why. Can you cancel an order straight after placing it?

[Codolophier]

Can you cancel an order straight after placing it?

@mateuszdrab Yes, you can. However, if you want to test the bot's purchasing flow, you can just change the auto_submit value in the config.json to false. This way, the bot will got all the way up to the CVV input, but won't proceed any further. Use a cheap item though, just in case :) For example, a Micro SD card (e.g., N82E16820313309).

A note on the Newegg account defaults: Originally, I had some problems with the flow. I thought that all of my defaults were set, but I was still getting prompted for payment and delivery method and billing address. Long story short, I had to go through my account settings to the Address Book and give a name to my default address. Then, in the Payment Options, despite having my credit card as the default payment, I removed any other payments types - such as PayPal. Also under the Payment Options, under the selected card, I selected the Select from Address Book option in order to set my Billing Address to the one I had saved earlier and given a name to. I then gave the card a name, clicked the Make Default box, then the checkout flow no longer prompted me for additional information.

[technaustin]

N82E16820313309

I just did this test with a cheap item, and set to false, and it did work once, for a cheaper item. When I tried it with something more expensive, I ran into the same problem I have been having with GFX cards. Here is a snapshot of the N95 pop up, with my PowerShell Window. It looks like sometimes, this window pops up and interrupts the bot. The good news is we can replicate the issue with the false flag. I tried on a combo item Combo.4210760. We could tell it not to bother us again, but once the cache is gone it comes right back, and the browser is new each time. Also, there isn't a way I know of to restart the bot in the same window / browser, so we could remember the setting.

UPDATE: I Found if you add the n95 masks to your cart, it will ignore it next time, but it tries to make the purchase for them lol... almost bought some masks.

[Codolophier]

@technaustin If I can reproduce - either directly or via unit test - I can add some code to detect the popup and click the I'm not interested button. I'm currently working on a new version that should be capable of handling these type of interruptions. It may take me a few days (day job gets in the way sometimes :)), but I'll do what I can as soon as possible.

PS: If you - or anyone else - encounters this modal popup again, could you send the HTML for the popup/page? It would help to use that as a model for the compensation code if I cannot reproduce those results.

[technaustin]

@technaustin If I can reproduce - either directly or via unit test - I can add some code to detect the popup and click the I'm not interested button. I'm currently working on a new version that should be capable of handling these type of interruptions. It may take me a few days (day job gets in the way sometimes :)), but I'll do what I can as soon as possible.

PS: If you - or anyone else - encounters this modal popup again, could you send the HTML for the popup/page? It would help to use that as a model for the compensation code if I cannot reproduce those results.

Awesome! I’ll get it to you tomorrow morning. I can force it to happen again.

[mateuszdrab]

I fixed most of the issues in this script, took me like 4 hours to get it to checkout to the last stage since for me Newegg doesn't auto skip to review page on purchase. The steps needed to figure out how to get through the steps of the checkout were quite irritating but I presume this is some sort of Newegg protection from bots that no buttons have IDs. Still, I decided not to buy from Newegg since they're trash to me due to forcing the combos so I don't want to spend my money with them.

     try {
                await page.waitForSelector('button#Masks_addtocart.btn.btn-primary' , {timeout: 10000})
                await page.click('button.btn[data-dismiss="modal"]')
                await report('Skipped mask advert')
                }catch(err) {
                await report('when skipping mask')
                await report(err)
        }

[dexmar]

@mateuszdrab did you fork the repository or just working locally?

[mateuszdrab]

@dexmar, nope, but I can do so and push my changes back to it if it helps you

[dexmar]

@mateuszdrab that would be great! both interested in landing a 3080 (still!) and interested in learning Node. thank you!!

[mateuszdrab]

@dexmar https://github.com/mateuszdrab/NeweggBot

It's messy, I am in no way a Node dev but I just wanted it to work.

[dexmar]

wow, lightning fast! thank you! @mateuszdrab

[mazukou87]

@Codolophier and @mateuszdrab Using yours with an item thats in stock (nvme ssd) to test it, it puts it in the cart, and then just keeps adding more and never actually goes to the checkout, so i think im missing something since the same issue is happening on multiple scripts, and no one else seems to be having this issue. I threw a tube of kryonaut into the config for testing, it just kept adding them until they had no more stock available for it to add (kind of amusing to see, but definately becomes an issue when im trying to buy a 900$ cpu)

Also did what you said about the defaults @Codolophier and its still throwing the error in the script and pausing everything, not sure what else to try

[Codolophier]

@mazukou87 It looks like the problem is that Newegg changed - or at least is varying - the resulting URL for the cart when an item has been added. The code is currently checking for cart. From some tests I've done, I can see that sometimes it is cart, other times Cart. To fix this, change the line that reads:

if (page.url().includes("cart")) {
 ...
}

to

if (page.url().toLowerCase().includes("cart")) {
...
}

[mazukou87]

if (page.url().toLowerCase().includes("cart")) {

did that, still keeps adding, . I never really learned much html or java script, spent most of my time on c# and ruby, but that part looks like it would be right, but unless im missing something, which clearly i am, it seems like its noticing its on the cart but keeps going. Would a better option be to check the "x Item" line and look for a non zero integer? That at least least seems to be consistent

looking thru the else if statement, that seems to be the only thing i notice, if cart is found in url, then go to checkout,, but it seems to break at the cart check

EDIT: i just reread what you said again checked, at least for me, whether the cart is empty or has an item in it, the url is still https://secure.newegg.com/shop/cart, which means that check would always equal true, but that explain why it keeps adding them. If i am thinking about this right, it should always be true, so it should never attempt to add the item again after the first attempt.

Side note, i appreciate all your work. I hope it doesnt sound like im being unappreciative or nit picking, this is just how i talk, very blunt and to the point haha

[Codolophier]

@mazukou87

hope it doesnt sound like im being unappreciative or nit picking, this is just how i talk, very blunt and to the point haha

Don't sweat it, we're all here working together, honesty/directness is always best in programming terms :). I tested the changes using an NVME with Item # N82E16820156245. Things are working on this end. You may want to try debugging the code, stepping through from the point the item is added to the cart to see where it is dumping out - or go old school and put a bunch of console.log statements everywhere (step debugging is the easiest though... and less cleanup when done) :)

[mazukou87]

@mazukou87

hope it doesnt sound like im being unappreciative or nit picking, this is just how i talk, very blunt and to the point haha

Don't sweat it, we're all here working together, honesty/directness is always best in programming terms :). I tested the changes using an NVME with Item # N82E16820156245. Things are working on this end. You may want to try debugging the code, stepping through from the point the item is added to the cart to see where it is dumping out - or go old school and put a bunch of console.log statements everywhere (step debugging is the easiest though... and less cleanup when done) :)

how do i step into javascript. i tried doing like i do c# in VS, but it just tells me to install some firefox package...despite it already being installed. This is my first foray in to JS lol

[Codolophier]

It depends on your IDE, but typically it isn't so bad. I usually use Visual Studio Code (VSCode - free and "easy"). Here are some articles to follow for both VS Code and Visual Studio proper.

• https://code.visualstudio.com/docs/nodejs/nodejs-debugging
• https://docs.microsoft.com/en-us/visualstudio/javascript/debug-nodejs?view=vs-2019

[mazukou87]

It depends on your IDE, but typically it isn't so bad. I usually use Visual Studio Code (VSCode - free and "easy"). Here are some articles to follow for both VS Code and Visual Studio proper.

* https://code.visualstudio.com/docs/nodejs/nodejs-debugging

* https://docs.microsoft.com/en-us/visualstudio/javascript/debug-nodejs?view=vs-2019

20 hour work day makes me dumb

EDIT: I think i found WHERE it breaks.it will hit line 102, then go to 103, then back to 102, and gets stuck in a loop there, it never hits the BREAK to exit the loop and move on the the checkout page. so it just keeps executing the "await page. goto....../addtocart

problem now is, i dont know javascript so im not sure how to fix it