scrapy-playwright icon indicating copy to clipboard operation
scrapy-playwright copied to clipboard

Windows WSL scrapy-playwright chromium headful mode not working

Open gelodefaultbrain opened this issue 1 year ago • 1 comments

Hi!

Apologies if I am not writing this correctly. But I just wanted to raise this issue. I am currently running a scrapy spider on my Windows machine, particularly in WSL. When I run the program I am getting this. Please see image attached.

image

As you can see, it's just like that. We can see the browser (chromium) exists by looking at the sides of it but at the middle it's completely blank. Like you can see through it. Weird thing is the spider runs successfully and I am able to scrape the data I want accordingly. May I ask what could be the solution for this? Do we need to install something that is a dependency requirement?

Additional Information: Just wanted to add that I tried switching to "firefox" and it worked well. I can clearly see what is happening. I tried it with "webkit" as well but it's the opposite of "chromium" , the middle part is completely white. But using all the browsers I am able to scrape the data gracefully.

My custom settings:

    custom_settings = {
        "PLAYWRIGHT_LAUNCH_OPTIONS": {
            "headless": False,
            "slow_mo": 9000,
        },
    }

Thank you!

gelodefaultbrain avatar Jun 27 '24 12:06 gelodefaultbrain

Found https://github.com/microsoft/playwright/issues/21813#issuecomment-1476754006 via https://stackoverflow.com/a/78320155. In short, looks like you could solve it by disabling GPU support in WSL.

elacuesta avatar Jun 27 '24 22:06 elacuesta

Found microsoft/playwright#21813 (comment) via https://stackoverflow.com/a/78320155. In short, looks like you could solve it by disabling GPU support in WSL.

The solution there was what made it working for me after 12h of headwalling

krukonshedul avatar Jul 03 '24 00:07 krukonshedul

Hi sorry for the late reply YES!! disabling GPU support made it work. It can be done via GUI by just disabling this by the way.

DeepinScreenshot_select-area_20240703092745

Thanks all for the help! Closing this now as solved!

gelodefaultbrain avatar Jul 03 '24 01:07 gelodefaultbrain