web-poet
web-poet copied to clipboard
scrapinghub
Web scraping Page Objects core library
```py class MyPage(ItemPage): @field def name(self) -> str: return "123" page = MyPage() page.name # type is Any, not str ```
Stemming off from the discussion in https://github.com/scrapinghub/web-poet/pull/84#discussion_r995643314.
Adding and fixing some use cases before **Approach 1** from https://github.com/scrapinghub/web-poet/issues/77 begins development.
For this discussion, we'll focus on the subclasses of `web_poet.WebPage` which requires the `web_poet.HttpResponse` as a dependency. # Problem There are some scenarios where we might need to perform some...
https://stackoverflow.com/a/76654866/939364 Seen in a production page object.
There's a way to specify where to create the tests https://scrapy-poet.readthedocs.io/en/stable/testing.html#configuring-the-test-location, but actual paths may look like this: `some_project/tests/fixtures/some_project.page_objects.homedepot.com.products.HomedepotComProductPage/test-1` which is way too long and it would be nice to...
Right now tests created for page objects aren't [discoverable](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#conventions-for-python-test-discovery) by `pytest`, therefore IDEs (e.g. VS Code) aren't aware of tests presence, though there might be other unit tests discoverable by...
I see three useful ways to run tests: by passing the `fixtures` directory / not passing anything, by passing the `fixtures/` directory and by passing the `fixtures//` directory. Out of...
There was a previous discussion about this before in one of the PRs. I'm re-opening this for tracking since this part of `w3lib.util.to_unicode` breaks: https://github.com/scrapy/w3lib/blob/master/w3lib/util.py#L46-L49 In particular, doing something like:...