apify
Replace demo-webstore.apify.org with Shopify demo store
After a discussion with @mnmkng a decision has been made that we want our courses to use the https://warehouse-theme-metal.myshopify.com/ sample shop by Shopify, because it's a sweet spot between
- Stable but sterile learning environment
- Unstable but real-world learning environment
The sample shop uses most of the technologies we need for demonsration of scraping techniques useful in the real world scenarios, but won't change from week to week. Also we do not have to maintain it.
Possible risk is that the template disappears or gets replaced one day, and we'll have to rework our courses, but the risk is low and can be easily mitigated.
As per this decision we should phase out https://demo-webstore.apify.org/ as our own demo website.
I wanted to kill fakestore in https://github.com/apify/apify-docs/pull/1104, but now I discovered that the How to scrape from dynamic pages tutorial heavily uses specific features of the sample website. It looks like there are images which are dynamically loaded by JavaScript, lazily.
Both those aspects (dynamic & lazy) are used in the tutorial as good reasons why to take PuppeteerCrawler for a walk, and I think that's how it should be. But there isn't an alternative for this in the sample Shopify store.
This reveals a significant flaw our new approach has. If we want to teach something that isn't in the sample Shopify store, after killing the fakestore there won't be a similar playground where we could demo the situation for a stable and repeatable learning path.
Without fakestore, I can think of only two solutions:
- Use the sample Shopify store, and sprinkle the text with "what if" sentences. There won't be the actual situation and it'd gonna be more about "imagining" the right situation and solving a theoretical problem than a going through a hands-on lesson.
- Find a website which does it, somewhere out in the wild. First, I don't really know how to find it other than randomly browse the web. Second, they can redesign the website next week, flushing the tutorial to the toilet.
Albeit far from perfect, if we do want to get rid of the fakestore, I consider the first option better than the second. Ideas?
Or perhaps the tutorial is on a topic which we could consider as covered by the whole P&P course, and we can… remove it? (redirect it, of course…)
I would bet that when I last checked (years ago 😄) there was some random thing on the Shopify page that was lazy loaded. Like reviews or something. I'll double check.
But yeah, if I can't find anything, 1 feels certainly better. Or we could do 1 with 2 as exercises.
Not sure about the P&P. Can't remember how similar they are or not.
Ah, you're right this isn't P&P alone, but PupeteerCrawler from Crawlee. That's different. Disregard my last comment.