skrape.it
skrape.it copied to clipboard
[QUESTION] Are there cases where BrowserFetcher does not fully support CSR?
describe what you want to archive I am going to create a user guide page for my app and I need to crawl that page in my app. (I need to crawl certain urls in the app as well as notion pages) https://fundevstudio.notion.site/524eafbfa8f2414898d6d8d79f222c05?pvs=4
However, even if i use the initial BrowserFetcher
,cannot get the title of the loaded page.
Please let me know if there is any additional way I can do it.
Code Sample
fun main(args: Array<String>) {
skrape(BrowserFetcher) {
request {
url = "https://fundevstudio.notion.site/524eafbfa8f2414898d6d8d79f222c05?pvs=4"
}
response {
htmlDocument {
println("title: $titleText")
}
}
}
}
[expect] title: 인사이트 플로우 가이드 [but] title: Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.
If it is not possible, waitUntill
property value similar to playwright, puppeteer: load
, networkidle
, documentLoaded
Please consider providing options.