talk
talk copied to clipboard
Scraper failing, unsure why
I am trying to use the scraper to generate titles for stories, here is the error it gives me:
{
"errors": [
{
"message": "Could not scrape the URL http://localhost:8080/my/test/url/.",
"locations": [
{
"line": 1,
"column": 3
}
],
"path": [
"debugScrapeStoryMetadata"
],
"extensions": {
"code": "SCRAPE_FAILED",
"id": "1d8ed340-dfef-11ec-8f87-2500d726475a",
"traceID": "1d8e0ff0-dfef-11ec-8f87-2500d726475a",
"type": "INVALID_REQUEST_ERROR",
"message": "Could not scrape the URL http://localhost:8080/my/test/url/."
}
}
],
"data": {
"debugScrapeStoryMetadata": null
}
}
As for metadata, I have tried various formats without success:
<title>My Title</title>
<meta property="og:title" content="My Title">
<meta property="title" content="My Title">
Rescraping the metadata from the admin UI fails as well.
Any ideas as to what might be causing this?
Why is the '.' behind the last slash in the url? Try to do a 'curl' to the url to see if that works
Yeah, that scared me at first too, but I think the period is just because it's part of the message. It's saying "could not scrap the url {the url}." I tried going to the url anyways, and it just loads the webpage normally with a status code of 200.