Froren
Intermittently fails with 403 with localhost:3000 and with CORS enabled
I am using localhost with chrome and with CORS enabled (same thing with firefox) The API works for a while then it starts returning 403. Switch to Firefox, and it works for a while then it fails. All in a sudden, it works again for a while and then it returns the error. I have tried to clear the browser cache, and it does not help. I have tried restarting the localhost and it does not help. Nothing was changed in the logic, and it just simply stops working.
Please help.
Source
static getListings(options, callback) { realtor.post(options) .then(data => { debugger callback(data) }) .catch(err => { debugger console.log(err) }); }
Error
403 - "<html style="height:100%">
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"><meta name="format-detection" content="telephone=no"><meta name="viewport" content="initial-scale=1.0"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><script src="/1d1b2b9ff80a163b7c7e711b7da13c06" async><body style="margin:0px;height:100%"><iframe id="main-iframe" src="/_Incapsula_Resource?CWUDNSAI=22&xinfo=4-81430764-0%20PNNN%20RT%281593375407092%200%29%20q%280%20-1%20-1%20-1%29%20r%281%20-1%29%20B15%2814%2c0%2c0%29%20U6&incident_id=305000120123612946-279287541296857860&edet=15&cinfo=0e000000c60b&rpinfo=0" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 305000120123612946-279287541296857860"" name: "StatusCodeError
Request
Request URL: https://api2.realtor.ca/Listing.svc/PropertySearch_Post Request Method: POST Status Code: 403 Remote Address: 45.223.28.18:443 Referrer Policy: no-referrer-when-downgrade
Request Header
:authority: api2.realtor.ca :method: POST :path: /Listing.svc/PropertySearch_Post :scheme: https accept: application/json accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9 content-length: 207 content-type: application/x-www-form-urlencoded origin: http://localhost:3000 referer: http://localhost:3000/ sec-fetch-dest: empty sec-fetch-mode: cors sec-fetch-site: cross-site user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36
Form Data
CultureId: 1 ApplicationId: 1 PropertySearchTypeId: 1 LongitudeMin: -79.37638906619264 LongitudeMax: -79.36106828829958 LatitudeMin: 43.90685458332996 LatitudeMax: 43.92027197699288 PriceMin: 100000 PriceMax: 4100000
Response Header
Access-Control-Allow-Credentials: false Access-Control-Allow-Headers: status,content-type,cache-control,content-length,x-iinfo,set-cookie,set-cookie Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH Access-Control-Allow-Origin: * Access-Control-Expose-Headers: status, content-type, cache-control, content-length, x-iinfo, set-cookie, set-cookie cache-control: no-cache content-length: 808 content-type: text/html Content-Type: application/json; charset=utf-8 set-cookie: visid_incap_2271082=YZAjHAmERTe6k/DamZEDHM8D+V4AAAAAQUIPAAAAAABdt+LR3LW4hmvTWwRDjSUb; expires=Mon, 28 Jun 2021 13:06:34 GMT; HttpOnly; path=/; Domain=.realtor.ca set-cookie: incap_ses_305_2271082=i3HeKHBMq0hu35xI/pM7BM8D+V4AAAAAMhmugRVDKrXlPDMotf8NhA==; path=/; Domain=.realtor.ca status: 403 X-Content-Type-Options: nosniff x-iinfo: 9-81635212-0 PNNN RT(1593377743104 0) q(0 -1 -1 -1) r(0 -1) B15(14,0,0) U6
Response
hey @j8ckch8n, thanks for the descriptive issue.
That's certainly very strange. I haven't run into this myself but it sounds like something on the realtor.ca server is blocking the requests.
@GoodForOneFare seems to have run into this issue as well but I'm unsure if they found a way to get past it.
Error --StatusCodeError: 403 - "<html style="height:100%">
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"><meta name="format-detection" content="telephone=no"><meta name="viewport" content="initial-scale=1.0"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><script src="/1d1b2b9ff80a163b7c7e711b7da13c06" async><body style="margin:0px;height:100%"><iframe id="main-iframe" src="/_Incapsula_Resource?CWUDNSAI=22&xinfo=4-34061043-0%20NNNY%20RT%281593932602970%2023%29%20q%280%20-1%20-1%20-1%29%20r%281%20-1%29%20B15%2814%2c0%2c0%29%20U6&incident_id=120000030097193786-194081802518266052&edet=15&cinfo=0e000000be09&rpinfo=0" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 120000030097193786-194081802518266052"
Changing the url from
const API_URL = 'https://api2.realtor.ca/Listing.svc/PropertySearch_Post';
to:
const API_URL = 'https://api.realtor.ca/Listing.svc/PropertySearch_Post';
Resolved the issue for me. Only time will tell if it continues to work.
Just wanted to say thanks for the solution @Kris523 ! I was running into the same issue just now and that seems to work for me (for now, at least). I wonder if api2 has some sort of rate limiting built in.
Also, thanks for building this package! I can't seem to find any documentation or info on the API—did you happen to know of any when building this @Froren ? Many thanks in advance.
@younglee327 Glad you like it! Nope, I did not. A bit of the origin is explained on https://github.com/Froren/realtorca/issues/4
Thanks for the quick reply :) That post is a wild ride, wow.
I seem to only be getting one image result per listing under Photo whenever I search. Do you know if that's normal or if I'm setting a parameter incorrectly? Is there a way to ensure all photo links are returned? It seems like the photo filenames are numbered numerically so I could generate the rest using the first one, but I have no way of telling the total number of photos.
And so, I was wondering if there were a way to either 1) get all photo links for a listing or 2) get the total number of photos in a listing. Thanks again!
@younglee327 That's normal, the API is only rendering what you get from the search view, which only needs to show one photo per listing.
I think you'd need to find a way to dig into an individual listing to get all the photos listed. I'm not yet aware of a way to do this, but let me know if you find something.
hey quick note, @iwasrobbed found a great way to get the listing information: https://github.com/Froren/realtorca/issues/19
Hey @Froren I just ran into this same issue, but @Kris523's solution worked for me. Can I make a pull request to fix the issue since it seems to be a common problem? I also doubt it has to do with rate limits since I only made 9 or 10 requests before it started failing for me EDIT: After some time using 'api' fails for me too.
