question: Get lots of : select: Invalid argument

[liu0fanyi]

after curl http://localhost:3000/\?_escaped_fragment_\= get lots of select: Invalid argument, and can't stop

I do these steps:

1. meteor add jazeee:spiderable-longer-timeout
2. add Meteor.isReadyForSpiderable = true; in one of my react component's componentDidMount()

I use react-router, so .... I can't find Router.onAfterAction... I want some suggestions or sample..... Thanks a lot!

[dr-dimitru]

Hello @liu0fanyi ,

Could you please post full curl response please. And all errors/logs from server/phantomjs

[liu0fanyi]

Hello @dr-dimitru I have solved my issue, there is another <meta name="fragment" content="!">, I used before for prerenderIo. After delete it, looks ok now... Thanks a lot...I will close this.

[dr-dimitru]

Please, support this project by:

• Star on GitHub
• Star on Atmosphere

[liu0fanyi]

@dr-dimitru Sorry...I get them back... I try to use meteor npm start > errorlog 2>&1 to save error log, but It will curl just ok...there are no erros in errorlog... When I try meteor npm start, then curl, I will get these errors:

: Invalid argument
W20170502-21:28:42.426(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.426(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.426(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.426(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.427(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.427(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.427(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.427(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.428(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.428(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.428(8)? (STDERR) select: Invalid argument
W20170502-21:28:42.432(8)? (STDERR) select: Invalid argument

and curl nothing, curl just block. there are too many errors, I can't get the start of log......

Is there a way to get all errors from server/phantomjs? meteor npm start, is this: meteor --settings settings-development.json on Mac os, meteor is the latest one, 1.4.4.1

Thanks very much....I can't get what's wrong here...

[dr-dimitru]

Is phantomjs itself installed into your system? To make sure run phantomjs -v in console.

[liu0fanyi]

@dr-dimitru Yes, I can get phantomjs -v, it's 2.1.1 and I get the error from start of log:

[34mW20170502-22:44:19.875(8)? (STDERR) [39m[35m spiderable: phantomjs failed: { [Error: stderr maxBuffer exceeded][39m
[34mW20170502-22:44:19.876(8)? (STDERR) [39m[35m  cmd: 'phantomjs  --load-images=no --ssl-protocol=TLSv1 --ignore-ssl-errors=true --web-security=false /Users/mac/learn/meteor/meteor_test/.meteor/local/build/programs/server/assets/packages/jazeee_spiderable-longer-timeout/lib/phantom_script.js "http://localhost:3000/documents/DctT76kiNsDf67oo5"' }

[liu0fanyi]

and I try to use phantomjs alone:

phantomjs  --load-images=no --ssl-protocol=TLSv1 --ignore-ssl-errors=true --web-security=false /Users/mac/learn/meteor/meteor_test/.meteor/local/build/programs/server/assets/packages/jazeee_spiderable-longer-timeout/lib/phantom_script.js "http://localhost:3000/follow"

It was ok, without any error......

[dr-dimitru]

@liu0fanyi ,

Looks like you bump into #49 Should be fixed as soon as requested changes will be fulfilled.

@nabilfreeman could you take a look on #49 ?

[dr-dimitru]

Please update to latest release. See bufferSize option.

[dr-dimitru]

@liu0fanyi let me know if latest release solve your issue

[dr-dimitru]

Hello @liu0fanyi ,

Have you figured out how to fix this issue tuning bufferSize option?

[liu0fanyi]

Hello @dr-dimitru sorry to replay late... I have tried to update this package and update bufferSize to 100m...in server/main.js but no help... How could I get more info about this question?enable debug in this package?

[dr-dimitru]

@nabilfreeman could you please take a look on this thread?

[perak]

The same thing happens to me. After few hours of debugging: application without accounts-password doesn't produce this error. When I add accounts-password error appears.

Looks like error is produced by phantomjs stderr (but I'm not 100 % sure).

I tried with multiple versions of phantomjs, tried downgrading meteor version... still no luck.

[dr-dimitru]

Hello @perak , @liu0fanyi ,

Thank you for the update.

Could you try to execute phantomjs directly (or fork this package and edit flags) with next flags:

phantomjs --load-images=false --ssl-protocol=any --ignore-ssl-errors=true --disk-cache=true --disk-cache-path=/data/phantomjs --cookies-file=/data/phantomjs/cookies.txt --local-storage-path=/data/phantomjs --local-to-remote-url-access=true --offline-storage-path=/data/phantomjs --web-security=false

Make sure /data/phantomjs directory exists and writable.

Let's see if this will help, perhaps it's cookies, localStorage or cache issue, as it can't be stored in memory is some special circumstances.

[liu0fanyi]

Hello @dr-dimitru I have tried phantomjs directly with up flags: phantomjs --load-images=false --ssl-protocol=any --ignore-ssl-errors=true --disk-cache=true --disk-cache-path=/data/phantomjs --cookies-file=/data/phantomjs/cookies.txt --local-storage-path=/data/phantomjs --local-to-remote-url-access=true --offline-storage-path=/data/phantomjs --web-security=false /Users/xxx/meteor_test/.meteor/local/build/programs/server/assets/packages/jazeee_spiderable-longer-timeout/lib/phantom_script.js "http://localhost:3000"

It was ok, I dont get error info

cookies.txt:

[General]
cookies="@Variant(\0\0\0\x7f\0\0\0\x16QList<QNetworkCookie>\0\0\0\0\x1\0\0\0\x6\0\0\0Qajs_user_id=null; expires=Sun, 27-May-2018 13:37:12 GMT; domain=localhost; path=/\0\0\0Rajs_group_id=null; expires=Sun, 27-May-2018 13:37:12 GMT; domain=localhost; path=/\0\0\0__ga=GA1.1.843230367.1495892233; expires=Mon, 27-May-2019 13:37:13 GMT; domain=localhost; path=/\0\0\0\x61_gid=GA1.1.1230482198.1495892233; expires=Sun, 28-May-2017 13:37:13 GMT; domain=localhost; path=/\0\0\0|ajs_anonymous_id=%22d640a636-6c70-4b8e-a0cd-2b480ad068fc%22; expires=Sun, 27-May-2018 13:37:13 GMT; domain=localhost; path=/\0\0\0G_gat=1; expires=Sat, 27-May-2017 13:38:13 GMT; domain=localhost; path=/)"

http_localhost_3000.localstorage:

�����z!11�tableItemTableItemTableCREATE TABLE ItemTable (key TEXT UNIQUE ON CONFLICT REPLACE, value BLOB NOT NULL ON CONFLICT FAIL)[!}1indexsqlite_autoindex_I����%Iajs_user_traits{}/]ajs_group_properties{}!0debugundefined
!���"Iajdebug%

[dr-dimitru]

@liu0fanyi it's a great news. I'm going to update this package to include those flags. Stay tuned

[dr-dimitru]

@perak @liu0fanyi ,

No update is needed, custom flags can be passed via METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS env.var. So, I'm going to simply update the docs.

Let me know if it will work for you

[perak]

@dr-dimitru @liu0fanyi Thanks, I'll try and let you know. I hope it works... I already spent too much time with this and I'm clise to give up and stick to prerender. BTW, the same error happens with original spiderable package and also with few other forks.

[dr-dimitru]

@perak keep me updated. This should solve it. Updated docs with explanation for every flag is on the way

[perak]

:+1: tnx!

[perak]

BTW, @dr-dimitru, I'm just curious, can you reproduce/do you have the same issue?

meteor create app
cd app
meteor add spiderable accounts-password
meteor

curl http://localhost:3000/?escaped_fragment=

and... kaboom! select: invalid argument

(didn't tried yet to set env var as you suggested in last messages, typing from mobile phone, will try as soon as I reach my computer)

[dr-dimitru]

@perak Just tried on [email protected] and [email protected] - no, no error. But this one is familiar to me, either it was back in 2015, it was caused by localStorage/cache/appcache/cookies.

The one difference phantomjs I'm using is always locally compiled from sources. See:

• https://github.com/ariya/phantomjs/issues/13175
• https://github.com/ariya/phantomjs/issues/11706

[perak]

The same OS and phantomjs but installed via brew install phantomjs. Also tried phantom 1.9.x.

And the same problem on Ubuntu 16.04 & phantomjs.

OK, now I'm with my computer...

[dr-dimitru]

Published as v1.3.3.

Thank you for contribution. Let me know how it will work for you.

[perak]

@dr-dimitru worked on hello world app, but my excitement is quickly stopped when I tried the same thing on real project - the same issue :'(

Hello world - before setting env var - error. After setting env var - works like a charm. Real project - before and after setting env var the same issue.

/data/ is writable (chmod 777 just for testing) and phantom writes something there.

BTW, I also tried to increase maxBuffer but no luck.

[perak]

Increased bufferSize to 500 MB and now I got "invalid string length" (after ~5 seconds waiting). Looks like rendered page is too big (?) but it's not. See: https://www.meteorkitchen.com there is no a lot of text/images there...

[perak]

Maybe problem is somewhere in my code... but where? All subscriptions returns proper cursor or this.ready() ... simply don't know where to look.

[dr-dimitru]

@perak I have no error requesting https://www.meteorkitchen.com with phantomjs. But it's not getting rendered, I'm getting same results as I were using curl. Going to investigate it

[perak]

Somehow Meteor is not accessible. I made simple phantom script that waits while typeof Meteor is undefined inside page.eval and that takes forever. (?). When I run kitchen locally, then same script returns errors like "undefined is not a function" (but no errors when I run in normal browser e.g. chrome). Strange, I spent almost all night trying to debug....