RMONGO with streaming data

[louisdv1]

Hi , I am pulling data out of an API with R and it seems that the insert to mongodb keeps failing? This is the message I am having: Error in .jcall(rmongo.object@javaMongo, "S", "dbInsertDocument", collection, : com.mongodb.util.JSONParseException:

[tc]

What query command are you running? what does your mongo collection look like?

[louisdv1]

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url="
https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo =
"cacert.pem")},error=function(e){})

if (length(result)>0)  dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" [email protected] wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17290176 .

[tc]

What is z in dbInsertDocument(mongo, "stocktwits7", z)? It looks like an unreferenced variable.

[tc]

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 [email protected] wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" [email protected] wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17296054 .

Tommy Chheng

[louisdv1]

Stocktwits is just the name of tge collection and z is a json response fetched by rcurl On 1 May 2013 20:39, "Tommy Chheng" [email protected] wrote:

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 [email protected] wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" [email protected] wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17296054> .

Tommy Chheng

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17301621 .

[tc]

Try printing out the z variable to see if it's malformed. That could be the reason why it's not inserting correctly.

On Wed, May 1, 2013 at 12:42 PM, louisdv1 [email protected] wrote:

Stocktwits is just the name of tge collection and z is a json response fetched by rcurl On 1 May 2013 20:39, "Tommy Chheng" [email protected] wrote:

You can also use rjson to extract the messages array out of the url response.

On Wed, May 1, 2013 at 11:01 AM, louisdv1 [email protected] wrote:

Hi Tommy and thank you for the fast response. I am pulling data out of stocktwits and pushing it to mongodb. Sometimes I have a parsing problem but the most problematic is that the json feed representing several messages seem to be bundled up together so that mongo only recognizes one message per call . Here is my code

for (i in 1:5) {

result=tryCatch({getURL(url=" https://api.stocktwits.com/api/2/streams/symbol/msf.json",cainfo = "cacert.pem")},error=function(e){})

if (length(result)>0) dbInsertDocument(mongo, "stocktwits7", z)

Sys.sleep(10)

}

Thank you very much On 1 May 2013 17:26, "Tommy Chheng" [email protected] wrote:

What query command are you running? what does your mongo collection look like?

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17290176> .

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17296054> .

Tommy Chheng

— Reply to this email directly or view it on GitHub< https://github.com/tc/RMongo/issues/18#issuecomment-17301621> .

— Reply to this email directly or view it on GitHubhttps://github.com/tc/RMongo/issues/18#issuecomment-17301832 .

Tommy Chheng