Error in add_edges: long vectors not supported yet

[tardigradus]

Running igraph with R 3.3.3 and passing a 13 GB TransitionLayer object to 'shortestPath' from the package 'gdistance' I get the following error:

Error in add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight) : 
   long vectors not supported yet: ../../src/include/Rinlinedfuns.h:138

Is this issue with igraph or R core?

[gaborcsardi]

R core for sure, but probably both.

[tardigradus]

OK, so whom do I need to provide with what information? Or is it more a question of requesting developers on both sides to address lifting a known limitation?

[izahn]

@tardigradus Did you see https://github.com/igraph/rigraph/issues/82 ?

@gaborcsardi I'm pretty sure fixing this in igraph is doable today. What specifically are you waiting for R core to do or provide?

[gaborcsardi]

@izahn I am not waiting for anything.

But that error message is coming from R core, not from igraph.

[ltierney]

The error message is correct: It's coming form calling length() on a long vector and that is an error. If you have C code you want to work with long vectors you need to rewrite it to use R_xlen_t indices and xlength(). Without a stack trace it's impossible to tell whether the call to length() is in core C code or package C code.

[tardigradus]

@izahn Yes, I saw #82, but that refers to R 3.2.0 and I don't know to what degree this is still an issue with subsequent versions of R core.

[tardigradus]

@ltierney Here's the stacktrace:

> traceback()
12: .Call(C_R_igraph_add_edges, graph, as.igraph.vs(graph, edges) - 
        1)
11: add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight)
10: graph.adjacency.sparse(adjmatrix, mode = mode, weighted = weighted, 
        diag = diag)
9: graph.adjacency(y, mode = mode, weighted = TRUE)
8: .shortestPath(x, origin, goal, output)
7: .local(x, origin, goal, ...)
6: shortestPath(conduct, origin = sites@coords[a, ], goal = sites@coords[b, 
       ], output = "SpatialLines")
5: shortestPath(conduct, origin = sites@coords[a, ], goal = sites@coords[b, 
       ], output = "SpatialLines") at calculate_shortest_path.r#68
4: eval(expr, envir, enclos)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("./calculate_shortest_path.r")

[ltierney]

I actually meant a C level stack trace as you would get from gdb or lldb. That would tell you more precisely where to look in the C code. But this already tells you the issue in the package C code for C_R_igraph_add_edges or something called from there.

On Thu, 15 Feb 2018, Tardi Gradus wrote:

@ltierney Here's the stacktrace:

traceback() 12: .Call(C_R_igraph_add_edges, graph, as.igraph.vs(graph, edges) - 1) 11: add_edges(res, edges = t(as.matrix(el[, 1:2])), attr = weight) 10: graph.adjacency.sparse(adjmatrix, mode = mode, weighted = weighted, diag = diag) 9: graph.adjacency(y, mode = mode, weighted = TRUE) 8: .shortestPath(x, origin, goal, output) 7: .local(x, origin, goal, ...) 6: shortestPath(conduct, origin = sites@coords[a, ], goal = sites@coords[b, ], output = "SpatialLines") 5: shortestPath(conduct, origin = sites@coords[a, ], goal = sites@coords[b, ], output = "SpatialLines") at calculate_shortest_path.r#68 4: eval(expr, envir, enclos) 3: eval(ei, envir) 2: withVisible(eval(ei, envir)) 1: source("./calculate_shortest_path.r")

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.[ADvaVHOpYN30SpaTg3Qek4ZVmBcmyKmPks5tVDzRgaJpZM4SGfXp.gif]

-- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: [email protected] Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu

[pfgherardini]

I stumbled upon the same issue. In my case, the igraph entry point that is calling the error is graph.adjacency. Is there a workaround?

[tardigradus]

I didn't persue this any further, as in my case it was possible to partition the data such that the error didn't occur for the subsets.

[szhorvat]

Large graphs / vectors will be supported in the soon-to-be released 2.0, to the extent that R allows.