Performance issue with add_interactions_from method

[yanirmr]

See my question on stackoverflow about the efficiency of converting a list of static graphs to a dynamic graph. In light of the size of the data, I believe the current duration of the process is unexplained.

[GiulioRossetti]

Hi, just a few considerations:

• it is completely normal that creating a snapshot list without the merging phase is less costly than merging them in a DynGraph: this is prevalently due to the fact that temporal information for replicated edges have to be compressed as edge's attributes;

• the random graphs you are generating are dense (50% of the edges are present, something unrealistic in most real contexts) and this requires constant edges' attributes updates. By reducing the number of edges you'll be able to scale up to bigger networks. Just as an example, consider that for the ER model you are simulating it suffices a p=1/N (where N is the number of nodes in the graph) to guarantee a supercritical regime (i.e., a single connected component);

• dynetx is built extending networkx that is not particularly scalable (both in terms of memory consumption and execution times): when dealing with dense, heavily edge-attributed, graphs such limitations are more evident than ever;

• the way you are building the dynamic graph is likely the most time-consuming one available. You are adding interactions among each pair of nodes without leveraging the knowledge of their effective duration. If the interaction (u,v) takes place k times from t to t+k you can insert such edge just once specifying its vanishing time, thus reducing the graph manipulation operations.

Of course, everything can be further optimized: if you like to work on optimizing graph creation you have all my support!