ndtv speed issues

[skyebend]

(transfered from statnet trac issue #492)

I've got code to reconstruct the wals movie, but seems like it would take two days to finish the ndtv layout and rendering. Seems like there a number of issues and places where things could be speeded up.

• would be great to have a non-java ( C? ) implementation of the SMACOV optimization
• OpenJDK seems alot slower than OracleJDK (needs to test this) when using MDSJ
• the plot storage mechanism from the animation library is probably eating up memory and putting an upper limit on the duration (number of frames/slices) of movies that can be rendered) investigate saving out the temp images earlier in the process? i.e. render directly to image files and call ffmpeg without the replay-in-plot-window functionality

comment:1 Changed 3 years ago by skyebend

could only render 50 timesteps of the 1k node network for Martina's presentation. Any more would run out of memory. need to implement saving out the static images for each frame during render, instead of caching plots using animation lib.

comment:2 Changed 3 years ago by skyebend

Proposal to deal with temp images problem:

I'm planning to add a render.cache=c('plot.list','none') argument to render.animation(). The default plot.list option would maintain current functionality (caching all plots rendered to the current device onto a list) 'none' would do nothing with the plots. This should make it so that a call of

saveMovie(render.animation(nd,render.cache='none'),...)

would directly save the movie to the image files for each frame to disk without clogging up memory, but ani.replay would not work because there would no in-memory cache of the plot frames.

comment:3 Changed 3 years ago by skyebend

implemented render.cache option above in [12241/statnet_commons]. Able to render the 100 timesteps of the 1Kn movie dramatically faster without memory problems.

comment:5 Changed 2 years ago by skyebend

seems like if I detect if it will be doing anything with the weights, could call network.extract instead of network collapse, save some time there