ergm
ergm copied to clipboard
statnet
Parallelise MPLE?
In principle, there is nothing stopping us from splitting the dyads up into batches, calling the C code on worker nodes, then concatenating the results.
@chad-klumb , @martinamorris , is this something that you have a use case for?:
I think it would be handy to able to do that, and will be able to point to a use case soon (the paper is presumably wending its way to arXiv, assuming that my student Peng Huang submitted it yesterday as planned). Short version: we implemented MPLE for count ERGMs (which is not AFAICT present in current code), and it turns out that for graphs with high edge variance, the MPLE can be very good. But also, when such graphs are at all large, you really need MPLE for computational reasons, and there is a huge speed gain from parallel processing. (Edge subsampling, as well, of course.)
Those speed gains are also there in the binary case, of course, but somewhat less profound because you don't have a non-trivial normalizing factor at the edge level (which you do for count ERGMs). And the MPLE itself is less compelling in the binary case, for the usual reasons. As we get into models of networks where the per-edge entropy is higher, I suspect we will find the MPLE to become a stronger contender. (It also has some assets in the multiple network case.)
