graspologic icon indicating copy to clipboard operation
graspologic copied to clipboard

Considering removing gensim as a dependency

Open bdpedigo opened this issue 1 year ago • 11 comments

Considering completely removing gensim (and therefore, node2vec) as a dependency in graspologic

They constantly have dependency issues because the package is not heavily maintained (currently releasing on PyPI about once per year). For instance, right now builds are failing due to an issue with Scipy. https://github.com/piskvorky/gensim/issues/3525

If anyone really wants to keep node2vec around, then I'd like to discuss how to get away from these dependency conflict issues or to have someone help with maintenance when they arise

bdpedigo avatar Oct 08 '24 22:10 bdpedigo

cc @daxpryce @darthtrevino: I know you're likely busy with other things, but if you two know of anyone else at MSFT using graspologic that might care, could you please cc them as well? Thank you in advance

bdpedigo avatar Oct 08 '24 22:10 bdpedigo

I believe that @darthtrevino and company still use it, but I would agree it has become such a millstone around the neck re: library versions, python version limitations, etc, that you're better off removing it until a... less challenging version can be built or found and used. What do you think @darthtrevino ?

daxpryce avatar Oct 08 '24 22:10 daxpryce

We mostly use this library for Leiden in GraphRAG, but we do have node2vec as an optional feature. We've been discussing moving to GEE. Is there a way to support n2v without gensim? Maybe an alternative implementation?

darthtrevino avatar Oct 08 '24 23:10 darthtrevino

this one popped up for me with 1.2k stars https://github.com/eliorc/node2vec I have also definitely seen other libraries

bdpedigo avatar Oct 08 '24 23:10 bdpedigo

If possible, we'd prefer to have the n2v embedding operation supported, but we're not married to gensim per se

darthtrevino avatar Oct 08 '24 23:10 darthtrevino

I would assert the main reason we added node2vec in topologic and then graspologic is specifically so we could swap out implementations but provide a consistent API. I think the effort to replace it rather than remove it is good, though I also could see removing it in the next couple versions until you can get the next one in place - if only so development can continue without dependency requirement conflicts

daxpryce avatar Oct 08 '24 23:10 daxpryce

looks like things are working now https://github.com/graspologic-org/graspologic/actions/runs/11259180298

i somehow missed that they had published a fix on PyPI for the current issue.

going to leave this issue open though, i think this is a general problem that we have a pretty wide scope of dependencies

bdpedigo avatar Oct 09 '24 16:10 bdpedigo

gensim also broken on python 3.13 with no fix in sight https://github.com/piskvorky/gensim/issues/3601

jcaplan avatar Jul 02 '25 16:07 jcaplan

that sounds great to me

bdpedigo avatar Jul 07 '25 16:07 bdpedigo

gensim also broken on python 3.13 with no fix in sight https://github.com/piskvorky/gensim/issues/3601

not anymore, from 3.4.0 they support Python 3.13

taranarmo avatar Nov 19 '25 08:11 taranarmo