Handling dependencies

[sinasamavati]

At the moment, Mad downloads dependencies and puts them in ~/.otp/deps and then makes symlink in deps directory, e.g. lkv/deps/leptus -> ~/.otp/deps/leptus-master. Yes, a dependency's name is handled like: [name]-[branch/tag/hash].

I have only two ideas about handling dependencies:

• making it work like Bundler, I mean something like an exec parameter for running other escripts, so Mad should update code path and evaluate the escript in the running VM, but I'm not sure if it is possible or not. another way to do this is to update ERL_LIBS, then evaluate the escript and after that revert ERL_LIBS to its original state, well, this way might make Mad so slow.
• since the deps directory is popular, keeping Mad the way it handles dependencies may be better, namely, making symlink as I explained above.

any idea?

[hamidreza-s]

I think symlink solves the problem in a clean and straightforward manner, and also it's good from performance perspective. However, the Bundler architecture is so mature and flexible. I think It's worth implementing if you have enough time.

[sinasamavati]

I'm positive with making symlink since almost everything uses deps directory. I'm also positive with the Bundler architecture, but it might make Mad so slow and incompatible with other things.

[hamidreza-s]

In the case of Bundler architecture, I think the trade-off between "low performance" and "more features" is reasonable. But incompatibility would become a serious issue over time. So, count another positive vote for Symlink :+1:

[sinasamavati]

Totally right. there are two things that really should be considered, compatibility and features.

[ericbmerritt]

@s1n4 Tristen pointed this issue out to me and suggested I comment. I think thats a good idea :). We have done a ton of work to get 'bundler' like features into https://github.com/erlware/relx. Given dependency constraints and the actual dependencies being on the box it will does a very good job of assembling those dependencies into a release and it has a 'devmode' that is basically the bundler like functionality you talk about above. We could pretty trivially support ~/.otp/deps as a default source. That way, In the best case you could delegate that to the relx command, in the worst case use relx as a library to accomplish the same thing (its designed to be used that way as well).

It looks like mad fills a good niche in getting dependencies on to the box (there could probably be some decent sharing there in relx's dependency solver). That is something that relx doesn't do, and, by design, never will. Relx fills the assembly niche and it seems like a shame to duplicate that work. Better we work together to add any features that are missing and improve any integration issues that arise.

[sinasamavati]

Thanks to you and Tristen. Actually, Mad will never do what a release creation tool does, and I was thinking of making Mad compatible with Relx. It's going to be good that we work together to do an integration and/or fix any incompatibility that there might be. But I recently noticed that I'm getting and handling dependencies in a wrong way that made more questions in my head:

1. it really should be compatible with rebar configuration? if so, then making symlink is wrong since some specify a branch and no locking is done.
2. if making symlink is wrong, what about having something like a cache layer (which should get a copy of dependencies that are already been downloaded)? if it's okay, the machine X is gonna have N copies of Y.
3. if it shouldn't be compatible with rebar configuration, do I have to invent a configuration form and/or make a builder system that should fetch repositories, compile them then make them ready for the user as a tarball?

I think a better solution for getting/handling dependencies is really needed. Also I'll be thankful for any idea. Thanks.