ecosystem icon indicating copy to clipboard operation
ecosystem copied to clipboard

Modules Inside Distributions Are Next to Impossible to Find

Open zoffixznet opened this issue 9 years ago • 24 comments

Some Perl 6 distributions listed on modules.perl6.org contain multiple modules, but those are currently unsearchable.

For example, say you want to test the output of your program. You search for keyword "test" and browse the available offerings. You find nothing and you proceed to write Test::Output, just so a couple of days later someone points out IO::Capture::Simple distribution that actually includes Test::IO::Capture module, but alas, it doesn't show up in search results.

zoffixznet avatar Jan 14 '16 15:01 zoffixznet

encouraging the use of appropriate tags would help with this. But yes, the "provides" should be searchable probably.

jonathanstowe avatar Jan 14 '16 16:01 jonathanstowe

Neither does panda display installing modules defined in "provides" so its an easy vector for Trojans (I should check zef behaviour too)

I patched the precomp stuff to display this information if a debugging env var is set since it helps with the common case of a typo in provides making a module "disappear" but it's not a good solution.

"provides" needs rethinking as does the way panda works.

On 14 January 2016 at 16:03, Jonathan Stowe [email protected] wrote:

encouraging the use of appropriate tags would help with this. But yes, the "provides" should be searchable probably.

— Reply to this email directly or view it on GitHub https://github.com/perl6/user-experience/issues/1#issuecomment-171683375 .

4096R/EA75174B Steve Mynott [email protected]

stmuk avatar Jan 14 '16 17:01 stmuk

Search still does not return that particular module. Moving things to CPAN will probably help.

JJ avatar Apr 12 '18 05:04 JJ

Search still does not return that particular module. Moving things to CPAN will probably help.

I was under the impression that CPAN was optional and that some authors prefer to release through Github. Something to be documented, I think.

nxadm avatar Apr 12 '18 06:04 nxadm

p6c is "soon to be deprecated" https://docs.perl6.org/language/modules.html#Distributing_Modules

JJ avatar Apr 12 '18 06:04 JJ

Is this accurate? Where would it leave people who don't want to use CPAN or create a PAUSE id? Or just prefer github? Removing options isn't very Perl!

stmuk avatar Apr 12 '18 06:04 stmuk

I am not against a canonical way to find (!) and publish modules. However, I wonder how CPAN will deal with modules with the same and different author, something Perl 6 supports. I don't know if people actively oppose creating a PAUSE account. Because the importance of an ecosystem for a "new" language, It's an issue that deserved careful considiration so a decision can be implemented to last.

nxadm avatar Apr 12 '18 06:04 nxadm

@stmuk well, it's quite clear in the documentation and in the latest submissions to this ecosystem. Clear that it's going to happen, not that clear when. That's why I have inserted perl6/ecosystem#393 @nxadm I guess that feature will be forgotten for the sake of convenience, CPANTS testing, and all the goodies that go with CPAN.

JJ avatar Apr 12 '18 06:04 JJ

We really use CPAN just as storage space. The CPAN indexer ignores Perl6 directories, so CPAN doesn't care if the same distribution or module name appears in multiple author directories. That's our ecosystem code's job to deal with and it does already do so quite well. I would also not be surprised if PAUSE would gain support for OAuth, Auth0 or the like, so people with a GitHub account can just upload to CPAN without registering.

niner avatar Apr 12 '18 07:04 niner

@nxadm If someone with existing modules on the ecosystem doesn't have a PAUSE id and can't be bothered to create one do those modules just disappear from being usable by the Perl 6 community.

Most Perl modules exist in the ecosystem and not on CPAN.

@JJ I've seen no evidence that the documentation is correct on this point. Do you have any links to discussion on any IRC channel about this?

"Forgetting" basic perl 6 features like allowing identical module names shouldn't happen without further discussion.

stmuk avatar Apr 12 '18 07:04 stmuk

@niner I really have no idea. There's no issue saying the documentation is not correct. I'll try and see who inserted it and why.

JJ avatar Apr 12 '18 07:04 JJ

Here you go Some recent posting here, but that's it...

JJ avatar Apr 12 '18 07:04 JJ

@niner That makes sense (allowing people with existing Github accounts to upload perl6 modules to CPAN).

However this does suggest progress towards this is blocked on CPAN adding on adding this feature. I don't see the current ecosystem going away soon and talk of its depreciation seems premature.

stmuk avatar Apr 12 '18 07:04 stmuk

Pulling from the thread... this is the PR which points to this post by Zoffix

JJ avatar Apr 12 '18 07:04 JJ

It does not say anywhere on that post that p6c is going to be obsolete. It just says that if you upload them to CPAN, please remove them from the ecosystem listing. That's all.

JJ avatar Apr 12 '18 07:04 JJ

Zoffix says "encouraged to remove the p6c version" not that the ecosystem is going away. I think the docs are wrong on this point and talk of "deprecation" is too strong.

stmuk avatar Apr 12 '18 07:04 stmuk

Pulling from the thread... this is the PR which points to this post by Zoffix

I just missed that PR claimed deprecation when I merged it; fixed. I'm not aware of any plans or good reasons to deprecate p6c and I personally prefer it over CPAN.

zoffixznet avatar Apr 12 '18 07:04 zoffixznet

2018-04-12 9:18 GMT+02:00 Steve Mynott [email protected]:

Zoffix says "encouraged to remove the p6c version" not that the ecosystem is going away. I think the docs are wrong on this point and talk of "deprecation" is too strong.

Or directly wrong... I was asking for a clarification when Zoffix's commit appeared on the log. So that's cleared. Thanks everyone!

JJ avatar Apr 12 '18 07:04 JJ

I guess that feature will be forgotten for the sake of convenience, CPANTS testing, and all the goodies that go with CPAN

If someone with existing modules on the ecosystem doesn't have a PAUSE id and can't be bothered to create one do those modules just disappear from being usable by the Perl 6 community

We really use CPAN just as storage space. The CPAN indexer ignores Perl6 directories, so CPAN doesn't care if the same distribution or module name appears in multiple author directories.

I'm not aware of any plans or good reasons to deprecate p6c and I personally prefer it over CPAN.

Rereading the thread, I have this questions/remark:

  • if CPAN is just a backend, what is the added value? The thing I like about CPAN is strict versioning, something that a master branch on github does not have (I am looking at you, Go). Tags on github would help.
  • Multi author modules of the same name, certainly if already there, is not a feature to throw away. E.g., in Go is it very handy to just fork a lib while you're waiting for a PR to be merged and reference your fork in your code. Again, this would only be an issue if CPAN was more than just a storage (indexing, testing, versioning).
  • Some authors prefer not to use CPAN. If CPAN doesn't offer much, I woudn't force people to migrate. If it does and it offer us unique opportunities to have a great ecosystem (versioning, testing) like the one of Perl 5, it should be considered.

nxadm avatar Apr 12 '18 07:04 nxadm

  • Multi author modules of the same name, certainly if already there, is not a feature to throw away. E.g., in Go is it very handy to just fork a lib while you're waiting for a PR to be merged and reference your fork in your code. Again, this would only be an issue if CPAN was more than just a storage (indexing, testing, versioning).
  • Some authors prefer not to use CPAN. If CPAN doesn't offer much, I woudn't force people to migrate. If it does and it offer us unique opportunities to have a great ecosystem (versioning, testing) like the one of Perl 5, it should be considered.

I don't think this is an issue any more. It's going to be just an alternative. The only thing asked from authors is that they choose one of the two places. Personally, I'm opting for CPAN.

JJ avatar Apr 12 '18 07:04 JJ

Yes, I realise that. But that also means we aren't solving anything (or just thinking of improvements). On the top of my head:

  • Github versioning is weak, I am unclear about CPAN. Very often when running zef, the installation breaks because a random dependency. Versioning would make everything more stable.
  • CPAN has a great testing culture, that predates Travis, Appveyor and the like. Do we strongly suggest CI? Force? Look at CPANTS?

nxadm avatar Apr 12 '18 08:04 nxadm

Some more general discussion (unrelated to the now resolved issue)

@nxadm The main advantage of CPAN is getting a mirrored, global file system for free so we are less dependent on Github (although note its possible to use gitlab with the ecosystem too).

I share your concerns about versioning but I think github has killed it. Ultimately versioning has become optimal for the module author and thus less likely to happen.

Github with its README.md has probably killed widespread use of pod6 for module documentation as well (one of my personal bugbears!).

I'm not keen on github's influence for either of these things but this is sadly the reality of things.

stmuk avatar Apr 12 '18 08:04 stmuk

Github versioning is weak

It doesn't have to be. tony-o++ has a script that pulls out versions from META6.json changes and maps versions to commits. So to "release" a new version the author merely needs to bump the version in the META file. I thought zef was already using that map, though my tests with WWW module right now didn't find any but the latest version.

zoffixznet avatar Apr 12 '18 08:04 zoffixznet

@stmuk As you are aware, the Go people are working on the versioning problem. I don't see it as a lost battle. tony-o's solution may work, although I would prefer something more obvious like a tag.

nxadm avatar Apr 12 '18 09:04 nxadm