Consider splitting the package out across multiple packages / modules

[Raynos]

Both async and underscore currently suffer from being a "kitchensink" of anything possibly uesful, these modules didn't start out that way but more and more stuff was added to it.

Projects like lodash or underbar have mitigated against this effect by splitting the module out across multiple modules / packages. @jdalton & @Matt-Esch have done good work to allow usage of the functionality piece by piece.

This allows consumers of the module to use a tighter subset of the functionality.

@gozala also had a similar approach with splitting out reducers and reducible and then a set of optional functions all over npm like buffer-reduce , tree-reduce , dom-reduce and others.

There probably is still value in having a module that combines everything together in a "kitchensink" for people that prefer that just like lodash allows you to get it all or get the functions one by one.

[evocateur]

+1

I'm also mildly concerned about portability across versions of Node when an entire copy of EventEmitter and util.inherits is present inside the library. If these bits are just for browser compatibility, they should be used to build a browser version (using code from isaacs/inherits and Gozala/events optionalDependencies instead of copied), perhaps distributed via bower or component.

I'm really impressed by the library overall, and will be looking for opportunities to use it in the near future.

[dashed]

+1.

Ideally, I'd like to see something like http://lodash.com/custom-builds

[Bartvds]

+1

I'd use this for the excellent streaming features but there are quite a lot of helpers that are present in any general purpose util (underscore/lodash).

Without these general-purpose utils there also might be less temptation to use _ as example identifier: I personally feel that is per convention claimed by underscore/lodash etc.

[Raynos]

You can also use browserify --standalone to generate a browser compliant script file and just use node style commonJS.

[caolan]

Yep, I think more modularity would be good too. Already talked about using browserify for dependencies... definitely on the todo list.

[Raynos]

awesome, if you get stuck with the refactor or want a review let me know, love to help.

[aaronmccall]

+1

[prateekbhatt]

+1

[calvinmetcalf]

you can use readable-streams module, it's the same one as in node 0.10.0 but if you require it via npm you'll get the same api in older version of node (and if you aren't using streams 2.0 you should get on that, it's like 100000% easier then the old one).

[caolan]

btw @Raynos we're now using browserify to build the standalone highland.js file

[Raynos]

@caolan awesome. That's a great improvement :)

The next step is to split the 2000 line file across multiple logical modules.

[dashed]

Wish this is also done in async.js ;)

[DamonOehlman]

Taking a look at underbar I'd definitely :+1: going with that approach over others.

Personally, I love being able to communicate to someone "Hey, you should go try highland.js" and also this involves is them having to:

a. look at the highland.js project page / website b. start using it in their projects (be it node or something browserified) by simpling installing one package (npm install highland --save)

Then using the intelligence of browserify's static analysis to only bring in the functionality that is required based on what I have used is fantastic. I agree that the work done with lodash is an improvement on the kitchen sink approach, but my personal opinion is that using that approach creates a lot of module pollution. While I'm sure npm can handle all those separate modules, I'm not sure that it's efficient way for us to communicate as humans. Just my 2c - but it's 2c I'm pretty passionate about...

[calvinmetcalf]

You can have a kitchen sink module that collects them all in addition to individual modules, take a look at liar On Feb 27, 2014 6:10 PM, "Damon Oehlman" [email protected] wrote:

Taking a look at underbar https://github.com/Matt-Esch/underbar I'd definitely [image: :+1:] going with that approach over others.

Personally, I love being able to communicate to someone "Hey, you should go try highland.js" and also this involves is them having to:

a. look at the highland.js project page / website b. start using it in their projects (be it node or something browserified) by simpling installing one package (npm install highland --save)

Then using the intelligence of browserify's static analysis to only bring in the functionality that is required based on what I have used is fantastic. I agree that the work done with lodash is an improvement on the kitchen sink approach, but my personal opinion is that using that approach creates a lot of module pollution. While I'm sure npm can handle all those separate modules, I'm not sure that it's efficient way for us to communicate as humans. Just my 2c - but it's 2c I'm pretty passionate about...

Reply to this email directly or view it on GitHubhttps://github.com/caolan/highland/issues/3#issuecomment-36303719 .

[caolan]

@DamonOehlman quite like the look of the underbar approach you posted :)

[DamonOehlman]

Kudos to @Raynos to finding that one, I didn't know it before he mentioned it...

It's probably also worth seeing if @kriskowal has any thoughts also as IMO he's done some excellent work with the MontageJS Collections repo along the same lines as what we are talking about here...

[caolan]

It would be nice if there was a way to require extensions to the Stream prototype that didn't suck too. Perhaps returning a new instance of the module with an updated Stream?

On 9 March 2014 23:44:10 GMT+00:00, Damon Oehlman [email protected] wrote:

Kudos to @Raynos to finding that one, I didn't know about that one :)

It's probably also worth seeing if @kriskowal has any thoughts also as IMO he's done some excellent work with the MontageJS Collections repo along the same lines as what we are talking about here...

---

Reply to this email directly or view it on GitHub: https://github.com/caolan/highland/issues/3#issuecomment-37144224

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[DamonOehlman]

Is there another issue where this is worth discussing? It's probably easy enough to bring in some of the stream heavyweights to get their opinion on it...

[calvinmetcalf]

if you require('foo') and inside the foo module it did require('readable-streams).Readable.prototype.foo = blah that would be cached and apply to everything else that made streams using that module.

[Raynos]

@caolan what do you mean about stream extension ?

[caolan]

@Raynos well, the chaining method API is nice, but it's annoying to provide your own methods outside of 'Highland core'.

[Raynos]

@caolan chaining can be done in user land. with something like enchain

Baking such a DSL into the library is meh.

[kriskowal]

I did support a collections.js kitchen sink module that would install all of the collections in global scope, https://github.com/montagejs/collections. Folks found the behavior of require("collections") understandably confusing on Node.js, where I encourage use of individual modules. If you go this route, I would hide the kitchen sink module from users and use it only for a build step in generating a script.

[Majiir]

+1

I wanted to use batchWithTimeOrCount today, but I was surprised to see that I would have to include the entire library in my Browserify bundle.