Decouple TsProject from gulp and rename to TsBundler, Ts2Js, TsMinifier

[scniro]

Gulp is great, but lately I've been getting away from it. This library should be able to be used as a standalone module with no coupling to gulp.

• Why is it currently used exclusively with gulp?
• Can we do without?

[ToddThomson]

@scniro TsProject uses gulp only because I use gulp in all my personal build scenarios. However, I also do see the need to decouple TsProject from the gulp build pipeline. I am considering a couple of name changes:

• The node module (essentially your request) will be called TsBundler. Essentially this is a Typescript single module bundle builder for a developers own source code modules ( apps, components, libraries ).
• TsBundler will provide an API method, src(), that can be used to stream bundle vinyl files through the gulp build pipeline.

What are your thoughts?

[ToddThomson]

TsProject will be decoupled from gulp and a separate gulp plugin will be built using the resultant TsPackage node module. The gulp plugin will be called gulp-tspackage.

[ToddThomson]

Additionally the Typescript minifier will be broken out as a node package called TsMinify

[ToddThomson]

TsPackage will have essentially the following node.js API:

const tsbundler = require( "tsbundler" );

tsbundler.build( pathToTsConfigJson, ( buildResult ) => {
   if ( !buildResult.Succeeded() ) {
       // Handle errors here
   }
   // Done processing
});

or

const tsbundler = require( "tsbundler" );

const builder = tsbundler.build( tsConfigPath );

builder.build( ( buildResult ) => {
   if ( !buildResult.Succeeded() ) {
       // Handle errors here
   }
   // Done processing
});

or for gulp

const tsbundler = require( "tsbundler" );

const builder = tsbundler.builder( tsConfigPath );

const bundleStream = builder.src();

[ToddThomson]

@scniro I'd be happy to look at what you would like for builder.run() to return in the callback. I'm thinking about passing an optional output stream (file, stream, buffer). If not provided then the build uses the Typescript config json file options to determine where the bundle output goes. If provided then the output goes to the output stream. Let me know what you think.

[scniro]

@ToddThomson hey!! Thanks for being so awesomely responsive and open to working on this. I'd gladly help out to get something cool for us. I'm quite busy as of late, but I'll chime in on what I have and what sort of vision I had as well.

I dug through the source a bit and came up with a hacky gulp-free work around already that can surely be improved on. In TsProject.ts, I noticed this function returns a stream.Readable. With this I can easily leverage fs to write out my results as follows. Surprisingly easily after digging for a bit. Here's what I came up with...

let fs = require('fs');
let tsproject = require('tsproject');

tsproject.src('./').on('data', function (file) {

    fs.writeFile(someSortOfPath, file.contents, function (err) {
        //[...]
    });
});

Like I said I'm pretty booked this weekend but I'll try to breakaway and get back to this thread to come up with an idea for an api that makes sense and looks appealing 👍 I do definitely like what you came up with already, maybe we can get some out-of-the-box-write functionality in there to mimic gulp.dest like I somewhat came up with maybe with an outDir option or something? 😖

[niemyjski]

That sounds good to me :). It would be nice to split these out and rev them independently.

[niemyjski]

It would be nice for existing users to make the minimal amount of changes to the api surface of the current gulp integration. But if there is a good change to be had, make it!

[ToddThomson]

@niemyjski The node package you will need to import/require will change to "tspackage-stream". Essentially this package will wrap TsPackage and provide a streamed output from the project build. With TsProject the API was tsProject.src(). I am not sure if the function name "src" will still be appropriate. I'll know more as I make progress on this issue.

[ToddThomson]

The new TsPackage will support compilation to memory or to disk. The callback passed to TsPackage will return a BuildResult object that will provide an optional "output" property which will contain the emittedFiles output from the compiler. When compiling to disk the BuildResult object will contain a list of all files emitted.

TsPackage will default to compiling to disk. Setting the option compileToMemory will force compilation to memory.

[ToddThomson]

@niemyjski I need your input on this...

I think that TsPackage will treat the compilation of the tsconfig.json "files" as intermediates for the processing of bundles. The tsconfig.json project "files" will still be emitted to disk depending on the Typescript compiler options, but the BuildResult will only include success or failure of building bundles. This means that the TsPackage-stream node package will only stream bundle files.

The TsPackage setting for compiling to memory will be "compileBundlesToMemory". Intermediates will only ever be compiled to disk.

This seems much cleaner to me. The packager just deals with making bundles from ts source files/modules.

[niemyjski]

I'm not sure I 100% follow. I'm not sure tsconfig.json should know anything about how it's about to be compiled, that would be an option inside of the tspackage-stream package. I think that it would just take the output and output to disk. Would the stream always contain a in memory version of the bundle? I haven't had to touch gulp in 6+ months so It's not so fresh in my head.

[ToddThomson]

@niemyjski TsPackage-Stream.src() will have the same API as TsProject. However, TsPackage-Stream will be dependent on TsPackage. TsPackage is now just a module bundler for Typscript. It will not be dependent on gulp and streaming vinyl files. So, to keep the TsPackage API clean I will no longer do anything other than write the compiled project files to disk. The TsPackage API will return a BuildResult object that only is concerned with the bundle compilation results.

The difference for you is that TsProject streamed ALL emitted files (project files and bundle files ) through the gulp build pipeline. TsPackage-Stream will only stream emits from bundle compilation. I'm pretty sure that this will be fine for you: project file emits get written to disk and bundles get streamed through the gulp build pipeline.

[niemyjski]

Yeah, I think that will be fine for our use cases. This should also make it much cleaner and allow more people to use it!!

[ToddThomson]

Update on package names:

The Typescript bundle builder will be called TsBundler. The Typescript minifier will be called TsMinifier The Typescript compiler will be called Ts2Js

[niemyjski]

I Like it :)

[ToddThomson]

This issue will now be tracked by toddthomson/tsbundler#1 and toddthomson/tsminifier#2

[ToddThomson]

Please see TsMinifier and TsBundler issues to follow the decoupling task.

[ToddThomson]

Additionally, the compiler from TsProject will be broken out to a separate package called Ts2Js

[ToddThomson]

Reopening until deprecation.

[ToddThomson]

Instead of having a separate gulp plugin for TsBundler, I will provide a bundleBuilder.src() function to stream bundle files ( vinyl files ).

[ToddThomson]

@niemyjski I am almost done with this task! It looks like it will provide a huge upgrade to the process of bundling, minifying and compiling Typescript projects!

Here is the gulp way of consuming the output from the bundler:

import * as tsb from "./src/tsbundler";
var gulp = require( "gulp" );

var bundlerOptions: tsb.BundlerOptions  = {
    logLevel: 0,
    verbose: true
};

var bundleBuilder = TsBundler.builder( "./src", bundlerOptions );

bundleBuilder.src()
    .pipe( gulp.dest( "./dist" ) );

[ToddThomson]

@scniro The new TsBundler by default will compile and emit all bundle files to memory and return the output in a BuildResult.

@niemyjski @scniro The new bundler no longer emits the Typescript project files ( emit is set to false in the CompilerOptions for this step in the build ). TsBundler only emits bundle specific output. This output by default is a CompilerOutput object:

interface CompilerOutput {
        fileName: string;
        emitSkipped: boolean;
        codeFile?: CompilerFile;
        mapFile?: CompilerFile;
        dtsFile?: CompilerFile;
        diagnostics: ts.Diagnostic[];
    }

My question is about having the output written to disk. I feel that this should be handled by a BundlerOptions property named outDir. When set the output will ALSO be written to files using the OutDir offset to the CWD. Let me know if that is sufficient for your purposes.

EDIT: BundlerOption.outputToDisk: boolean Set to true to emit CompilerFiles to disk. Note this option does not apply to the src() streaming interface.

EDIT NOTE: The TsBundler still uses the [bundles] section in the Typescript project configuration file to drive the bundling process ( that really is the whole point of this set of packages - driving the bundling and minification process from a project build structure ). There is already an outDir property defined there so having the outDir in the BundleOptions does make sense.

[ToddThomson]

I am moving this issue to Milestone: TsProject Release 4.1.

Almost all the implementation has been completed. The TsProject release 4.0 will focus only on getting the bundling component of TsProject to work with Typescript 4.0.