JsMin and source maps

[dazinator]

I am wondering if it might be possible to enhance the C# JsMin implementation to also produce sourcemaps. Is this something you would be interested in?

I found a C# library for dealing with sourcemaps: https://github.com/nippur72/SourceMaps.CSharp

But I am going to have to do some serious revision before I try to implement anything.. Source Maps don't appear to be trivial.

[Shazwazza]

JsMin only processes one file at a time so not sure what part of the process JsMin would actually need to be involved. Wouldn't generating source maps be done at the bundle processing level not the individual file pipeline level ?

Apparently you can also have inline source maps which exist directly in your code, it could be an option. It would make the minified file a bit bigger but might be an easy way to get source maps working directly with a file processor

[dazinator]

Yeah JsMin would produce a sourcemap for the individual file, which maps back from the minified file, to the original (unminified) source file.

I can't remember which order smidge processes in, whether it first produces the composite file then minifies that, or whether it minifies individual files, then combines them - but I don't think that makes much of a difference in terms of adding general source map support to JsMin.

So given a file:

• foo.js

JsMin would produce:

• foo.js.min
• foo.js.min.map

Yup, you can inline source maps into the generated file in which case it wouldn't produce a seperate .Map file.

It would also need to handle the fact that the unminified source file foo.js may itself, already have a reference to an existing sourcemap if it is already the result of a transformation. For example:

1. foo.ts --> (foo.js | foo.js.map) -> (foo.js.min | foo.js.min.map)

The same is true when adding source map support at the composite / bundle level. As each script is combined into a single bundle file, an index source map can be built for the bundle file (and inlined into the bundle file)

So given the files:

• foo.js.min
• foo.js.min.map (perhaps already inlined into foo.js.min)
• bar.js

Bundling produces:

• bundle.js
• bundje.js.map (perhaps inlined into bundle.js)

The latter (index / composite source map) isn't hard to produce. Its easier than producing "regular" source maps. I managed to get this working in chrome.. It didn't work with edge though. Hope it's just an Edge issue not yet supporting v3 index source maps.

[dazinator]

Have started work on a c# source map library, as I looked at a few but they weren't up to scratch. Once this is complete I'll have a go at updating JsMin to produce sourcemaps.

[dazinator]

Made some good progress today on a c# source map library: I have a test which shows how to build a very simple source map

Next step is to take JsMIn, and make it use the SourceMapBuilder as it makes modifications. That will then produce a sourcemap, which it can inline into the minified js file.

[dazinator]

Ohh wow.. going through JsMin now.. I'd like to congratulate Douglas Crockford on some of the most unreadable code ever.. theA ? I am going to refactor it to make it a bit less of a head **** if that's possible

[dazinator]

So far, the biggest challenge with adding source map support to JsMin is that JsMin seems to write a single character at a time. To build the source map, I need to map positions of whole identifiers. So for example, given this content:

function(){
var x = "foo";
};

JsMin will process and write an individual character at a time:

f
u
n 
c
t
I
o
n

etc.

And as it processes each character in the source it may decide to skip over certain things like whitespace characters etc, and this is what results in a minified output.

However to build a source map, I need to find a place in the code where I can grab hold of a complete identifier i.e the word "function" in this case, so I know its starting position.

So it's a case of trying to understand JsMin now, and trying to understand if it can be changed to "build up" individual characters into a complete token, before they are written, so that they can also be added to the source map.

I was surprised to find that JsMin doesn't seem to perform any re-naming of things. So variable names and function names all stay the same. I think this might simplify the source mapping.

[Shazwazza]

Haha! You think that is unreadable, you should have seen it before I've fixed it up over the years :) here's the source of jsmin that is in smidge https://github.com/Shazwazza/JsMinSharp

But yes because it doesn't build an AST you're going to be in a mine field of pain to do this. I have code in there that tracks a return statement so you could do something like that to track function but i dare say that its not going to be fun. The other reason the code is near impossible to follow is because it doesn't create many vars and allocates very little. I wonder if it's just going to be easier to stick with Nuglify if you want source maps created since that will do all this for you i think. Otherwise Uglify via NodeServices definitely will. I know this is much slower but I've experienced a lot of pain fixing jsmin for various things and its not fun! ;)

[dazinator]

It will be much easier to stick with a node service for this, but I am not giving up just yet!

I have made some more progress today.

I have actually got the source map being produced and inlined, to the point where you can now see the original source along side the minified file:

However attemptijng to set a break point at a particular location isn't lining up correctly (you will see some of those line numbers are greyed out), and this is because I am still trying to get my head around the mapping.

After researching some more about the mappings, I now think I can produce the mapping without having to map whole identifiers (variable names and function names) at all.. I think I can just produce a mapping that maps individual character positions.. I'm still fiddling but in the end, even if the mapping aspect doesn't work perfectly, it still might be handy to be able to peak at the original source like I have it working at the mo..

[Shazwazza]

Closing this since it's just not worth doing this