Refactor to facilitate amd modules.

[xavierzwirtz]

This refractors the globals map in InternalCompiler to a map of modules containing methods. Also adds dependency tracking for module methods, so that when output to an AMD module it can load its dependencies.

The next step is to add outputting to an AMD module, but so far I have not been able to think of a way that does not require a pretty hefty refactor.

[alfonsogarciacaro]

Hi Xavier:

Great work, thanks for that. I've made quite a refactoring myself lately so we need to be careful when merging the code. Your proposal seem very interesting and I'd like to have a look at the code and ask you about it later. Will this also be compatible for CommonJS/node modules?

Cheers!

[xavierzwirtz]

That is exactly what I am thinking @alfonsogarciacaro. We should be able to make it compatible with CommonJS/node modules as well very easily.

The big issue is going to be supporting three different output types,

• Global Variables
• AMD Modules
• Common JS Modules

Currently DefineGlobal returns a Var. I could not see an easy way of transforming that into a property call when it goes to print. They seem to be heavily dependent on the scope, and theres not a way to transform the accessing of a Var to the accessing of a property on a module.

[alfonsogarciacaro]

I think the trick is to wrap the generated code in a closure and change the parameters depending on the context. Have a look at the last lines of the Ractive.js library, but please give a couple of days to have a proper look at your code. In any case it's great to have people jumping in to improve FunScript! Welcome to the team!

[xavierzwirtz]

I was originally thinking of an amd module per fsharp module, I see what you mean about the Ractive.js code though. Doing a single module per variable would make the implementation easier, and would allow for more optimizations using the require.js optimizer.

Glad to be a part of the team, this project is looking more awesome the more I dig into it.

[xavierzwirtz]

The F# code that we're compiling:

module Foo

    let x() =
        "x"
    let y() =
        "y"
    let z() = 
        x() + y()

My original intention was for it to be compiled to this:

define('Foo', [], function() {
    var fmodule = {
        x: function() {
            return "x";
        },
        y: function() {
            return "y";
        },
        z: function() {
            return fmodule.x() + fmodule.y();
        }
    }
    return fmodule;
});

However, this would be easier to implement, and more flexible:

define('Foo.x', [], function() {
    return function() {
        return "x";
    };
});
define('Foo.y', [], function() {
    return function() {
        return "y";
    };
});
define('Foo.z', ['Foo.x', 'Foo.y'], function(Foo_x, Foo_z) {
    return function() {
        return Foo_x() + Foo_z();
    };
});

Most of the changes having to do with type JSModule could be removed,and just keep the dependency tracking pieces.

So we could switch back to a simpler globals map, instead of a list of JSModule that contains a list of JSModuleMembers

type JSModule = 
    { Dependencies: seq<JSModule>;
      Name: string;
      Var: Var;
      Assignment: List<JSStatement> }

let mutable globals = Map.empty<string, JSModule>

[alfonsogarciacaro]

Mmm, I'm not sure it's a good idea to tight so much the generated code to a certain specification. Take into account that the code must be ready for a variety of scenarios: browser, node, Edge.js... The unit tests, for example, work by wrapping the code in a callback function and passing it to Edge.js (see here). What I'm thinking at the moment is creating a namespace object and then use this object as needed depending on the context, something along these lines:

    var ns = {};
    ns['module1'] = {};
    ns['module1']['f1'] = function() { /* ... */ };
    ns['module1']['f2'] = function() { /* ... */ };
    ns['module2'] = {};
    ns['module2']['f1'] = function() { /* ... */ };

And then, if necessary, wrap it like this:

(function(global) {
    /* Previous code */

    // export as Common JS module...
    if ( typeof module !== "undefined" && module.exports ) {
    module.exports = ns;
    }

    // ... or as AMD module
    else if ( typeof define === "function" && define.amd ) {
    define( function() {
        return ns;
    } );
    }

    // ... or as browser global
    global.FunScript = ns;
}(typeof window !== 'undefined' ? window : global ) );

What do you think?

[xavierzwirtz]

Ah, I was attacking it from the perspective of having different output methods availible at compile time, ie AMD, CommonJS, globals. A single unified output that works with all 3 would be much easier for the end user though.

The current Compile function could return JS like your first example, and then a CompileModules that compiles more like the second example.

It looks like in your second example you are including all of the generated JS in a single module. It was my goal for the modules to be separated as much as possible. The app I am going to start supplementing with FunScript has a very large client side code base, but most of the modules are not used on every request. I need for the generated modules to be as modular as possible. For instance, I need the following F# to be compiled into at least 2 modules:

module Foo
    let x() =
        "x"
module Bar
    let y() =
        "y"

What about making the api this:

• Compile Same as the exisiting compile, outputs global variables.
• CompileModule Would compile to a single module, with handling for CommonJS and AMD. Same as your example
• CompileModules Would compile to seperate modules, with handling for CommonJS and AMD. Same as your example, just with more modules

[alfonsogarciacaro]

Thanks for the clarifications, I'm getting your point better now. I would reduce your proposals to two main user cases:

1. FunScript is the last layer of the app and the user is not interested in making the code available to other clients. This is what the compiler assumes now, which allows several optimizations:
   1. All the functions are made global (the generated code can be wrapped in a closure to prevent pollution of the global JS namespace) which should be more quickly accessible than a property in an object.
   2. The function names can be compressed.
   3. The code that is not being used won't be compiled to JS.
2. The user wants to create a "library" so the structure of the generated JS must reflect the that of the F# source as much as possible.
   1. Functions (methods/properties/constructors) must be put in an object and have the same names as in F#.
   2. All the user code must be compiled to JS.

It should be possible to make these two options available to the user when compiling. What I'm not sure yet about is generating one AMD module per F# one. If you want to prevent the loading of certain modules, they must be in separate files. Thus we must add this feature to the compiler or simply force the user create a project per module (a "project" can be a simple .fsx file).

Another thing we need to take into account is that, if users are going to use several independent modules, it may make sense to put all the FunScript.Core code (Array, List, String, Event...) in an independent library so the different modules can share it.

[xavierzwirtz]

I agree with your spec, and I think ive already solved the issue of module separation.

I think it should be fairly easy to separate F# modules into separate AMD modules, this PR is already doing most of the work for that. There are two methods for defining 'globals' in my PR:

• DefineGlobalExplicit (moduleName : string) (name : string) Adds a property with the key of name to module moduleName
• DefineGlobal (moduleType : Type) (name : string) Calls DefineGlobalExplicit with moduleType.FullName and name.

Using the code in this PR, the following F# would be separated into multiple JSModules, but would still be output a single blob of JS.

module Foo
    let x() =
        "x"
module Bar
    let y() =
        "y"

Is translated into this data:

{ Name="Foo"
  Members: [
    { Name:"x"
      Var: ...;
      Statements: ... }]}
{ Name="Bar"
  Members: [
    { Name:"y"
      Var: ...;
      Statements: ... }]}

My PR is already doing the separation into different modules internal to FunScript, the only thing that remains is making it output to separate modules.

[alfonsogarciacaro]

Just a question: if you need to develop your app in a modular fashion, why don't you keep the library modules in F# and later link them from your app code? There are several advantages in doing this: particularly because you'll only get the code you really need each time.

I assume you're writing both the library modules and the app code in F#. Or do you really need to call compiled F# from native JS? You still can do this setting explicitly the variables you want to export, but the more I think about it, the more difficulties I foresee in creating a "proper" JS library with FunScript, because the project wasn't designed with this in mind and there are many quirky ways wherein everything can fail.

[mfrawley]

@alfonsogarciacaro This is a really cool if very alpha project. As another outsider I would vote for proposal 2 above, if only because it makes sense for my use case too ;) (a small embedded library which needs to be called from JS). Keep up the good work.

[alfonsogarciacaro]

Hi there! I'm working on a full new version, so this is a bit stopped right now. I'm not sure yet if it's a good idea to create a full library in JavaScript resembling the F# code, because the mechanisms for reference are very different: in F# this is automatic, in JS you usually need to pass an object containing the methods you need. Because of this, even if it's a bit more work, for users wanting to expose functionality to JS clients I think it's better to use a standard JS exporting functionality like module.exports. In any case, I'm still trying to figure out the easiest way to interact with JS for the new version, so any idea is welcome!

[mfrawley]

@alfonsogarciacaro I understand and I would basically be happy with any module standard as long as I can export an interface per module (exposed methods/properties).

e.g. your own proposal from Mar 8th above:

var ns = {};
    ns['module1'] = {};
    ns['module1']['f1'] = function() { /* ... */ };
    ns['module1']['f2'] = function() { /* ... */ };

This seems totally logical to me instead of the current approach of just throwing everything into the global namespace with prefixed names as you do:

Utils__doc$ = (function(unitVar0)
{
    return (window.document);
});

I'm a total F# noob so not sure what's easiest but can you just the JS module pattern and F#'s access modifiers to control what gets bound to exports for a particular module, and what gets assigned to a local?

https://msdn.microsoft.com/en-us/library/dd233188.aspx

module Module1
  let internal myInternalObj = new MyInternalType()
  let publicAdd a b = a + b

Could compile to:

var Module1 = function() {
  var self = {};
  var myInternalObj = new MyInternalType();
  self.publicAdd = function(a,b) { return a + b; }
  return self;
};
//and then export for AMD/CommonJS etc
module.exports  = Module1;

Pros - It's easy to represent internal/private members/properties, it's easy to read and understand. No issues with the dynamic "this" scoping in JS.

Cons - Not as performant as binding to the function prototype. Some JS purists will hate you for using self ;)

Apologies if this is all obvious, and all already considered, I'd like to help if I can.

[alfonsogarciacaro]

This seems very sensible to me (your code is only missing the Module1 function calling itself). As an F# program may contain any number of modules I would mark one (and only one) with an attribute (like JSExport) so the compiler knows which one it should be exported.

I need to think a bit what to do with non-public properties because right now for simplicity the new version attach directly all properties to the type/module object no matter the access level in F#, and changing this would require changing a few other things.

If you want to have a peek at the new version you can have a look at the fnext branch of my fork. However, I got many new ideas after F# eXchange so there'll still be some big changes.

To see how the generated code with the new version looks like, have a look here. Please note that it's less than half of the code generated with the current version (here for comparison) and hopefully somewhat cleaner.

To read about the ideas behind the new version, check issue #171.

Cheers! Alfonso

[mfrawley]

Thanks, that also answers my questions about the TS type providers not working for me (except the basic DOM types which work great). If you want someone to test once you're finished with the refactoring let me know. I'm checking out WebSharper too but this project is closer to what I want/need.