[feature request] Bundling imported files

[Le0Developer]

Essentially, I'd love if yuescript had bundling support buildin (similar to esbuild and the JS ecosystem).

Splitting code across multiple files and importing it improves readability but sadly makes minification worse when finally shipping to users.

Current solution

I am currently using my own solution which I wrote a few years ago which works by statically looking up all require() calls and then inlining the files by loading them into package.preload.

For example:

-- entry.yue
import "other" as :eprintln

eprintln "hello world"

-- other.yue
export eprintln = (...) -> print "[err]", ...

Turns into something like (very simplified example):

package.preload["other"] = function()
  local _module_0 = {}
  _module_0.eprintln = function (...)
    return print("[err]", ...)
  end
  return _module_0
end

local eprintln
do
  local _obj_0 = require("other")
  eprintln = _obj_0.eprintln
end

eprintln("hello world")

This approach is very simple and reuses the packaging system of lua and works out-of-the-box on pretty much all existing lua code.

However, it has its drawbacks. Mainly, it uses a lot of strings that cant be minified. For example:

-- before minification
do
  local _obj_0 = require("other")
  eprintln = _obj_0.eprintln
end
-- after minification
local a=require"other";
local b=a.eprintln

Ideal solution

I'd love if the yuescript compiler could:

1. scan imported modules (probably only the ones using the import "module" syntax) (if it couldnt be found at compile time, fallback to a require() call like currently)
2. if found, inline the code (see example)

So using our original example, the result could look something like:

-- other.yue file
local eprintln -- exported function
do
  eprintln = function(...)
    return print("[err]", ...)
  end
end

-- entry.yue
eprintln("hello world")

This allows for much better minification, as no strings are used for tables or imports in the resulting code.

See:

local a=function(...)return print("[err]",...)end
a("hello world")

[pigpigyyy]

I think it would be better to set up a workflow by reusing your luapack project for packaging, rather than implementing everything directly in the YueScript repository. We could potentially expand on the design and add functionality that allows the YueScript compiler tool to be easily integrated into various build systems.

[Le0Developer]

Had an idea on how to use macros for this and made a new project which does exactly what I need: https://github.com/Le0Developer/yuepack

It converts

-- main.yue
text = $import "module", "text"

print "Hello", text

-- module.yue
$export "text", "world"

Into this intermediate yue code:

local _export_module_text
macro import = (module, name) ->
  map = {["module"]: {["text"]: "_export_module_text"}}
  resolved = map[module::sub 2, -2]
  resolved = resolved[name::sub 2, -2] if resolved?
  -- fallback to require if we can't resolve statically
  return "require%q[%q]"::format module, name unless resolved?

  resolved -- this will be the name of a local variable
macro _set_current_module = (module) ->
  global _current_module = module
  ""
macro export = (name, expr) ->
  map = {["module"]: {["text"]: "_export_module_text"}}
  uplocal = map[_current_module::sub 2, -2]
  uplocal = uplocal[name::sub 2, -2] if uplocal?
  -- we cant fallback to an export here, so throw error
  -- this shouldnt happen in the first place
  return "error 'export %q not found in %q'"::format name, _current_module unless uplocal?

  "%s = %s"::format uplocal, expr


-- module
$_set_current_module "module"
do
  -- module.yue
  $export "text", "world"
  

-- _entry
$_set_current_module "_entry"
do
  text = $import "module", "text"
  
  print "Hello", text
  

Which is transformed into

-- yuepacked using 0.1.0
local _export_module_text
do
	do
		_export_module_text = "world"
	end
end
do
	local text = _export_module_text
	print("Hello", text)
end

Which can then be minified into

local a="world"local b=a;print("Hello",b)