deno icon indicating copy to clipboard operation
deno copied to clipboard

Disable snapshots when cross compiling

Open littledivy opened this issue 3 years ago • 4 comments

When cross-compiling, snapshots are built for the host architecture instead of the target. There is no easy workaround for this.

Unblocks our CI to produce the following targets:

  • aarch64-android-linux
  • aarch64-unknown-linux-gnu
    • https://github.com/denoland/deno/issues/1846
    • https://github.com/denoland/deno/issues/4862

littledivy avatar Mar 14 '22 10:03 littledivy

I've been looking into this to support the Conda packages (conda-forge/deno-feedstock#68), and have successfully cross-built Deno for Linux aarch64 (the Conda CI is currently failing, but I have working local builds).

The things I see needing to be done (or nice to have) here:

  • [ ] Fix feature defaults to enable a build without unconditionally activating include_js_files_for_snapshotting on dependencies [mandatory, the patch in conda-forge does this crudely]
  • [ ] Correct cli/ops/mod.rs to include source code when snapshotting is disabled [mandatory, patch in conda-forge does this]
  • [ ] Revisit feature flags for selecting or disabling snapshotting, perhaps to make snapshotting behind a feature flag that is enabled by default [patch in conda-forge has vv. incomplete start on this]
  • [ ] Modularize cli/build.rs and gate more things behind feature flags, so it doesn't depend on Deno & the runtime when it isn't snapshotting (otherwise cross-builds compile Deno twice, once for the build arch to support the snapshots that aren't created, and again for the target arch)

Relatedly: one of the challenges is on the things behind include_js_files_for_snapshotting, which causes binaries to include files by reference instead of including their source code. Would it be worth considering always including the sources? A rough count finds 18M of JavaScript & TypeScript source in the runtime, cli, and ext trees; that would be a substantial add. Brotli-compressing the whole bundle brings it down to 2.2M while separate Deflate compression yields a 4.3M zip, suggesting compressed versions of the source code could be unconditionally included with a modest increase in binary size.

mdekstrand avatar Oct 31 '23 19:10 mdekstrand

A further hitch has arisen: with Deno 1.39.0, snapshotting is more deeply baked into Deno's working, so the patches I currently have to work without snapshotting cause Deno.build to be incorrectly populated.

mdekstrand avatar Dec 21 '23 14:12 mdekstrand

Deno.build fields are being populated during snapshot time (improving startup time perf):

https://github.com/denoland/deno/blob/cdbf902499dc62a4fb9f8cbf2a47a7b446623929/runtime/ops/bootstrap.rs#L32-L46

which means its currently not possible to compile Deno CLI without snapshots.


Discussed this offline and will be working towards aarch64 linux builds with snapshots.

littledivy avatar Dec 22 '23 10:12 littledivy

Thanks — looking forward to official aarch64 builds!

It would be useful for packagers, however, to support building the Deno CLI without snapshots — packaging environments often require cross-builds. The conda-forge CI infrastructure, for example, only supports native builds for x86-64 (I co-maintain the package there).

mdekstrand avatar Dec 22 '23 14:12 mdekstrand

Official aarch64 linux builds are available. Closing

littledivy avatar Jun 13 '24 12:06 littledivy