js_of_ocaml
js_of_ocaml copied to clipboard
Published
20 hours ago •
ocsigen
ocsigen
[BUG] ppx_expect output is being included in built javascript files
Describe the bug
I have tests in my code using ppx_expect. The string of these tests can be found in the generated .bc.js file from JSOO
Expected behavior That this code would not survive JS code generation.
Versions 5.5.2
Js_of_ocaml does a fine job of dead-code elimination, but because expect-tests are enabled or disabled vis environment variable, you need to tell jsoo that the expect-test env var is always empty. I forget which var it is, and I also forget how to configure dune to do this, but it does work!
In general, you want this code to survive JS code generation so that you can run test in JavaScript.
You should be able to drop the tests with the following js_of_ocaml compile flags.
--setenv FORCE_DROP_INLINE_TEST=true.
This allows js_of_ocaml to perform static evaluation of the following code https://github.com/janestreet/ppx_inline_test/blob/f88526ee10c8d815653fff621a2816834c81c12d/runtime-lib/ppx_inline_test_lib.ml#L118
let force_drop =
try
ignore (Sys.getenv "FORCE_DROP_INLINE_TEST" : string);
true
with
| Not_found -> false
;;
let get () =
(* This is useful when compiling to javascript.
Js_of_ocaml can statically evaluate [Sys.getenv "FORCE_DROP_INLINE_TEST"]
and inline the result ([`Ignore]) whenever [get ()] is called.
Unit tests can then be treated as deadcode since the argument [f] of the [test]
function below is never used. *)
if force_drop then `Ignore else !action
;;
Thanks. Unfortunately I am still seeing the test strings in the file even after adding
(js_of_ocaml
(flags (:standard --setenv FORCE_DROP_INLINE_TEST=true)))
to my dune config
You also need to build with --profile release dune flag. Otherwise, separate compilation is used for JavaScript compilation and no cross module dead code happen
Or you can set (js_of_ocaml (compilation_mode whole_program)) in dune env. See https://dune.readthedocs.io/en/stable/reference/dune/env.html#env
I can't reproduce. I see the expect_test being dropped when use what has been mentioned in the issue. Can you share some code ? if you're using dune, you should probably try to pass --verbose to check that the expected flags are there.
The code base I am working with is https://github.com/stan-dev/stanc3.
I put together a gist with my dune config, the output of dune verbose, and the generated .js file.
The command I used in full was dune build src/stancjs/ --profile=release --force --display=verbose > dune_output.txt 2>&1
Here is a string which only appears in the source code as part of the expect tests:
(RequireAllCondition (Exact stan::is_vt_not_complex) (TemplateType T0__1__))
$ grep "(RequireAllCondition (Exact stan::is_vt_not_complex) (TemplateType T0__1__))" ./stancjs.bc.js -c
1
Could you try to downgrade to v0.16.0 (assuming you are indeed using v0.17.0)
I am using 0.16.0. Here's my opam list
Details
# Packages matching: installed
# Name # Installed # Synopsis
angstrom 0.16.0 Parser combinators built for speed and memory-efficiency
astring 0.8.5 Alternative String module for OCaml
base v0.16.3 Full standard library replacement for OCaml
base-bigarray base
base-bytes base Bytes library distributed with the OCaml compiler
base-threads base
base-unix base
base-windows v0.16.3 Full standard library replacement for OCaml
base_bigstring v0.16.0 String type based on [Bigarray], for use in I/O and C-bindings
base_bigstring-windows v0.16.0 String type based on [Bigarray], for use in I/O and C-bindings
base_quickcheck v0.16.0 Randomized testing framework, designed for compatibility with Base
base_quickcheck-windows v0.16.0 Randomized testing framework, designed for compatibility with Base
bigstringaf 0.9.1 Bigstring intrinsics and fast blits based on memcpy/memmove
bin_prot v0.16.0 A binary protocol generator
bin_prot-windows v0.16.0 A binary protocol generator
bisect_ppx 2.8.3 Code coverage for OCaml
camlp-streams 5.0.1 The Stream and Genlex libraries for use with Camlp4 and Camlp5
chrome-trace 3.10.0 Chrome trace event generation library
cmdliner 1.2.0 Declarative definition of command line interfaces for OCaml
conf-bash 1 Virtual package to install the Bash shell
conf-flambda-windows 1
conf-gcc-windows 1
conf-gcc-windows64 1 Virtual package specifying the location of GCC cross-compiler for 64-bit Windows
conf-libpcre 1 Virtual package relying on a libpcre system installation
conf-pkg-config 3 Check if pkg-config is installed and create an opam switch local pkgconfig folder
core v0.16.1 pinned to version v0.16.1
core-windows v0.16.1 Industrial strength alternative to OCaml's standard library
core_kernel v0.16.0 Industrial strength alternative to OCaml's standard library
core_unix v0.16.0 Unix-specific portions of Core
cppo 1.6.9 Code preprocessor like cpp for OCaml
csexp 1.5.2 Parsing and printing of S-expressions in Canonical form
dot-merlin-reader 4.9 Reads config files for merlin
dune 3.10.0 Fast, portable, and opinionated build system
dune-build-info 3.10.0 Embed build information inside executable
dune-configurator 3.10.0 Helper library for gathering system configuration
dune-configurator-windows 2.5.1 Helper library for gathering system configuration
dune-private-libs 2.5.1 Private libraries of Dune
dune-private-libs-windows 2.5.1 Private libraries of Dune
dune-rpc 3.10.0 Communicate with dune using rpc
dyn 3.10.0 Dynamic type
either 1.0.0 Compatibility Either module
expect_test_helpers_core v0.16.0 Helpers for writing expectation tests
fiber 3.7.0 Dune's monadic structured concurrency library
fieldslib v0.16.0 Syntax extension to define first class values representing record fields, to get and set record fields, iterate and fold over all fields of a record and create new record values
fieldslib-windows v0.16.0 Syntax extension to define first class values representing record fields, to get and set record fields, iterate and fold over all fields of a record and create new record values
fix 20230505 Algorithmic building blocks for memoization, recursion, and more
flexdll-windows 0.42 FlexDLL is a Windows dynamic linker extension allowing DLLs to depend on each other
fmt 0.9.0 OCaml Format pretty-printer combinators
fmt-windows 0.9.0 OCaml Format pretty-printer combinators
fpath 0.7.3 File system paths for OCaml
gen 1.1 Iterators for OCaml, both restartable and consumable
int_repr v0.16.0 Integers of various widths
int_repr-windows v0.16.0 Integers of various widths
jane-street-headers v0.16.0 Jane Street C header files
jane-street-headers-windows v0.16.0 Jane Street C header files
jbuilder 1.0+beta20.2 Fast, portable and opinionated build system
js_of_ocaml 5.7.0 Compiler from OCaml bytecode to JavaScript
js_of_ocaml-compiler 5.7.0 Compiler from OCaml bytecode to JavaScript
jst-config v0.16.0 Compile-time configuration for Jane Street libraries
jst-config-windows v0.16.0 Compile-time configuration for Jane Street libraries
lambda-term 3.3.2 Terminal manipulation library for OCaml
landmarks 1.4 A simple profiling library
landmarks-ppx 1.4 Preprocessor instrumenting code using the landmarks library
logs 0.7.0 Logging infrastructure for OCaml
lwt 5.7.0 Promises and event-driven I/O
lwt_react 1.2.0 Helpers for using React with Lwt
menhir 20230608 An LR(1) parser generator
menhir-windows 20230608 An LR(1) parser generator
menhirLib 20230608 Runtime support library for parsers generated by Menhir
menhirLib-windows 20230608 Runtime support library for parsers generated by Menhir
menhirSdk 20230608 Compile-time library for auxiliary tools related to Menhir
menhirSdk-windows 20230608 Compile-time library for auxiliary tools related to Menhir
merlin 4.11-414 Editor helper, provides completion, typing and source browsing in Vim and Emacs
merlin-lib 4.11-414 Merlin's libraries
mew 0.1.0 Modal editing witch
mew_vi 0.5.0 Modal editing witch, VI interpreter
num 1.4 The legacy Num library for arbitrary-precision integer and rational arithmetic
num-windows 1.1 The legacy Num library for arbitrary-precision integer and rational arithmetic
obelisk 0.6.0 Pretty-printing for Menhir files
ocaml 4.14.1 The OCaml compiler (virtual package)
ocaml-base-compiler 4.14.1 Official release 4.14.1
ocaml-compiler-libs v0.12.4 OCaml compiler libraries repackaged
ocaml-compiler-libs-windows v0.12.0 OCaml compiler libraries repackaged
ocaml-config 2 OCaml Switch Configuration
ocaml-lsp-server 1.17.0 LSP Server for OCaml
ocaml-options-vanilla 1 Ensure that OCaml is compiled with no special options enabled
ocaml-syntax-shims 1.0.0 Backport new syntax to older OCaml versions
ocaml-version 3.6.2 Manipulate, parse and generate OCaml compiler version strings
ocaml-windows 4.14.1 A meta-package depending on any OCaml cross-compiler
ocaml-windows64 4.14.1 OCaml cross-compiler for 64-bit x86 Windows targets
ocaml_intrinsics v0.16.0 Intrinsics
ocamlbuild 0.14.2 OCamlbuild is a build system with builtin rules to easily build most OCaml projects
ocamlc-loc 3.10.0 Parse ocaml compiler output into structured form
ocamlfind 1.9.6 A library manager for OCaml
ocamlfind-windows 1.9.1 A library manager for OCaml
ocamlformat 0.26.1 pinned to version 0.26.1
ocamlformat-lib 0.26.1 OCaml Code Formatter
ocamlformat-rpc-lib 0.26.1 Auto-formatter for OCaml code (RPC mode)
ocamlgraph 2.1.0 A generic graph library for OCaml
ocp-indent 1.8.1 A simple tool to indent OCaml programs
ocplib-endian 1.2 Optimised functions to read and write int16/32/64 from strings and bigarrays
odoc 2.2.1 OCaml documentation generator
odoc-parser 2.0.0 Parser for ocaml documentation comments
opam-core 2.1.5 Core library for opam 2.1
opam-file-format 2.1.6 Parser and printer for the opam file syntax
opam-format 2.1.5 Format library for opam 2.1
opam-installer 2.1.5 Installation of files to a prefix, following opam conventions
ordering 3.10.0 Element ordering
parsexp v0.16.0 S-expression parsing library
parsexp-windows v0.16.0 S-expression parsing library
patdiff v0.16.0 File Diff using the Patience Diff algorithm
patience_diff v0.16.0 Diff library using Bram Cohen's patience diff algorithm
pcre 7.5.0 Bindings to the Perl Compatibility Regular Expressions library
pp 1.2.0 Pretty-printing library
ppx_assert v0.16.0 Assert-like extension nodes that raise useful errors on failure
ppx_assert-windows v0.16.0 Assert-like extension nodes that raise useful errors on failure
ppx_base v0.16.0 Base set of ppx rewriters
ppx_base-windows v0.16.0 Base set of ppx rewriters
ppx_bench v0.16.0 Syntax extension for writing in-line benchmarks in ocaml code
ppx_bench-windows v0.16.0 Syntax extension for writing in-line benchmarks in ocaml code
ppx_bin_prot v0.16.0 Generation of bin_prot readers and writers from types
ppx_bin_prot-windows v0.16.0 Generation of bin_prot readers and writers from types
ppx_cold v0.16.0 Expands [@cold] into [@inline never][@specialise never][@local never]
ppx_cold-windows v0.16.0 Expands [@cold] into [@inline never][@specialise never][@local never]
ppx_compare v0.16.0 Generation of comparison functions from types
ppx_compare-windows v0.16.0 Generation of comparison functions from types
ppx_custom_printf v0.16.0 Printf-style format-strings for user-defined string conversion
ppx_custom_printf-windows v0.16.0 Printf-style format-strings for user-defined string conversion
ppx_derivers 1.2.1 Shared [@@deriving] plugin registry
ppx_derivers-windows 1.2.1 Shared [@@deriving] plugin registry
ppx_deriving 5.2.1 Type-driven code generation for OCaml
ppx_deriving-windows 5.2.1 Type-driven code generation for OCaml
ppx_disable_unused_warnings v0.16.0 Expands [@disable_unused_warnings] into [@warning "-20-26-32-33-34-35-36-37-38-39-60-66-67"]
ppx_disable_unused_warnings-windows v0.16.0 Expands [@disable_unused_warnings] into [@warning "-20-26-32-33-34-35-36-37-38-39-60-66-67"]
ppx_enumerate v0.16.0 Generate a list containing all values of a finite type
ppx_enumerate-windows v0.16.0 Generate a list containing all values of a finite type
ppx_expect v0.16.0 Cram like framework for OCaml
ppx_expect-windows v0.16.0 Cram like framework for OCaml
ppx_fields_conv v0.16.0 Generation of accessor and iteration functions for ocaml records
ppx_fields_conv-windows v0.16.0 Generation of accessor and iteration functions for ocaml records
ppx_fixed_literal v0.16.0 Simpler notation for fixed point literals
ppx_fixed_literal-windows v0.16.0 Simpler notation for fixed point literals
ppx_globalize v0.16.0 A ppx rewriter that generates functions to copy local values to the global heap
ppx_globalize-windows v0.16.0 A ppx rewriter that generates functions to copy local values to the global heap
ppx_hash v0.16.0 A ppx rewriter that generates hash functions from type expressions and definitions
ppx_hash-windows v0.16.0 A ppx rewriter that generates hash functions from type expressions and definitions
ppx_here v0.16.0 Expands [%here] into its location
ppx_here-windows v0.16.0 Expands [%here] into its location
ppx_ignore_instrumentation v0.16.0 Ignore Jane Street specific instrumentation extensions
ppx_ignore_instrumentation-windows v0.16.0 Ignore Jane Street specific instrumentation extensions
ppx_inline_test v0.16.0 Syntax extension for writing in-line tests in ocaml code
ppx_inline_test-windows v0.16.0 Syntax extension for writing in-line tests in ocaml code
ppx_jane v0.16.0 Standard Jane Street ppx rewriters
ppx_jane-windows v0.16.0 Standard Jane Street ppx rewriters
ppx_let v0.16.0 Monadic let-bindings
ppx_let-windows v0.16.0 Monadic let-bindings
ppx_log v0.16.0 Ppx_sexp_message-like extension nodes for lazily rendering log messages
ppx_log-windows v0.16.0 Ppx_sexp_message-like extension nodes for lazily rendering log messages
ppx_module_timer v0.16.0 Ppx rewriter that records top-level module startup times
ppx_module_timer-windows v0.16.0 Ppx rewriter that records top-level module startup times
ppx_optcomp v0.16.0 Optional compilation for OCaml
ppx_optcomp-windows v0.16.0 Optional compilation for OCaml
ppx_optional v0.16.0 Pattern matching on flat options
ppx_optional-windows v0.16.0 Pattern matching on flat options
ppx_pipebang v0.16.0 A ppx rewriter that inlines reverse application operators `|>` and `|!`
ppx_pipebang-windows v0.16.0 A ppx rewriter that inlines reverse application operators `|>` and `|!`
ppx_sexp_conv v0.16.0 [@@deriving] plugin to generate S-expression conversion functions
ppx_sexp_conv-windows v0.16.0 [@@deriving] plugin to generate S-expression conversion functions
ppx_sexp_message v0.16.0 A ppx rewriter for easy construction of s-expressions
ppx_sexp_message-windows v0.16.0 A ppx rewriter for easy construction of s-expressions
ppx_sexp_value v0.16.0 A ppx rewriter that simplifies building s-expressions from ocaml values
ppx_sexp_value-windows v0.16.0 A ppx rewriter that simplifies building s-expressions from ocaml values
ppx_stable v0.16.0 Stable types conversions generator
ppx_stable-windows v0.16.0 Stable types conversions generator
ppx_stable_witness v0.16.0 Ppx extension for deriving a witness that a type is intended to be stable. In this context, stable means that the serialization format will never change. This allows programs running at different versions of the code to safely communicate.
ppx_stable_witness-windows v0.16.0 Ppx extension for deriving a witness that a type is intended to be stable. In this context, stable means that the serialization format will never change. This allows programs running at different versions of the code to safely communicate.
ppx_string v0.16.0 Ppx extension for string interpolation
ppx_string-windows v0.16.0 Ppx extension for string interpolation
ppx_tydi v0.16.0 Let expressions, inferring pattern type from expression.
ppx_tydi-windows v0.16.0 Let expressions, inferring pattern type from expression.
ppx_typerep_conv v0.16.0 Generation of runtime types from type declarations
ppx_typerep_conv-windows v0.16.0 Generation of runtime types from type declarations
ppx_variants_conv v0.16.0 Generation of accessor and iteration functions for ocaml variant types
ppx_variants_conv-windows v0.16.0 Generation of accessor and iteration functions for ocaml variant types
ppx_yojson_conv_lib v0.16.0 Runtime lib for ppx_yojson_conv
ppxlib 0.30.0 Standard infrastructure for ppx rewriters
ppxlib-windows 0.29.1 Standard library for ppx rewriters
re 1.11.0 RE is a regular expression library for OCaml
re-windows 1.8.0 RE is a regular expression library for OCaml
react 1.2.2 Declarative events and signals for OCaml
result 1.5 Compatibility Result module
result-windows 1.5 Compatibility Result module
sedlex 3.2 An OCaml lexer generator for Unicode
seq base Compatibility package for OCaml's standard iterator type starting from 4.07.
seq-windows base Compatibility package for OCaml's standard iterator type starting from 4.07.
sexp_pretty v0.16.0 S-expression pretty-printer
sexplib v0.16.0 Library for serializing OCaml values to and from S-expressions
sexplib-windows v0.16.0 Library for serializing OCaml values to and from S-expressions
sexplib0 v0.16.0 Library containing the definition of S-expressions and some base converters
sexplib0-windows v0.16.0 Library containing the definition of S-expressions and some base converters
spawn v0.15.1 Spawning sub-processes
splittable_random v0.16.0 PRNG that can be split into independent streams
splittable_random-windows v0.16.0 PRNG that can be split into independent streams
stdio v0.16.0 Standard IO library for OCaml
stdio-windows v0.16.0 Standard IO library for OCaml
stdlib-shims 0.3.0 Backport some of the new stdlib features to older compiler
stdlib-shims-windows 0.3.0 Backport some of the new stdlib features to older compiler
stdune 3.10.0 Dune's unstable standard library
time_now v0.16.0 Reports the current time
time_now-windows v0.16.0 Reports the current time
timezone v0.16.0 Time-zone handling
topkg 1.0.7 The transitory OCaml software packager
trie 1.0.0 Strict impure trie tree
typerep v0.16.0 Typerep is a library for runtime types
typerep-windows v0.16.0 Typerep is a library for runtime types
tyxml 4.5.0 A library for building correct HTML and SVG documents
uchar 0.0.2 Compatibility library for OCaml's Uchar module
utop 2.13.1 Universal toplevel for OCaml
uucp 15.1.0 Unicode character properties for OCaml
uunf 15.1.0 Unicode text normalization for OCaml
uuseg 15.1.0 Unicode text segmentation for OCaml
uutf 1.0.3 Non-blocking streaming Unicode codec for OCaml
variantslib v0.16.0 Part of Jane Street's Core library
variantslib-windows v0.16.0 Part of Jane Street's Core library
xdg 3.10.0 XDG Base Directory Specification
yojson 2.1.0 Yojson is an optimized parsing and printing library for the JSON format
yojson-windows 2.1.0 Yojson is an optimized parsing and printing library for the JSON format
zed 3.2.3 Abstract engine for text edition in OCaml
@TyOverby, it seems that the code structure generated by ppx_expect defeat the logic in place inside ppx_inline_test to allow deadcode of tests. In particular, the call to Ppx_inline_test_lib.test is inside a functor Test_block.Make. It prevents Ppx_inline_test_lib.test to be inlined and prevent the arguments to Ppx_inline_test_lib.test to be collected.
@hhugo thanks for the digging! Unfortunately updating the ppx dependencies is harder for us than JSOO, so I may be stuck with the broken behavior for a while, but I'll open a ticket on their repo
xref the ppx_expect issue to here. I can provide some guidance on how to fix the ppx_expect code.
For the record, you need at least two expect tests to reproduce the issue otherwise, inlining of the functor happens and the code is properly DCE
For the record, you need at least two expect tests to reproduce the issue
Ah, that explains some of the difficulties reproducing.
Thanks for getting a fix in before I even got around to writing a new issue!
An alternative version of https://github.com/janestreet/ppx_expect/pull/56 was merged and should be present in the next version of ppx_expect (v0.18.0 probably)