mosml icon indicating copy to clipboard operation
mosml copied to clipboard
verified publisher icon kfl

Add release automation, Docker support, Windows cross-compilation, and enhanced REPL experience

Open Francesco146 opened this issue 2 months ago • 12 comments

This PR introduces significant infrastructure improvements across build systems, CI/CD pipelines, containerization, and user experience. The changes enable automated releases, reproducible builds, cross-platform support, and a modernized interactive REPL.

Release Automation & CI/CD

  • Comprehensive GitHub Actions workflow (buildAndTest.yml):
    • Multi-platform build matrix: Linux x86_64, macOS x86_64, macOS arm64
    • Automated artifact creation for each platform (mosml-<platform>-<arch>.tar.gz)
    • Windows cross-compilation via Docker (mingw32)
    • Windows installer generation using Inno Setup
    • Automatic GitHub Release creation on version tags (v*.*.*)
  • Docker build workflow (docker.yml):
    • Automated image builds for GitHub Container Registry and Docker Hub
    • Multi-registry support with proper tagging (latest + semver)
    • Build caching for improved CI performance

Docker Support

  • Dockerfile:
    • Multi-stage build for minimal image size
    • Ubuntu 22.04 base with runtime dependencies only
  • Docker Compose configuration for local development
  • Windows cross-build Dockerfile (Dockerfile.w32):
    • Ubuntu 14.04 i386 base for legacy MinGW compatibility
    • Automated Windows binary packaging
    • Cross-compilation script with extensive validation

Windows Support

  • Cross-compilation infrastructure:
    • build-win.sh script with comprehensive options and error handling
    • Support for custom source/output directories and parallel builds
    • Dry-run mode for debugging
    • Proper DESTDIR handling for staged installations
  • Windows installer improvements:
    • Italian language support added

Enhanced REPL Experience

  • Intelligent rlwrap integration in mosml launcher:
    • Automatic detection and usage when available
    • XDG Base Directory compliance for history storage
    • Fallback to $HOME/.local/share/mosml/history when XDG_DATA_HOME not set
    • Opt-out via DISABLE_RLWRAP=1 environment variable
    • Features: command history, arrow key editing, line navigation
  • Cross-platform history support including macOS, Linux, and Windows (MSYS/Git Bash)

Build System Improvements

  • Robust GMP detection:
    • Prefer pkg-config when available
    • Intelligent fallback to common system paths (Homebrew, apt, system libs)
    • Support for explicit GMPINCDIR and GMPLIBDIR overrides
    • Clear warning messages when GMP not found
  • Fixed compiler warnings and errors:
    • Standardized main() signatures across auto-aux helpers
    • C99/C11 compliance improvements
    • Header includes added where missing
  • Improved Makefile robustness:
    • Error-tolerant clean targets (continue on missing directories)
    • Support for install_w32 target in lex
    • Proper DESTDIR and PREFIX handling throughout

Bug Fixes

  • Word64 module fixes:
    • Proper handling of overflow in toInt
  • Runtime system fixes:
    • Fixed SA_NOMASK → SA_NODEFER for POSIX compliance
    • Missing includes added (stdlib.h, fail.h, memory.h)

Repository Hygiene

  • Comprehensive .gitignore:
    • Build artifacts, temporary files, OS-specific files
    • Docker shared volumes, staging directories
    • Generated from gitignore.io templates (Linux, macOS, Windows, C, StandardML, LaTeX)
  • .dockerignore for efficient Docker builds
  • Version bump to 2.10.5

Documentation Updates

  • Updated installation instructions for Windows (install.txt.w32)
  • Improved Inno Setup README
  • Release notes template with platform-specific installation guides

Testing

All changes have been tested across:

  • [x] Ubuntu (latest)
  • [X] macOS Intel (x86_64)
  • [X] macOS Apple Silicon (arm64)
  • [X] Windows (cross-compiled via Docker)
  • [X] Docker containers (REPL with rlwrap, history persistence)

Breaking Changes

All changes are backward compatible. The rlwrap integration is opt-in by availability and can be disabled. The CI process for pushing mosml images to Docker HUB require two repository secrets:

Francesco146 avatar Nov 05 '25 14:11 Francesco146

the only thing that i can think of for a working mosml implementation on Windows is my port of mosml to Docker:

https://github.com/Francesco146/mosml-docker

Francesco146 avatar Nov 05 '25 14:11 Francesco146

update: https://github.com/kfl/mosml/wiki/Notes-about-how-to-build-Moscow-ML-installers#for-windows this seems to work, it still uses the deprecated i586-mingw32msvc-gcc, but a simple change in the Makefile worked and produced:

camlrt.dll  camlrunm.exe  mosml.exe  mosmlc.exe  mosmlyac.exe

i'll push the changes here

Francesco146 avatar Nov 06 '25 09:11 Francesco146

Thanks for this great work. I am rusty on Github and on the technical details, and @kfl is now department head and probably way too busy to look into it. Hence I am tempted to just accept this pull request (if that's what's needed).

sestoft avatar Nov 11 '25 15:11 sestoft

I'm not too busy for Moscow ML work 😉.

I'm planning to review the PR within the next two weeks (at worst over the end of year holidays).

kfl avatar Nov 11 '25 15:11 kfl

hello! I recently figured out how to compile a working exe in the CI. I'll see if I can push it one of these days. Thank you very much!

edit: after this PR gets eventually merged, maybe we could take a look at the documentation and build instructions. I'll make another PR in order to not stress the review process here :)

Francesco146 avatar Nov 11 '25 15:11 Francesco146

@Francesco146 could I get you you to change the base for this PR from the master branch to the devel branch? That would make the review process much easier for me. Thanks in advance.

kfl avatar Nov 12 '25 08:11 kfl

sure thing, i can work it out on Friday. I'll need to test:

  • [x] if any of the unreleased commits in the devel branch break some of my commits (related to #78)
  • [x] test the automated windows build (related to #49)
  • [x] confirm that the issues this PR can potentially close: #75, #71, #68, #57, #48, #44, #37, #32, #77, #72, #56

in a later PR i could:

  • [ ] document build instructions and install instructions in all environments (related to #33)
  • [ ] simplify all the setup needed. might be cool adding a .devcontainer

Francesco146 avatar Nov 12 '25 09:11 Francesco146

okay, update: i've published the automatic build system for Windows. it all uses InnoSetup, so that's cool. i changed the branch to devel, but as i suspected, it introduced some regression:

File “Word64.sml”, line 171, characters 25-44:
!         if w > fromWord_ 0wx3FFFFFFFFFFFFFFF then raise Overflow
!                          ^^^^^^^^^^^^^^^^^^^
make[1]: *** [Word64.uo] Error 2
make[1]: *** Waiting for unfinished jobs....
! Lexical error: word constant is too large.
make[1]: Leaving directory `/mosml/src/mosmllib'
make: *** [world] Error 2

i'll take a look at these right now

Francesco146 avatar Nov 14 '25 12:11 Francesco146

i found an issue in the Word64 conversion layer:

  1. the *X conversion functions in Word64.sml (toIntX, toLargeIntX, toLargeWordX, toLargeX) were incorrectly bound to the zero-extending primitive boxed_uint64_toword. these functions are intended to perform sign extension, so this mismatch could lead to incorrect results for values where bit 63 is set.

i've updated (https://github.com/kfl/mosml/pull/80/commits/84b12501530dad80988b4b823ef08bd38a1a2b29) the SML prim_val binding to reference a new primitive boxed_uint64_toword_signed and added the corresponding implementation in boxed_num.c, which:

  • reads the 64-bit boxed value
  • sign-extends bit 63 into the host integer/word
  • returns the correctly boxed result
  • follows the same runtime registration and error handling as the existing zero-extending primitive

Francesco146 avatar Nov 14 '25 13:11 Francesco146

also, in the Word64 shift functions:

  1. the shift functions in Word64.sml (<<, >>, ~>>) were using toInt k to check shift bounds. for very large k, toInt k can raise Overflow, making the functions unsafe for large shift counts.

i've updated (https://github.com/kfl/mosml/pull/80/commits/82e40e37400258f448e7ca4464b386c82f14b0a7) the functions to use word-level comparisons instead of converting k to an ML int. the changes:

  • compare k directly to fromInt 64 and fromInt 0 using existing word comparison primitives or less_/equal_ helpers
  • ensure shifts with k >= 64 or k < 0 return ZERO (or the sign-extended result for ~>>) safely

Francesco146 avatar Nov 14 '25 13:11 Francesco146

hmm, i think it's better to address these corrections in another PR. i believe Int64 and Word64 need some changes in terms of functionality, but also bug fixing and documentation in the signatures, should i revert this changes for now?

edit: note that https://github.com/kfl/mosml/pull/80/commits/15d25fffbca7e90d66992709a55fcf770ff4ac70 is needed for the CI/CD to succeed

Francesco146 avatar Nov 14 '25 14:11 Francesco146

done, sorry for the back and forth, i found a few edge cases that my additions didn't handle. everything should be fine now and ready for review. i think it's best to revert the two commits, but i'll wait to hear what you think :)

Francesco146 avatar Nov 14 '25 15:11 Francesco146