Resolution for Opam Build Issues in Opendal's OCaml Bindings during Release

[Ranxy]

During the release process, the OCaml bindings for Opendal encountered some issues, one of which is as follows. Opam, OCaml's package manager, builds from the project's root directory. This requires the presence of files such as opendal.opam and dune-project in the project's root directory. However, this is not suitable for the Opendal project because these files are specific to just one binding used by Opendal.

A feasible solution currently is to replace opendal = { path = "../../core" } with a specific version in the Cargo.toml file located in binding/ocaml. This allows the binding to be built independently. Consequently, during the release, we can package this binding separately as apache-opendal-$release_version-binding-ocaml-src.tar.gz, thus circumventing issues with Opam builds.

However, this approach introduces challenges, such as coordinating development between the binding and the core during ongoing development. I would like to hear everyone's thoughts on how to address these challenges.

[Xuanwo]

A feasible solution currently is to replace opendal = { path = "../../core" } with a specific version in the Cargo.toml file located in binding/ocaml.

I'm ok with pinning opendal to specific version with specific version.

I also believe that the OCaml binding should have its own versioning, independent of OpenDAL's. Although this implies we would need to release OCaml binding separately.

[Xuanwo]

I feel like we can release ocaml binding separately. For example:

1. Release the ocaml binding source first.
2. Submit PR to ocaml registry after ASF release approved.

[Ranxy]

Good idea, I think it's doable

[Xuanwo]

Hey @Ranxy, do you know any potential users for our OCaml binding?

[Xuanwo]

Hi, @Ranxy, will opam work if we provide a tar that contains both core and bindings/ocaml? Can opam works inside bindings/ocaml with path = "../../core"?

[Ranxy]

Hi, @Ranxy, will opam work if we provide a tar that contains both core and bindings/ocaml? Can opam works inside bindings/ocaml with path = "../../core"?

As far as I know this doesn't seem to work unless we add dune-project and opendal.opam to the top level of the directory. Maybe there are other solutions we can ask in the OCaml community

[Xuanwo]

I plan to release file like apache-opendal-bindings-ocaml-0.0.0+core.0.44.2-src.tar.gz with file tree:

├── bindings
│   └── ocaml
└── core
    ├── benches
    ├── Cargo.lock
    ├── Cargo.toml
    ├── CHANGELOG.md -> apache-opendal-0.45.0-src/../CHANGELOG.md
    ├── CONTRIBUTING.md
    ├── DEPENDENCIES.md
    ├── DEPENDENCIES.rust.tsv
    ├── edge
    ├── fuzz
    ├── README.md
    ├── src
    └── tests

9 directories, 7 files

Maybe we can perform some actions inside opam to make it work?

Like libwasmer:

build: [
	["mkdir" ".cargo"]
	["mv" "config.toml" ".cargo"]
	["sh" "-c" "gunzip vendor.tar.gz && tar xf vendor.tar"]
	[make "build-capi"]
	[make "package-capi"]
	["cp" "package/lib/libwasmer.dylib" "package/lib/libwasmer.so"] {os = "macos"}
]

[Ranxy]

Wasmer seems to be separating the binding from the core, and only building the capi in the core step, while leaving the binding part to be done in an additional project. I'm not sure if this approach will affect opam's ability to build ocaml projects once we've merged them together. Maybe we can use ocaml-ci to verify feasibility