odin: add vendor folder

[joakin]

Odin's compiler includes both core and vendor folders. Both should be available where the compiler is installed.

https://pkg.odin-lang.org/vendor/

• [x] Have you followed the guidelines for contributing?
• [x] Have you ensured that your commits follow the commit style guide?
• [x] Have you checked that there aren't other open pull requests for the same formula update/change?
• [ ] Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
• [ ] Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
• [ ] Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

---

[joakin]

Hm this is failing

    * Unexpected universal binaries were found.
      The offending files are:
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos/libraylib.4.5.0.dylib
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos/libraylib.450.dylib
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos/libraylib.dylib
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos-arm64/libraylib.4.5.0.dylib
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos-arm64/libraylib.450.dylib
        /opt/homebrew/Cellar/odin/2023-12/libexec/vendor/raylib/macos-arm64/libraylib.dylib

I think those are files the Odin compiler will use when compiling odin programs, not dependencies for building the odin compiler itself.

Is there something we can do?

[SMillerDev]

Is there something we can do?

We need to convince the compiler to make optimised, not universal binaries

[cho-m]

Those look like pre-built libraries (https://github.com/odin-lang/Odin/tree/master/vendor/raylib/macos) which we do not allow in homebrew-core. You will need to build them from source if you want them to be included with the formula.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[laytan]

If vendor can't be added to the homebrew installation, can you please remove Odin from homebrew, we get regular questions about homebrew installations missing vendor from beginners.

Is there an easy way to include vendor but delete the offending files/binaries in the formula, that might be a good middle ground?

[SMillerDev]

Is there an easy way to include vendor but delete the offending files/binaries in the formula, that might be a good middle ground?

If there is a way to build vendor files?

[cho-m]

If only raylib vendor path is bundling pre-built libraries, one option is to delete existing files and try replacing them with symlinks to Homebrew's raylib.

If there are other pre-builts, then they would need similar changes.

[laytan]

So pre-compiled binaries are mainly bundled for Windows, on macos only stb, glfw and raylib are pre-compiled and on Linux it is lua and raylib.

Deleting all these binaries should be easy as find vendor -type f \( -name "*.lib" -o -name "*.dll" -o -name "*.a" -o -name "*.dylib" -o -name "*.so.*" -o -name "*.so" \) -delete.

Symlinking sounds good, although I don't think stb is on brew, and we would need them as dependencies I guess, when they aren't actually dependencies?

[joakin]

To chime in, I’ve gotten value from Odin in homebrew even without vendor.

I clone the source repo and symlink vendor in the homebrew install to the folder in my clone.

This way I don’t have to manage downloading the releases manually.

I known someone is going to tell me that I should build the compiler from source but I swear for the life of me it keeps erroring out for different reasons and I still haven’t been able to.

On Tue, 23 Jan 2024 at 17:54, Laytan @.***> wrote:

So pre-compiled binaries are mainly bundled for Windows, on macos only stb and raylib are pre-compiled and on Linux it is lua and raylib.

Deleting all these binaries should be easy as find vendor -type f ( -name ".lib" -o -name ".dll" -o -name ".a" -o -name ".dylib" -o -name ".so." -o -name "*.so" ) -delete.

Symlinking sounds good, although I don't think stb is on brew, and we would need them as dependencies I guess, when they aren't actually dependencies?

— Reply to this email directly, view it on GitHub https://github.com/Homebrew/homebrew-core/pull/158496#issuecomment-1906500179, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFATL3QQCYNPC7DZB3N5CTYP7TL3AVCNFSM6AAAAABBGPN5COVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBWGUYDAMJXHE . You are receiving this because you authored the thread.Message ID: @.***>

[laytan]

Looked into it a bit, and with this change I just PR'ed to Odin: https://github.com/odin-lang/Odin/pull/3134, all the tests/checks I run locally against the formula with vendor added pass.

So if this PR is merged soon and a new release comes out early February this can be merged.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[joakin]

@laytan 's PR has been merged, March's release should have the changes

[SMillerDev]

That doesn't fix the problem though, the problem is that Homebrew does not allow pre-compiled binaries. The universal binary check just happens to be the one that caught it.

[laytan]

Why not? These restrictions seem so arbitrary for a package manager. Additionally, all checks pass with these changes.

You want me to pull the repo, remove all binaries, compile them again? If so, can you point me to a formula that compiles multiple binaries for reference?

[SMillerDev]

Why not? These restrictions seem so arbitrary for a package manager. Additionally, all checks pass with these changes.

I'm pretty sure Fedora, Debian or Arch don't want you to ship precompiled binaries either. Additionally, it's pretty well documented:

• https://docs.brew.sh/Acceptable-Formulae#stuff-that-requires-vendored-versions-of-homebrew-formulae
• https://docs.brew.sh/Acceptable-Formulae#we-dont-like-binary-formulae

[laytan]

We are on arch and Debian package managers without any changes, I think they just want the compiler itself to not be pre-compiled.

Anyway, I am willing to do the remove everything, recompile everything if you really want to because we get many questions why vendor isn't shipped with brew.

But, are there formulas I can reference that do something similar?

[SMillerDev]

Aur will ship anything people contribute so that doesn't really count (https://repology.org/project/odin-lang/versions)

---

But, are there formulas I can reference that do something similar?

I'm not sure what "something similar" is, but it seems the software you vendor is already in brew so you could just use it as a shared library and you wouldn't need to ship anything precompiled

[laytan]

Can't do that:

1. version on brew might be ahead or before what we need, I don't think you can specify a version number for the packages in brew? I tried to add a dependency on a specific llvm version before and that didn't work
2. we might need some different compile options (although I don't think that is the case)
3. not all of the vendor libraries are on brew (stb for example)
4. we will not change Odin to shared libraries only, that is just a bad idea for multiple reasons (we actually give a choice between static or dynamic for most)

I guess we can just rm all binaries, we would still get many questions about this and I prefer not to though

[SMillerDev]

version on brew might be ahead or before what we need, I don't think you can specify a version number for the packages in brew? I tried to add a dependency on a specific llvm version before and that didn't work

No, software in brew should work with the latest available dependencies.

not all of the vendor libraries are on brew (stb for example)

In that case they can be compiled as part of the build of odin

we will not change Odin to shared libraries only, that is just a bad idea for multiple reasons (we actually give a choice between static or dynamic for most)

It seems the best way is then to use the dynamic ones from brew and delete or compile them if those are not available.

[laytan]

Right, so brew and Odin aren't going to fully work together in that case, whatever we do here (besides just shipping them, or building all of these libraries from source in the formula) will get unhappy users and we will have to support them.

We don't even advertise being available via brew because of these issues but people just expect it to work.

[p-linnane]

As @SMillerDev & @cho-m have said, we do not ship pre-compiled binaries. We also do not remove software from homebrew-core by request. My best advice is to put a note in your install guide or FAQ stating that extra effort is required for users installing odin through brew, much like how @joakin described their use case.

[joakin]

Would it be ok to compile the libraries as part of odin's build even if the libraries are already in brew?

The reason is the odin compiler ships bindings to the specific vendor libraries bundled with the compiler, so it can't support all other arbitrary (later or earlier) versions of a library that may be installed with brew, because it needs to have Odin bindings.

If as part of building Odin, we build the binaries for odin's vendored raylib, would that work? Or is that against some other policy?

[laytan]

@p-linnane putting a note would probably not help, people see it on brew, install it, and don't read up on the installation page. Then it doesn't work, and they complain to us.

From your own docs btw (https://docs.brew.sh/Acceptable-Formulae#stuff-that-requires-vendored-versions-of-homebrew-formulae): "Increasingly, though: this can be (too) hard. Homebrew’s primary mission is to be useful rather than ideologically pure. If we cannot package something without using vendored upstream versions: so be it; better to have packaged software in Homebrew than not."

"We also do not remove software from homebrew-core by request.", so you would rather ship a broken package than remove it?

[p-linnane]

If as part of building Odin, we build the binaries for odin's vendored raylib, would that work? Or is that against some other policy?

This is fine with me. Thoughts @SMillerDev & @cho-m?

From your own docs btw (docs.brew.sh/Acceptable-Formulae#stuff-that-requires-vendored-versions-of-homebrew-formulae): "Increasingly, though: this can be (too) hard. Homebrew’s primary mission is to be useful rather than ideologically pure. If we cannot package something without using vendored upstream versions: so be it; better to have packaged software in Homebrew than not."

This generally applies to very popular software, with tens of thousands of installs per month, that one would be shocked to not see in a package manager. The install statistics for odin do paint it as that type of package:

==> Analytics
install: 226 (30 days), 541 (90 days), 1,235 (365 days)
install-on-request: 225 (30 days), 538 (90 days), 1,228 (365 days)
build-error: 3 (30 days)

"We also do not remove software from homebrew-core by request.", so you would rather ship a broken package than remove it?

I'm not going to argue the nuances of running a package manager with millions of users with you, but I will cautiously answer "yes" to this. To quote @joakin in this same PR:

To chime in, I’ve gotten value from Odin in homebrew even without vendor.

That's enough for me to not remove it.

odin is distributed under a BSD-3-Clause license, so there is no guarantee that it should work exactly the way upstream wants it to. The way we build and ship the package is not bound to any kind of upstream desire, including for removal. I'd like point you to the following article to better understand our positions, especially regarding situations like this: Open Source Maintainers Owe You Nothing.

If the other maintainers involved here agree, I'd welcome a pivot to this PR to build the necessary binaries within the formula, barring some kind of insane amount of compile time or the addition of a ton of build dependencies.

[laytan]

So to be clear, you are ok with the formula doing:

1. Pull the folder
2. Delete all binary files ala find vendor -type f \( -name "*.lib" -o -name "*.dll" -o -name "*.a" -o -name "*.dylib" -o -name "*.so.*" -o -name "*.so" \) -delete
3. Manually compile stb, glfw, and raylib (and lua for Linux) into the places we just deleted

And of course you don't owe me or Odin anything, I was just interested in the thought processes here, if I was the maintainer I think I would rather have a package not included than a partially broken package. But like I said, you do you, I am coming from the side of the Odin maintainers and community, I guess you can imagine the questions about it not working fully get annoying (for lack of a better word) after a while.

[karl-zylinski]

Hi, if we cannot get the vendor stuff to work properly with Odin on brew, then I'd argue to just remove Odin from brew.

People install stuff from brew and assume it works if it's on there.

Having Odin on there it makes it seem like a viable package to use, but it's actually broken and from the discussion above, not really possible to fix.

I get the whole "you don't owe Odin community anything" stance, but remember that this is a big platform, so projects will have an interest in not being misrepresented on there.

[p-linnane]

I get the whole "you don't owe Odin community anything" stance, but remember that this is a big platform, so projects will have an interest in not being misrepresented on there.

I'd like to clarify that it's less of "we don't owe the Odin community anything", and more of "we make no guarantees this works the same way upstream would like". We have an entire part of our DSL called caveats which can be used in situations like this where:

In case there are specific issues with the Homebrew packaging (compared to how the software is installed from other sources) a caveats block can be added to the formula to warn users.

@joakin stated in an earlier comment:

To chime in, I’ve gotten value from Odin in homebrew even without vendor.

I clone the source repo and symlink vendor in the homebrew install to the folder in my clone.

This way I don’t have to manage downloading the releases manually.

That seems like a good thing to add to caveats and then we can all move on here.

[laytan]

@MikeMcQuaid Which position is that? Adding a caveat or building the needed libraries from source?

[MikeMcQuaid]

@laytan Either.

[laytan]

Alright, I would like to try the build from source approach then as I don't like a caveat if it can be avoided. @joakin do you want to take a stab at this or should I?