Package-manager-friendly releases and/or sub-projects

[corporateshark]

Package managers (like this one https://github.com/Microsoft/vcpkg) require a link to a specific commit/tag/branch to work properly. STB does not contain any releases making integration into such package managers troublesome.

It would be cool if the first release version can be added.

[BSVino]

Not trying to speak for stb here, but I think you're missing the point of single file libraries if you add them to a package manager. It's a single file primarily because that makes it easy to distribute - wget will do the trick. Most people I've seen copy them into their project and commit them to their source control, and then update them occasionally when necessary. That said, if you want to add it anyway, stb never commits anything to master that he knows contains bugs, every commit is a version suitable for a package manager.

[corporateshark]

Of course. However, it can be useful to have multiple choices and a package manager can be just yet another choice to install popular library.

[rygorous]

The main stb repository is not a good one to link to for individual library dependencies (and definitely not the right one to use for a package manager) since it contains all stb libs and it's definitely possible that the same piece of code wants to have say an older version of stb_image and a newer version of stb_vorbis simultaneously.

Since there's also frequent requests to put stb_* under some explicit permissive open source license (which one is suggested varies), I think there's a good solution to both of these:

1. Tag the main stb repository with new library versions whenever they're released (e.g. tag commit 'aa1041bae807' as 'stbcc-0.94')
2. Otherwise, keep the main stb repository as-is (everything public domain, no license blurb, do what you want)
3. Also have separate repositories for the main libs that each contain a COPYING file, license blurb on top of the main file (license TBD), and tags for released versions of just that library, and are otherwise suitable for integration into open-source projects or automated slurping from a package manager.

The idea is that this can be fully automated. Step 1 can be accomplished by a small script that just tags commits with changes to README.md with the corresponding version (which should be sufficient to tag all official releases in the past as well). Step 3 can likewise be automated by a script that uses these same README.md updates to figure out when a new version of a library was released, and just creates one merged commit to the derived repository per version.

I can try and set this up if there's interest.

[nothings]

Main potential issues I see:

1. How much work is it to add a new library?
2. Any use of a scripting language not installed on my machine by default adds an extra step to what I need to do to set up a new machine to be github-ready (currently only requirement is 'install windows git')

Also I just don't know how widespread the demand for this is. (This is the second request I've seen, although I wasn't able to find the first one searching in Issues/PRs.)

[rygorous]

For question number 1, the tagging of vers on the stb repo shouldn't require any work at all outside of the "add to README.md" you do already. The tag names can be derived from just the library file name and version number. As for generating single-lib repos, that part would need some more set up (at the very least, you need to create those repos on Github, not sure how much setup the other plumbing would need, hopefully not much).

As for problem number 2, Windows git has commands implemented in shell scripts, Perl and TCL and thus ships with all three. Perl should work for this.

[nothings]

Right, It may use them, but it doesn't appear to put them on my path, so it must be doing shenanigans to use them. So you'd have to replicate those shenanigans (which I don't know what they are so I don't know how hard they are to do, but even just investigating to find out is more work than I care to do).

[technosaurus]

As a proof of concept, I started separating individual libs into their own projects at github.com/technosaurus/stb_* I have a few left to go, notably stb.h, because a) it may interfere with the main project and b) there was discussion about splitting it and renaming it stb_lib.h Once I get the rest of the libs forked with history, I plan to fork the main project and replace each lib with corresponding submodules. This will allow for easily adding new projects by simply adding any repo as a submodule. That also allows for separating new, experimental libs from the main project (like the ones in obbg)

[onitake]

While splitting off the individual sub-libraries may certainly be a good idea, it doesn't solve the issue that the whole stb library is still unversioned.

I'm a Debian package maintainer, and one of the projects I work on has recently started using stb_image. While it's possible to maintain a copy of stb_image with the source code or download the HEAD of stb at build time, both approaches are problematic. The first first is generally frowned upon, because it hides the dependency's version used at build time and possible bugs or security issues from the package manager. The second approach is basically impossible, because it imposes build environment requirements that are not acceptable as by Debian policy (internet access, dependency on external websites, stability of API).

Now, it's possible to prepare versioned packages from git-only releases, but it's a major pain and very error-prone, as it would require prepending each release with an arbitrary version counter to make the package version incremental. And it would make it very difficult to track when upstream releases a new package version.

I'd very much like to prepare -dev packages from libstb and introduce them into Debian, but it will be very difficult to do so without proper release tags. If anything, even a single version number would suffice, but it should be incremental and it should always point to an exact release.

I understand that packaging is only of minor importance to you and that you would like to finish separating stb into separate components first, but it would make my work easier if even stb as whole bundle was versioned.

Thank you!

[technosaurus]

@onitake Does this help? https://github.com/technosaurus/stb_image/releases/tag/v2.19

If so, I can tag a release for the others.

[onitake]

That's awesome! Thank you very much!

[onitake]

On second thought... Is that version "official" or will the scheme be changed later? I can go ahead with your fork, but if the repository is going to be moved around a lot, I'd rather hold off for a while.

[nothings]

Can somebody link me to a github help page for tagging releases so I know what's even being talked about here?

[onitake]

The concept is that you give an individual commit a "name", usually a release version. You can also do this retrospectively.

The basics of tagging: https://git-scm.com/book/en/v2/Git-Basics-Tagging Github doesn't really have much to say about it: https://help.github.com/articles/working-with-tags/ This Stackoverflow question is also quite concise: https://stackoverflow.com/questions/18216991/create-a-tag-in-github-repository

There's also the release feature, which is a bit more elaborate and allows adding release notes and other files like pre-built binaries to a tag: https://help.github.com/articles/about-releases/ But I think simple tags are fine. There's one cool feature about tags on Github: It will allow people to download a zip or tar.gz file containing the source code directly, without any further action from the developer. This is also possible for individual commits, but it's more tedious and requires one to search for the specific commit of a release version.

One example: If you wanted to "release" this commit as version 2.19, you would do:

git checkout fa2a1d9b3be0733b222fc14a7428e81ce940c2a7
git tag v2.19

Don't forget to push tags you created locally to Github:

git push --tags

[nothings]

Thanks. I could try to make this part of my release workflow, but I guarantee I would forget at least one step half the time (which is why I would prefer to do it on github where it would presmuably be a single step).

I remain unconvinced that this would be particularly superior to incorporating the header file into the source tree, no matter how much that is frowned on by some software-releasing organization, but I also recognize that the scenarios in which stb_image is now used exceed the original intentions for the library, so the security argument may be valid regardless of my claims of being unconvinced.

Note that any kind of semantic versioning of the full repository is impossible, as e.g. one library could be changed in a back-incompatible way while another library just receives a security fix, so I'm not really clear how this would ever be package-manager-compatible in the long run.

[onitake]

I don't think versions have to be tied together. Having a separate version for each header file would just mean they'll have to be packaged separately. If they all carry the same version, that's just as fine.

Since you're already working on separating the different parts of the library, it would probably not be very meaningful to start bundling stb as a whole. Should I start with @technosaurus 's stb_image, or are you going to move the fork to your own Github account soon?

[technosaurus]

I am more than happy to reassign ownership of my separated forks back to Sean if he wants them. I am maintaining them for my own experimental purposes and only update sporadically with upstream because I haven't figured out the git-fu to do it automatically.

That said, I do plan to keep them updated if Sean keeps his tree combined for my own minimalist reasons. Currently (offline) I have stb with submodules for the headers I split so far ... pending the possible stb.h split. Once that issue is settled the main stb repo will just be tests, examples and docs with all header code in the submodules' repos (and issues, wikis, etc... too)

Re: "official" version number - I just used the version number listed in the sources. You can probably just use a script to grep for them if you are packaging - most of the headers have a version number listed and it's usually updated in the readme.md

[nothings]

Two points, one regarding @technosaurus' submodule fork, and one regarding @onitake's versioning issues.

I don't really have much interest in the separate-libs fork because, as I said in 2016, I'm not comfortable with the work that would be required to have separate repos and submodules. I'm ok with something where the stb repository remains primary and the submodules are generated, which is what we were discussing then, but "this will allow for easily adding new projects by simply adding any repo as a submodule" is a non-starter for me. But I think that's ok because maybe bullet point 2 is good enough.

If I understand correctly, it is feasible for me to tag one version of the stb repo with, say, "stb_image/2.04" and "stb_image_write/0.99" and then a different version of the repo with "stb_image_write/1.00" and then a project can be written to require stb_image.h 2.04 and stb_image_write 1.00, and package managers (or the person packaging it? I have no idea how package management works and the details don't matter to me as long as the answer is yes) can deal with accessing two versions of the repo, extracting stb_image.h from one and stb_image_write.h from another, and putting them in the right place for that project w/o the project needing to have multiple stb directories in its tree?

If this understanding is correct, I just need to tag each release with the new version of each library's version number (if there is one), and that provides enough info to make package management work?

And in that case, I can almost (but not quite) automate this with my existing automation for generating the readme.md which contains the version numbers, and just have that add git tags, and then all I'd have to do at the end is 'git push --tags'. This seems VERY tractable for me. [This was never on the table before because I never understood the thing in the first paragraph before today--assuming that's actually true.]

(Unfortunately, the naive way of doing this with my existing automation would try to keep re-tagging the repo with the same version of a lib if that lib didn't change, which isn't what we want, although maybe that would actually turn out to work, if git rejects trying to reuse a tag that already exists.)

[onitake]

As for 2., I don't think that makes a lot of sense. If you tag individual header releases in the same repo with different versions, that will make things very confusing and lead to unnecessarily bloated source packages or more maintenance work (for separating the headers again).

If you keep everything in one repository (variation 1.), I recommend having common releases where a new version (like: v2.19) combines all the work that was done on all headers. If you decide separating the headers into multiple projects (variation 2.), it's easiest to just version each individually, but a common version could work too.

Both are valid possibilities. You could even do the separation work and still keep everything in one repository (variation 3.). In that case, it might make sense to structure the project a bit differently, for example by putting stb_image into /headers/image/stb_image.h. Or just keep them as they are.

A few examples of how other projects do packaging: Boost is separated into multiple independent projects, but they are usually released with a common version number. ImageMagick is one huge library/tool behemoth with loaders and writers for many different image formats, built from the same source tree. At Debian, we keep the upstream structure intact, but if a project mixes tools, development files and supplementary data together, they are normally separated into their own packages.

If we take boost, Debian packages it like this:

libboost-date-time1.67-dev
libboost-date-time1.67.0
libboost-date-time1.67.0-dbgsym
libboost-exception1.67-dev
libboost-random1.67-dev
libboost-random1.67.0
libboost-random1.67.0-dbgsym
...many others...
libboost1.67-all-dev
libboost1.67-dev
libboost1.67-doc
libboost1.67-tools-dev
libboost1.67-tools-dev-dbgsym

The -dev packages contain header files and possibly linker support files. Stuff that is only relevant for developers or package builders. The packages without a suffix contain just the runtime library (DLL) and are used as dependencies for applications that use them. The -dbgsym packages contain the stripped out debugging symbols for those who need them. That keeps the runtime library small. Then there's also a common -doc package, containing all the documentation bundled with the source code, an -all-dev package that just installs all the other -dev packages and a -tools-dev package that contains some scripts and tools probably only relevant to boost developers. As you can see, one of the boost libraries (boost::exception) only contains header files, so there is no runtime library package. This is what I would create for stb.

[kloczek]

So why stb has not been released qith regular version which coud be use by packagers?

[nothings]

Didn't I just tell you I wouldn't answer this specific question because it's already been addressed? It's a waste of my time and mental energy to write a new answer to this every time somebody asks.

But okay, here is the answer as far as I can remember offhand:

There aren't tagged releases because I don't have any interest in doing tagged releases. The stb projecct is not intended to be a package-manager friendly set of libraries. I didn't put it on github in 2014 to make it package-manager friendly, I put it on github in 2014 so there'd be a system for managing issues and pull requests that wasn't just "email patches to me".

In this thread I recognized the demand for tagged releases and expressed a willingness to do tagged releases if it wouldn't be too much friction for my workflow, but because in the end each library in the project (and there are 22) needs its own stream of tagged releases, there doesn't seem to be a viable workflow without splitting each library into its own project, which I'm not willing to do.

[kloczek]

Didn't I just tell you I wouldn't answer this specific question because it's already been addressed?

How it is adressed alredy? Which one version string I should use on packaging as version to be be able give you necessary details abot version if someone will spot for exampe some bug?

[N-R-K]

The entire point of single header libraries is so that they can be embedded directly into the project. Trying to package a single header lib is kinda nonsense.

So why stb has not been released qith regular version which coud be use by packagers?

Also you could've just skimmed through this thread and gotten your answer. It's very rude to be jumping into a thread without reading it first and asking questions which have already been answered.

[kloczek]

The entire point of single header libraries is so that they can be embedded directly into the project. Trying to package a single header lib is kinda nonsense.

Issue is that few pieces of software is using it as system available devel resources. So it doen't matter what it can be ..

[nothings]

Didn't I just tell you I wouldn't answer this specific question because it's already been addressed?

How it is adressed alredy? Which one version string I should use on packaging as version to be be able give you necessary details abot version if someone will spot for exampe some bug?

Motherfucker, read the rest of the fucking post you just quoted and stop wasting my time. You are one post away from a ban.

[nothings]

package-managment FAQ

How do I get tagged releases or otherwise use stb with a package manager?

You don't. It's not designed to do that.

But I need it!

That's not my job, that's your job. Sorry.

But why?

Read this post for a brief summary of the issues: https://github.com/nothings/stb/issues/359#issuecomment-1198320009

Is there a more detailed explanation than that brief summary?

Please read the relevant parts of the whole thread for more information on the issues.

What if I have a potential solution to the problem which accounts for the issues raised in the thread?

Great, please share it with us. That's why this issue remains open.

Is it ok to post in this thread if I haven't done the other things described in this FAQ?

No, your post will be ignored and deleted.

[nothings]

Some minor administrivia unrelated to the technical content:

Somebody reported my curse-word-laden reply earlier in the thread, even though I'm the owner of the repository. I'm assuming it was the guy asking questions. So, he's banned. If it wasn't him, no great loss.

Since I don't appear to be able to process the report the way the docs say (I think this might be another organization-only feature, or this abuse report is a report-to-github not report-to-project), and the fact it's reported makes it hidden by default and permanently hidden to people without an account and I can't find any way to change it, I'm copy/pasting the reported post here for better visibility. (I guess if it was somebody else who reported it, well, you can report this one as well, but I'm not sure what exactly you think this is accomplishing.)

Didn't I just tell you I wouldn't answer this specific question because it's already been addressed?

How it is adressed alredy? Which one version string I should use on packaging as version to be be able give you necessary details abot version if someone will spot for exampe some bug?

Motherfucker, read the rest of the fucking post you just quoted and stop wasting my time. You are one post away from a ban.

[wuch-g2v]

Issue is that few pieces of software is using it as system available devel resources. So it doen't matter what it can be ..

Looks like my colleague @kloczek has been blocked here only because he been asking the same questions as other people here. That is rude .. 🤔

[nothings]

Nah, he got blocked here for reporting the repository owner's comment. Learn to keep better company.

[corporateshark]

Closed after all these years... Nice!

[onitake]

Closed, but not solved. Oh, well. :shrug: