homebrew-core
homebrew-core copied to clipboard
Homebrew
gobject-introspection: split out Python code
- [ ] Have you followed the guidelines for contributing?
- [ ] Have you ensured that your commits follow the commit style guide?
- [ ] Have you checked that there aren't other open pull requests for the same formula update/change?
- [ ] Have you built your formula locally with
brew install --build-from-source <formula>, where<formula>is the name of the formula you're submitting? - [ ] Is your test running fine
brew test <formula>, where<formula>is the name of the formula you're submitting? - [ ] Does your build pass
brew audit --strict <formula>(after doingbrew install --build-from-source <formula>)? If this is a new formula, does it passbrew audit --new <formula>?
Copied from https://github.com/Homebrew/homebrew-core/pull/106041#issuecomment-1186877279
prefix/"lib/gobject-introspection/giscanner" contains .py files, cpython extension _giscanner.cpython-39-x86_64-linux-gnu.so, and a directory doctemplates for *.tmpl files. I'm not sure we can only split Python code like we did for glib-utils. In glib-utils, I glob for *.py[diwx] files and cpython extension *.cpython-3x-*.so. We may need to split the entire prefix/"lib/gobject-introspection/giscanner" folder rather than glob for Python files.
$ tree "$(brew --prefix gobject-introspection)/lib"
$HOMEBREW_PREFIX/opt/gobject-introspection/lib
├── girepository-1.0
│ ├── cairo-1.0.typelib
│ ├── DBus-1.0.typelib
│ ├── DBusGLib-1.0.typelib
│ ├── fontconfig-2.0.typelib
│ ├── freetype2-2.0.typelib
│ ├── Gio-2.0.typelib
│ ├── GIRepository-2.0.typelib
│ ├── GL-1.0.typelib
│ ├── GLib-2.0.typelib
│ ├── GModule-2.0.typelib
│ ├── GObject-2.0.typelib
│ ├── libxml2-2.0.typelib
│ ├── Vulkan-1.0.typelib
│ ├── win32-1.0.typelib
│ ├── xfixes-4.0.typelib
│ ├── xft-2.0.typelib
│ ├── xlib-2.0.typelib
│ └── xrandr-1.3.typelib
├── gobject-introspection
│ └── giscanner
│ ├── annotationmain.py
│ ├── annotationparser.py
│ ├── ast.py
│ ├── cachestore.py
│ ├── ccompiler.py
│ ├── codegen.py
│ ├── docmain.py
│ ├── doctemplates
│ │ ├── devdocs
│ │ │ ├── Gjs
│ │ │ │ ├── base.tmpl
│ │ │ │ ├── callback.tmpl
│ │ │ │ ├── class.tmpl
│ │ │ │ ├── default.tmpl
│ │ │ │ ├── _doc.tmpl
│ │ │ │ ├── enum.tmpl
│ │ │ │ ├── function.tmpl
│ │ │ │ ├── _index.tmpl
│ │ │ │ ├── interface.tmpl
│ │ │ │ ├── _methods.tmpl
│ │ │ │ ├── _method.tmpl
│ │ │ │ ├── method.tmpl
│ │ │ │ ├── namespace.tmpl
│ │ │ │ ├── _properties.tmpl
│ │ │ │ ├── _signals.tmpl
│ │ │ │ ├── _staticmethods.tmpl
│ │ │ │ └── _vfuncs.tmpl
│ │ │ └── meson.build
│ │ └── mallard
│ │ ├── base.tmpl
│ │ ├── C
│ │ │ ├── callback.tmpl
│ │ │ ├── class.tmpl
│ │ │ ├── constructor.tmpl
│ │ │ ├── default.tmpl
│ │ │ ├── enum.tmpl
│ │ │ ├── field.tmpl
│ │ │ ├── function.tmpl
│ │ │ ├── interface.tmpl
│ │ │ ├── method.tmpl
│ │ │ ├── namespace.tmpl
│ │ │ ├── property.tmpl
│ │ │ ├── record.tmpl
│ │ │ ├── signal.tmpl
│ │ │ └── vfunc.tmpl
│ │ ├── class.tmpl
│ │ ├── Gjs
│ │ │ ├── callback.tmpl
│ │ │ ├── class.tmpl
│ │ │ ├── constructor.tmpl
│ │ │ ├── default.tmpl
│ │ │ ├── enum.tmpl
│ │ │ ├── field.tmpl
│ │ │ ├── function.tmpl
│ │ │ ├── interface.tmpl
│ │ │ ├── method.tmpl
│ │ │ ├── namespace.tmpl
│ │ │ ├── property.tmpl
│ │ │ ├── record.tmpl
│ │ │ ├── signal.tmpl
│ │ │ └── vfunc.tmpl
│ │ ├── meson.build
│ │ ├── namespace.tmpl
│ │ └── Python
│ │ ├── callback.tmpl
│ │ ├── class.tmpl
│ │ ├── constructor.tmpl
│ │ ├── default.tmpl
│ │ ├── enum.tmpl
│ │ ├── field.tmpl
│ │ ├── function.tmpl
│ │ ├── interface.tmpl
│ │ ├── method.tmpl
│ │ ├── namespace.tmpl
│ │ ├── property.tmpl
│ │ ├── record.tmpl
│ │ ├── signal.tmpl
│ │ └── vfunc.tmpl
│ ├── docwriter.py
│ ├── dumper.py
│ ├── gdumpparser.py
│ ├── girparser.py
│ ├── girwriter.py
│ ├── _giscanner.cpython-39-x86_64-linux-gnu.so
│ ├── __init__.py
│ ├── introspectablepass.py
│ ├── maintransformer.py
│ ├── mdextensions.py
│ ├── message.py
│ ├── msvccompiler.py
│ ├── pkgconfig.py
│ ├── scannermain.py
│ ├── sectionparser.py
│ ├── shlibs.py
│ ├── sourcescanner.py
│ ├── testcodegen.py
│ ├── transformer.py
│ ├── utils.py
│ ├── _version.py
│ └── xmlwriter.py
├── libgirepository-1.0.so -> libgirepository-1.0.so.1
├── libgirepository-1.0.so.1 -> libgirepository-1.0.so.1.0.0
├── libgirepository-1.0.so.1.0.0
└── pkgconfig
├── gobject-introspection-1.0.pc
└── gobject-introspection-no-export-1.0.pc
11 directories, 116 files
The way I was handling it right now (which may not be correct or ideal) was to let prefix/"lib/gobject-introspection/giscanner/doctemplates" still be installed by gobject-introspection, and the rest of giscanner gets installed by gobject-introspection-utils. My reasoning here is that since gobject-introspection is a dependency of gobject-introspection-utils, then prefix/"lib/gobject-introspection/giscanner/doctemplates" is guaranteed to be installed.
One oddity that comes out of this is that prefix/"lib/gobject-introspection/giscanner/doctemplates" isn't probably needed for gobject-introspection on its own, and I had to manually install the rest of the files for prefix/"lib/gobject-introspection/giscanner/" in gobject-introspection-utils.
The other issue that comes up here is that g-ir-scanner is needed for the current test we use for gobject-introspection, which means gobject-introspection-utils is a test dependency of gobject-introspection. I'm not sure what the best solution is to this as it's sort of a cyclic dependency - we may just want to use a different test for gobject-introspection, or modify the existing one so it doesn't need g-ir-scanner.
Let's see what other maintainers think about the current situation. These problems seem solvable but may require some discussion about the best solution.
Everything seems to be working now, though I foolishly forgot to revision bump gobject-introspection. I don't love the solution we're using right and we should think about how some of this might be managed by brew, but I'd rather we not block merging of this PR on changes in brew that may take a while to figure out. From what I can tell, this is truly the final blocker to the [email protected] migration - this wasn't immediately obvious because of glib, but there are quite a few Python dependents of gobject-introspection.
Since dependent testing found no instances of breakage (texlive failed on Linux due to lack of disk space and the other failures there were unrelated), I'm thinking we can rerun CI now without dependent testing. That should let us merge this ASAP and begin the final push to migrate everything to [email protected].
I had to deparallelize the test for gobject-introspection-utils because it seems to experience a sporadic race condition. I think this is now ready to merge.
I did run dependent testing in the last run and it all looked fine but I can rerun it again now. The only thing I changed from the last run was a revision bump I forgot for gobject-introspection.
As with the previous dependent testing run, there were no failures on macOS and the only ones on Linux were unrelated.
There's a whole bunch of cached linkage warnings though. Those are probably going to be the same as what happened with Qt.
For example, on Monterey:
brew linkage --cached --test --strict cherrytree
brew linkage --cached --test --strict [email protected]
brew linkage --cached --test --strict freeciv
brew linkage --cached --test --strict gammaray
brew linkage --cached --test --strict libchamplain
brew linkage --cached --test --strict mgba
brew linkage --cached --test --strict mikutter
brew linkage --cached --test --strict ntopng
These will need to be fixed by adding the missing dependencies. No new bottles need to be published, but these need to be fixed before merging this.
This is a checklist of the formula with linkage failures for at least one tag:
- [ ] cherrytree -
sqlite - [ ] [email protected] -
xz - [ ] fceux -
mesa-glu- may be unrelated but should still be fixed - [ ] freeciv -
sqlite - [ ] gammaray - missing RPATH - may be unrelated but should still be fixed
- [ ] libchamplain -
sqlite - [ ] msc-generator -
mesa- may be unrelated but should still be fixed - [ ] mgba -
sqlite - [ ] mikutter -
xz - [ ] ntopng -
sqlite
One thing I'm not sure about - it seems that the linkage failures for sqlite and xz are coming up because the formula depends on brewed sqlite or xz rather than the versions provided by macOS. This happens because [email protected] uses depends_on "sqlite" and depends_on "xz", and because those are picked up by the linker before the system copies, they get used.
I think this means the only way resolve the linkage failure is to explicitly depend on the brewed versions of sqlite and xz, or to revision bump the formulae here so that they will use the macOS versions once [email protected] is removed from their dependency tree.
Do we want to go with the first option? I guess my only concern with that approach is that we could end up needlessly carrying dependencies on brewed sqlite and xz on macOS. We could of course revision bump the affected formulae afterward, but at that point we might as well just do the revision bumps here.
Just add the dependencies for now with a comment to investigate their removal, so it can be done without publishing new bottles or testing dependents.
I rebased this now that all the relevant linkage failures have been addressed. The gammaray RPATH audit failure is clearly unrelated as it was pre-existing and therefore should not block this PR.
One thing to watch out for is build-time breakages due to some binaries missing after splitting out, similar to what is seen with glib-utils.
Also will want to check on pkgconfig files. In case of glib, we still need to fix pkgconfig as they include variables pointing to binaries, e.g.
% pkg-config --variable=glib_mkenums glib-2.0
/usr/local/Cellar/glib/2.72.2_1/bin/glib-mkenums
% file /usr/local/Cellar/glib/2.72.2_1/bin/glib-mkenums
/usr/local/Cellar/glib/2.72.2_1/bin/glib-mkenums: cannot open `/usr/local/Cellar/glib/2.72.2_1/bin/glib-mkenums' (No such file or directory)
% file /usr/local/Cellar/glib-utils/2.72.2/bin/glib-mkenums
/usr/local/Cellar/glib-utils/2.72.2/bin/glib-mkenums: a /usr/local/opt/[email protected]/bin/python3 script text executable, ASCII text
% pkg-config --variable=g_ir_scanner gobject-introspection-1.0
/usr/local/Cellar/gobject-introspection/1.72.0/bin/g-ir-scanner
% file /usr/local/Cellar/gobject-introspection/1.72.0/bin/g-ir-scanner
/usr/local/Cellar/gobject-introspection/1.72.0/bin/g-ir-scanner: a /usr/local/opt/[email protected]/bin/python3 script text executable, ASCII text
One thing to watch out for is build-time breakages due to some binaries missing after splitting out, similar to what is seen with
glib-utils.Also will want to check on pkgconfig files. In case of
glib, we still need to fix pkgconfig as they include variables pointing to binaries, e.g.
Could we make relative symlinks glib/bin/gdbs-codegen / glib/bin/genmarshal / glib/bin/glib-mkenums point to the glib-utils/bin/*? Then include the symlinks in the bottle. The symlinks may be deadlinks if the user does not install glib-utils.
In glib/lib/pkgconfig/gio-2.0.pc:
prefix=/home/linuxbrew/.linuxbrew/Cellar/glib/2.72.3
includedir=${prefix}/include
libdir=${prefix}/lib
datadir=${prefix}/share
schemasdir=${datadir}/glib-2.0/schemas
bindir=${prefix}/bin
giomoduledir=${libdir}/gio/modules
gio=${bindir}/gio
gio_querymodules=${bindir}/gio-querymodules
glib_compile_schemas=${bindir}/glib-compile-schemas
glib_compile_resources=${bindir}/glib-compile-resources
gdbus=${bindir}/gdbus
-gdbus_codegen=${bindir}/gdbus-codegen
gresource=${bindir}/gresource
gsettings=${bindir}/gsettings
Name: GIO
Description: glib I/O library
Version: 2.72.3
Requires: glib-2.0, gobject-2.0
Requires.private: gmodule-no-export-2.0, zlib, mount >= 2.23
Libs: -L${libdir} -lgio-2.0
Libs.private: -ldl -lresolv
Cflags: -I${includedir}
In glib/lib/pkgconfig/glib-2.0.pc:
prefix=/home/linuxbrew/.linuxbrew/Cellar/glib/2.72.3
includedir=${prefix}/include
libdir=${prefix}/lib
bindir=${prefix}/bin
-glib_genmarshal=${bindir}/glib-genmarshal
gobject_query=${bindir}/gobject-query
-glib_mkenums=${bindir}/glib-mkenums
Name: GLib
Description: C Utility Library
Version: 2.72.3
Requires.private: libpcre >= 8.31
Libs: -L${libdir} -lglib-2.0
Libs.private: -pthread -lm
Cflags: -I${includedir}/glib-2.0 -I${libdir}/glib-2.0/include
Could we make relative symlinks
glib/bin/gdbs-codegen/glib/bin/genmarshal/glib/bin/glib-mkenumspoint to theglib-utils/bin/*? Then include the symlinks in the bottle. The symlinks may be deadlinks if the user does not installglib-utils.
Just adding symlinks will lead to conflicts as brew link will try to add both to HOMEBREW_PREFIX/bin.
I think pkgconfig paths will need an inreplace to fix. There aren't that many packages that use these, but we saw this in #106446.
This is the .pc file for gobject-introspection:
prefix=/usr/local/Cellar/gobject-introspection/1.72.0
includedir=${prefix}/include
libdir=${prefix}/lib
datadir=${prefix}/share
bindir=${prefix}/bin
g_ir_scanner=${bindir}/g-ir-scanner
g_ir_compiler=${bindir}/g-ir-compiler
g_ir_generate=${bindir}/g-ir-generate
gidatadir=${datadir}/gobject-introspection-1.0
girdir=${datadir}/gir-1.0
typelibdir=${libdir}/girepository-1.0
Name: gobject-introspection
Description: GObject Introspection
Version: 1.72.0
Requires: glib-2.0 >= 2.58.0, gobject-2.0 >= 2.58.0
Libs: -L${libdir} -lgirepository-1.0
Cflags: -I${includedir}/gobject-introspection-1.0
I think the only problem is with the g-ir-scanner, because that is now in gobject-introspection-utils. It should be simple enough to inreplace this.
I'd like to take a different approach here, but requires some changes in brew. See https://github.com/Homebrew/brew/pull/13690.
Can we just use uses_from_macos "python3" now that https://github.com/Homebrew/brew/pull/13690 is merged?.
No, we can't. See
https://github.com/Homebrew/homebrew-core/blob/3684df6736dfe27ccebdd26a5828ba0289a10483/Formula/gobject-introspection.rb#L26-L27
Definitely prefer to avoid the hacks if we have a working solution to avoid the dependency conflict. This doesn't ship any Python libraries so it should never have caused a dependency conflict in the first place.