prefix-dev
Low Level Python Bindings
As we discussed very shortly (hopefully more here) rattler-build only exposes high-level python bindings that doesn't meaningfully give us exposure to rattler-build's internal tools (which should be extremely exciting tbh) so I was thinking how should we go about this, planning main classes and classes for external tools that comes to my mind. A few ideas i had for tools are specially linter (replacing the existing github actions for conde-forge's linter, as well as making a vscode/pycharm/zed extensions) as well as extending our recipe generation and this linter/validation in runtime to MCP servers in order for these tools to actually use rattler-build's already existing tools easily with much better speed.
Classes we need
RecipeGenerator - wraps the existing generate-recipe commands so you can call them from Python and get back a Recipe object instead of just YAML text. Need this because MCP servers want to generate recipes programmatically.
RuntimeValidator - validates recipes as you're editing them and gives back structured errors with line numbers. Need this for real-time feedback in editors. We can technically add more stuff similar to clippy where WARN - ERROR etc levels exists if we can actually add duplicate/redudant line detections for rattler-build, but that would be a maybe overkill? or not even needed? I like to hear your opinions on this.
Recipe - represents a parsed recipe with all its parts (package info, dependencies, build steps). Need this so you can work with recipes as objects instead of raw YAML. This is basically one of the main "we need this, no matter what" classes where schema_version, context, package, source, build, requirements, tests, about has to be exposed in reader-only mode.
RecipeError - exception that includes the file location where the error happened. Need this so IDEs can highlight exactly where problems are. But also, this will be used to highlight the issues with runtime errors on our lint class. So we need to use miette for this in order for UX to be fully compatible tbh.
RecipeLinter - main linting class with two modes: IDE mode (lenient, allows undefined variables, returns warnings) and CI mode (strict, fails on errors).
IDESupport - provides autocompletion, hover info, and real-time diagnostics for VS Code extensions. Returns LSP-compatible diagnostic messages. Already YAML server provide good enough information, but maybe we want to take control and provide extra information with our API?
SelectorConfig - controls how recipes get parsed (strict vs lenient). IDE mode allows undefined variables, CI mode requires everything to be defined. This to give control over the behavior of RecipeLinter so that users can play around with their desired outcome.
VariantConfig - handles variant matrices (python versions, architectures, etc). The RecipeLinter and TemplateRenderer will use this to analyze all possible recipe variations.
TemplateRenderer - processes Jinja templates in recipes using Rust's MiniJinja. We will basically, as always, use our rust code as source of truth and our minijinja integration at the end is what i am propose to use.
Platform - represents target platforms like linux-64, win-64. Just wraps the existing Rust Platform enum. Need this for cross-platform recipe generation.
After a good amount of time of testing by myself to see some limits and practical considerations of these ideas with these classes, I think these are the ones I want to go for but obviously i might be missing some things, some application usages so I would be very glad to hear more from everyone for their additions to these.
Is IDESupport really needed in the initial version. Maybe you can also make a mock-up how you would use this API :)? Like what the user should type.
I think we could start with recipe generation for some practice of writing Python bindings. For now, they could return a String to Python.
We could then move on to feed that string to rattler-build and get an Output. Or even from a bare minimum recipe. And make sure that the fields & functions on an Output are accessible from Python. We should thne be able to call output.run_build(...) from Python to actually kick off the building of the package.
