conda-forge.github.io icon indicating copy to clipboard operation
conda-forge.github.io copied to clipboard
verified publisher icon conda-forge

support v1 recipe format via rattler-build

Open beckermr opened this issue 1 year ago • 17 comments

This issue is to track progress on supporting the v1 recipe format via rattler build.

Minimal Work Items for Uploads:

  • [x] ensure conda-forge-feedstock-check-solvable works for the new format (https://github.com/regro/conda-forge-feedstock-check-solvable/pull/58)
  • [x] ensure the other migrators in the bot don't choke (https://github.com/regro/cf-scripts/pull/3006 ~https://github.com/regro/cf-scripts/pull/2918~)
  • [x] ensure ABI migrations in the bot can go through
    • [x] enable build number bumps (https://github.com/regro/cf-scripts/pull/3009)
    • [x] add some tests in the bot for ABI+v1 recipe migrations (replicate the tests of yaml-based migrations in this file: https://github.com/regro/cf-scripts/blob/main/tests/test_migrators.py, https://github.com/regro/cf-scripts/pull/3011)
  • [x] fix v1 recipe linting (https://github.com/conda-forge/conda-smithy/issues/2067)
  • [x] fully turn on the linter (it still hints) (https://github.com/conda-forge/conda-forge-webservices/pull/685, https://github.com/conda-forge/conda-forge-feedstock-ops/pull/14)
  • [x] turn on uploads (last one! https://github.com/conda-forge/conda-forge-ci-setup-feedstock/pull/337)

Next Steps + Bug Reports

  • [x] https://github.com/conda-forge/conda-smithy/issues/2085
  • [ ] https://github.com/conda-forge/conda-smithy/issues/2082
  • [ ] https://github.com/conda-forge/staged-recipes/issues/27750
  • [x] https://github.com/conda-forge/staged-recipes/issues/27695
  • [x] https://github.com/regro/cf-scripts/pull/3018
  • [ ] https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/359
  • [x] https://github.com/regro/cf-scripts/pull/3525
  • [ ] enable selected minimigrators as they are needed (https://github.com/regro/cf-scripts/issues/3642)
  • [x] https://github.com/regro/conda-forge-feedstock-check-solvable/issues/61
  • [x] add linter lint+hint for recipe being able to be parsed so we can run automatic conversion (https://github.com/conda-forge/conda-smithy/pull/2141)

Running List of Blockers for Automatic Migration PRs from the Bot

  • [x] https://github.com/prefix-dev/rattler-build/issues/1455
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1451
  • [x] https://github.com/prefix-dev/rattler-build/issues/1445
  • [x] https://github.com/prefix-dev/rattler-build/issues/1443
  • [x] https://github.com/prefix-dev/rattler-build/issues/1437
  • [x] https://github.com/prefix-dev/rattler-build/issues/1439
  • [x] https://github.com/prefix-dev/rattler-build/issues/1440
  • [x] https://github.com/prefix-dev/rattler-build/issues/1438
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1317
  • [x] https://github.com/prefix-dev/rattler-build/issues/1474
  • [x] https://github.com/prefix-dev/rattler-build/issues/1476
  • [x] https://github.com/prefix-dev/rattler-build/issues/1480
  • [x] https://github.com/prefix-dev/rattler-build/issues/1491
  • [x] https://github.com/prefix-dev/rattler-build/issues/1493
  • [x] https://github.com/prefix-dev/rattler-build/issues/1498
  • [ ] https://github.com/prefix-dev/rattler-build/issues/889
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1521
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1638
  • [x] https://github.com/prefix-dev/rattler-build/issues/1636
  • [x] https://github.com/prefix-dev/rattler-build/issues/1436
  • [x] https://github.com/prefix-dev/rattler-build/issues/1639
  • [x] https://github.com/prefix-dev/rattler-build/issues/1643
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1784
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1750
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1748
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1733
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1671
  • [ ] https://github.com/prefix-dev/rattler-build/issues/1317
  • [x] https://github.com/prefix-dev/rattler-build/issues/1897
  • [x] https://github.com/prefix-dev/rattler-build/issues/1916
  • [ ] https://github.com/prefix-dev/rattler-build/issues/960

supersedes #1612

beckermr avatar Sep 19 '24 14:09 beckermr

I think for the conda-forge-feedstock-check solvable I will have a good grip on what needs to be done.

For the second point, ensure that ABI migrations in the bot can go through, I actually don't really know where to start. Do you have additional pointers?

wolfv avatar Sep 19 '24 16:09 wolfv

Right so for that, once the solver checks work, we should add a test to the bot repo that does an abi migration and make sure it all runs.

beckermr avatar Sep 19 '24 17:09 beckermr

https://github.com/regro/conda-forge-feedstock-check-solvable/pull/58 this is the PR for adding solvability checks with rattler-build.

wolfv avatar Sep 20 '24 16:09 wolfv

https://github.com/conda-forge/conda-forge-webservices/pull/685 opened to enable lints on PRs etc.

wolfv avatar Sep 23 '24 08:09 wolfv

@beckermr - I think I need some help / guidance in order to understand what's needed for the cf-graph items. How can I ensure that?

wolfv avatar Sep 23 '24 08:09 wolfv

Thanks @beckermr for updating the todo items. I can make the build number PR right away (it's a subset of the version bump PR) so we can get that one in straight away! If you let me know how / where to add tests for migrations I can do it tomorrow, too :)

wolfv avatar Sep 23 '24 18:09 wolfv

I updated the issue list above with a pointer to the correct test file. We should replicate all of the tests there for yaml migrations to ensure they work on v1 recipes.

beckermr avatar Sep 23 '24 20:09 beckermr

Added PR for build number bumps: https://github.com/regro/cf-scripts/pull/3009

wolfv avatar Sep 24 '24 08:09 wolfv

Only the tests to go! :)

wolfv avatar Sep 25 '24 08:09 wolfv

Ticked off the last one (uploading packages) :)

There are two more on staged-recipes:

  • [ ] https://github.com/conda-forge/staged-recipes/issues/27750
  • [ ] https://github.com/conda-forge/staged-recipes/issues/27695

wolfv avatar Oct 02 '24 21:10 wolfv

Opened

  • [ ] https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/359

isuruf avatar Oct 02 '24 21:10 isuruf

@beckermr are you OK if I edit your initial comment to include the additional TODO items?

wolfv avatar Oct 03 '24 10:10 wolfv

@wolfv there are some seeming important issues here (e.g., not including licenses in some cases).

I suspect it may be prudent to hold off on more conversions to v1 until more of them are fixed.

beckermr avatar Apr 23 '25 16:04 beckermr

@beckermr that particular issue (folders not being globbed if there is no / at the end) has been fixed since ~2-3 releases.

What are the big issues you are seeing?

wolfv avatar May 02 '25 10:05 wolfv

Oh that's great! Can we go through the rattler-build issue tracker and list of issues above and mark any that are fixed as closed?

I also think we still have open issues on recipes with merging build host and test env activation.

beckermr avatar May 02 '25 10:05 beckermr

Wow that's a lot of work on the issue tracker! Thanks for the big push on this!

beckermr avatar May 05 '25 11:05 beckermr

cc @wolfv

beckermr avatar May 05 '25 11:05 beckermr