Replace Addon's Library logic with Rez

[Minkiu]

Replace addons logic with Rez

This Pull Request, modifies the backend in order to leverage Rez to manage AYON addons (they should now include a package.py with the attribute plugin_for="ayon_server".

A loose explantation of the process is:

1. When we spin the server, we make sure a (meta) package named ayon_server exists in our repo, as of now /addons; and it matches the current ayon-backend version, otherwise we create one, and request a restart, this package has the attribute has_plugins=True
2. We try to find plug-ins for our ayon_server.
3. We, as before, attempt to initialize the add-ons defined in the bundles; we can now list all installed packages without actually initializing them, and keep a list of initialized and broken ones.
4. When installing an addon, we have a rezbuild.py that we fall back for simple addons; there's a bug in rez that makes it mandatory for the package.py to contain the build_command attribute set (tried to add it programmatically, but it won't install the package, due this bug).

All the rest should work as it does now, I really haven't tested the migration of settings between bundles, so that might not be working at the minute.

Reviewing

I tired to keep it tidy and compartemantilized, so there are 4 commits:

1. Implement the RezRepo class
2. Refactor AddonLibrary to make use of the RezRepo
3. Modify all instance of the AddonLibrary in the api folder, to adapt to any required changes
4. Modify all instances of the AddonLibrary in the ayon_server folder, to adapt to any required changes, this one has some bits that have changed more than simple method renames.

TODO

• Adapt AYON addons to rez packages (simply add a package.py at the root).
• Refactor the BaseServerAddon and move some of the attributes to the package.py then, read the info from there.

Testing note

I would suggest making a copy of the docker-compose.yml in the ayon-docker backend and build a new image called ayon-rez that build the new rez dependency; so it does not interfere with your existing deploys; also map the /addons folder to a different one than the one you are currently using.

version: "3.7"

services:
  postgres:
    image: postgres:15
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ayon"]
      interval: 5s
      timeout: 5s
      retries: 5

    expose: [5432]

    volumes:
      - "/etc/localtime:/etc/localtime:ro"
      - "db:/var/lib/postgresql/data"

    environment:
      - "POSTGRES_USER=ayon"
      - "POSTGRES_PASSWORD=ayon"
      - "POSTGRES_DB=ayon"

  redis:
    image: redis:alpine
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
      interval: 5s
      timeout: 5s
      retries: 5

    expose: [6379]


  server:
    image: localhost/ayon-rez:latest
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5000/api/info"]
      interval: 10s
      timeout: 2s
      retries: 3

    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_started
    # uncomment for older versions of Docker (>4.0.0) and comment out above "depends on" section.
    #depends_on:
      #- postgres
      #- redis

    expose: [5000]
    ports: ["5000:5000"]

    volumes:
      - "./rezrepo:/addons:Z"
      - "./storage:/storage:Z"

      # comment out the following line on Windows
      - "/etc/localtime:/etc/localtime:ro"

      # uncomment the following line if you need to work on the backend code
      - "/home/minkiu/Projects/ynput/ayon-backend:/backend:Z"

volumes:
  db: {}

You can use this PR to generate rez packages out of the OpenPype ones: https://github.com/ynput/OpenPype/pull/6169

[Minkiu]

Marking as draft so we can have a conversation about this.

[BigRoy]

Thanks for drafting a prototype! I haven't been involved much with the Ayon backend but just wanted to say that any work on this front in combination with Rez is greatly appreciated.

This PR's Rez integration is solely regarding the addons and modules in the backend, not the running of the actual DCC application and tools, right? So it's not for those that might have a huge ready-to-go Rez repository already with many applications and tools and allow them to launch those through OpenPype for certain hosts. Or am I mistaken?

[antirotor]

This PR's Rez integration is solely regarding the addons and modules in the backend, not the running of the actual DCC application and tools, right? So it's not for those that might have a huge ready-to-go Rez repository already with many applications and tools and allow them to launch those through OpenPype for certain hosts. Or am I mistaken?

For that we will need to implement alternative to applications addon that will use rez and also we will need to create rez package from ayon-launcher (that will be the new entry point as Igniter in OP is)

[BigRoy]

For that we will need to implement alternative to applications addon that will use rez and also we will need to create rez package from ayon-launcher (that will be the new entry point as Igniter in OP is)

Great. Just to confirm, so this particular PR is not intended to do that. Right?

[Minkiu]

You are right, this is not the PR for that.

And there are other changes happening on the same parts, that I'll need to rebase later, so this won't be merge in the near future I'd say.

[iLLiCiTiT]

Is with this PR possible to manually add/remove addon to/from server?

[Minkiu]

No, this PR doesn't implement the adding/removing side of thing, will wait till the bundle changes settle to rebase and piggyback on current implementation to install /remove and hook there Rez

[Minkiu]

I am finally satisfied with the commit history, and managed to catch some bugs, it is now ready to be reviewed, will update the PR to reflect changes.