semaphore icon indicating copy to clipboard operation
semaphore copied to clipboard

No collections requirements file found

Open kirolous opened this issue 2 years ago • 12 comments

image

File exists, however semaphore cant seem to find it

kirolous avatar Jan 12 '23 18:01 kirolous

on version 2.8.77 image

kirolous avatar Jan 12 '23 18:01 kirolous

I have the same problem since the update today. The reading of the Collections path has been changed. It is no longer searched globally for collections/requirements.yml, but in the folder where the playbook is located.

In my case this gives a big chaos, because I have structured my playbooks cleanly in folders and now I have to create many collections/requirements.txt files.

Example:
playbooks
├── linux
│   ├── xxx
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── yyy
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── zzz
│   │   ├── collections
│   │   │   ├── requirements.yml

Sure, the advantage is that you only have to install the Collections that are also needed for this Playbook. For larger projects, however, it is total chaos.

jp1337 avatar Feb 01 '23 16:02 jp1337

The problem is here: https://github.com/ansible-semaphore/semaphore/blob/864d8bd8aa5d55f334a9ae4f3c52aa44667a65ac/services/tasks/runner.go#L508

Whereas Roles looks like: https://github.com/ansible-semaphore/semaphore/blob/864d8bd8aa5d55f334a9ae4f3c52aa44667a65ac/services/tasks/runner.go#L537

The role implementation does this correctly and the collections implementation should be changed to look at the base of the project/repository then 'collections/requirements.yml'.

nlvw avatar Apr 20 '23 21:04 nlvw

I have the same problem since the update today. The reading of the Collections path has been changed. It is no longer searched globally for collections/requirements.yml, but in the folder where the playbook is located.

In my case this gives a big chaos, because I have structured my playbooks cleanly in folders and now I have to create many collections/requirements.txt files.

Example:
playbooks
├── linux
│   ├── xxx
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── yyy
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── zzz
│   │   ├── collections
│   │   │   ├── requirements.yml

Sure, the advantage is that you only have to install the Collections that are also needed for this Playbook. For larger projects, however, it is total chaos.

Until this is fixed my current workaround is to create symlinks that point to roles/requirements.yml. So each of my playbook directories have a 'collections' folder which contains said symlink. Even my collections/requirements.yml file is a symlink to roles/requirements.yml. Ansible galaxy allows you to define both roles and collections in a single file which is nice.

nlvw avatar Apr 20 '23 21:04 nlvw

Any workaround? I also faced the same issues.

Ye-Min-Tun avatar May 03 '23 09:05 Ye-Min-Tun

I have the same problem since the update today. The reading of the Collections path has been changed. It is no longer searched globally for collections/requirements.yml, but in the folder where the playbook is located. In my case this gives a big chaos, because I have structured my playbooks cleanly in folders and now I have to create many collections/requirements.txt files.

Example:
playbooks
├── linux
│   ├── xxx
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── yyy
│   │   ├── collections
│   │   │   ├── requirements.yml
│   ├── zzz
│   │   ├── collections
│   │   │   ├── requirements.yml

Sure, the advantage is that you only have to install the Collections that are also needed for this Playbook. For larger projects, however, it is total chaos.

Until this is fixed my current workaround is to create symlinks that point to roles/requirements.yml. So each of my playbook directories have a 'collections' folder which contains said symlink. Even my collections/requirements.yml file is a symlink to roles/requirements.yml. Ansible galaxy allows you to define both roles and collections in a single file which is nice.

This still doesn't work for me.

Ye-Min-Tun avatar May 03 '23 09:05 Ye-Min-Tun

@Ye-Min-Tun FWIW I've implemented something similar in my MR to resolve this issue. Please review/test it and see if it works for you https://github.com/ansible-semaphore/semaphore/pull/1240

andreas-marschke avatar May 25 '23 00:05 andreas-marschke

@andreas-marschke Sure sir. I resolved this by adding a requirement.yml file inside collection directory. Note: earlier I think yml and yaml are the same. Semaphore only look for yml, not yaml. That spent my half a day.

Ye-Min-Tun avatar May 25 '23 01:05 Ye-Min-Tun

big issue causing me much pain please fix

pydlv avatar May 26 '23 02:05 pydlv

@pydlv You just need to add requirements.yml file (with the collections you need to install) under the collections directory of your workspace.

Ye-Min-Tun avatar May 26 '23 03:05 Ye-Min-Tun

At now (in 2.8.90) "right path" is requirementsFilePath := path.Join(t.getPlaybookDir(), "collections", "requirements.yml")

In my project tree it looks as:

├── ansible.cfg
├── collections
│   └── requirements.yml (original file)
├── playbooks
│   ├── one.yml
│   ├── two.yml
│   └── collections
│       └── requirements.yml -> ../../collections/requirements.yml (hot fix symlink)
└── roles
    ├── one
    └── two

I think was best solution search requirements.yml in all popular places:

/requirements.yml (root project dir)
/collections/requirements.yml
/[getPlaybookDir]/collections/requirements.yml

guard43ru avatar May 29 '23 15:05 guard43ru

Fully agree!

But probably reverse the order, and use the first one found ("most specific one")?

alexbarton avatar May 29 '23 18:05 alexbarton

just for the record: i'm working around this like this. so i can include it in all my playbooks which need some collections.

---
# playbooks/test_play.yml
- name: "ダミープラグ on localhost"
  hosts: localhost
  gather_facts: false
  connection: local
  tasks:
    - name: "Install ansible collections"
      ansible.builtin.include_tasks:
        file: "../tasks/install_collections.yml"

---
# tasks/install_collections.yml
- name: Install collections
  ansible.builtin.command: "ansible-galaxy collection install -r ../collections/requirements.yml"
  register: collections_install
  changed_when: collections_install.rc != 0

rwaffen avatar Sep 14 '23 08:09 rwaffen

Any workaround? I also faced the same issues.

yea you would need to put a collections/requirements.yml folder in the same path where the playbook you are running is.

RusticPotatoes avatar Dec 02 '23 00:12 RusticPotatoes

This is resolved with version v2.9.109 I think the issue can be closed.

jp1337 avatar Jun 17 '24 09:06 jp1337

I think was best solution search requirements.yml in all popular places:

/requirements.yml (root project dir)

Why is /requirements.yml (root project dir) not installing?

https://github.com/semaphoreui/semaphore/blob/550c2a54315c85be0ad08ef6940dd20e3d2a6b3f/db_lib/AnsibleApp.go#L128-L144

DeadNews avatar Jul 31 '24 08:07 DeadNews

@DeadNews, the root folder doesn't work for me either ... but I guess this is covered in #1240?

alexbarton avatar Jul 31 '24 19:07 alexbarton