Force requirements install

[Arkhenys]

Hello,

I recently figured that when using Semaphore v2.9.37, requirements are not forcibly installed. I found this comment, a bit old now https://github.com/ansible-semaphore/semaphore/issues/682#issuecomment-897026897 where it is indicated that requirements are forcibly installed using ansible-galaxy install -r roles/requirements.yml --force

Did I miss something in my configuration ? Is there a way to forcibly install roles and collections requirements using Semaphore ?

Thanks a lot for your help :)

[Arkhenys]

Hello,

To be clearer, it seems that Semaphore doesn't even try to install requirements if nothing changed directly in the requirements.yml file. I think we should have the option, as a user, to specify if we want a requirements installation or if we want it to be skipped.

[Arkhenys]

Good afternoon @fiftin

Any plan to add the possibility to reinstall requirements for each task launched ? The current behavior is really annoying as we have to find a workaround to indicate to Semaphore that the requirements file has changed to trigger requiremetns installation.

The current problem is that if there is no change detected in requirements file, requirements won't be reinstalled. But for example, in that case:

---
#===============================================================================
# Ansible collections
#===============================================================================
collections:
  - name: [email protected]:project/path/to/collections/my_collection/collection.git
    type: git
    version: my_version

If I made a change in the code of collection.git on "my_version" branch, there is no change in the requirements.yml file but the code of the collection changed. However, semaphore won't reinstall requirements because no changes on the requirements file will be found. Adding the possibility to reinstall requirements even with no detected changes in requirements file would be really appreciated.

Thanks for your help, have a good day :)

[Arkhenys]

Hello @fiftin, any news on it ?

[manzomanze]

I am really interested in the force reinstall of the collections like @Arkhenys, any news @fiftin ?

[pshepelev]

+1 @fiftin

[occhioni-esteco]

+1 @fiftin

[maverick1982]

+1 @fiftin

[MascHman]

+1

[daniel-soler]

+1 @fiftin

[steadfasterX]

@fiftin it seems there are a lot (like me) wanting to see this added .. so I really hope u find the time integrating it ;)

[hagaram]

+1

[hagaram]

i dont quite understand the decision behind this behaviour or that it is not possible to override. But this is what I ended up doing. semaphore.patch

diff '--color=auto' -ruN semaphore-2.10.35/db_lib/AnsibleApp.go semaphore-2.10.35-patched/db_lib/AnsibleApp.go
--- semaphore-2.10.35/db_lib/AnsibleApp.go	2024-10-27 23:14:27.000000000 +0100
+++ semaphore-2.10.35-patched/db_lib/AnsibleApp.go	2024-11-14 07:44:24.446242879 +0100
@@ -9,6 +9,7 @@
 
 	"github.com/semaphoreui/semaphore/db"
 	"github.com/semaphoreui/semaphore/pkg/task_logger"
+	"github.com/semaphoreui/semaphore/util"
 )
 
 func getMD5Hash(filepath string) (string, error) {
@@ -26,6 +27,10 @@
 }
 
 func hasRequirementsChanges(requirementsFilePath string, requirementsHashFilePath string) bool {
+	if util.Config.AnsibleForceInstallRequirements {
+		return true
+	}
+	
 	oldFileMD5HashBytes, err := os.ReadFile(requirementsHashFilePath)
 	if err != nil {
 		return true
diff '--color=auto' -ruN semaphore-2.10.35/util/config.go semaphore-2.10.35-patched/util/config.go
--- semaphore-2.10.35/util/config.go	2024-10-27 23:14:27.000000000 +0100
+++ semaphore-2.10.35-patched/util/config.go	2024-11-14 07:44:24.447242871 +0100
@@ -196,6 +196,7 @@
 	// feature switches
 	PasswordLoginDisable     bool `json:"password_login_disable,omitempty" env:"SEMAPHORE_PASSWORD_LOGIN_DISABLED"`
 	NonAdminCanCreateProject bool `json:"non_admin_can_create_project,omitempty" env:"SEMAPHORE_NON_ADMIN_CAN_CREATE_PROJECT"`
+	AnsibleForceInstallRequirements bool `json:"ansible_force_install_requirements,omitempty" env:"SEMAPHORE_ANSIBLE_FORCE_INSTALL_REQUIREMENTS"`
 
 	UseRemoteRunner bool `json:"use_remote_runner,omitempty" env:"SEMAPHORE_USE_REMOTE_RUNNER"`

Snippet of build from Gitlab CI

    - git clone https://github.com/semaphoreui/semaphore.git
   - cp deployment/semaphore_docker/src/semaphore.patch semaphore/
   - cd semaphore
   - git checkout v${SEMAPHORE_VERSION}
   - patch -p1 < semaphore.patch
   - docker buildx build --pull -f deployment/docker/server/Dockerfile --build-arg SEMAPHORE_VERSION=${SEMAPHORE_VERSION} --tag ${CI_REGISTRY_IMAGE}/semaphore:v${SEMAPHORE_VERSION} .
   - docker push ${CI_REGISTRY_IMAGE}/semaphore:v${SEMAPHORE_VERSION}

Now I can configure semaphore to skip the check process. With ENV variable e.g. SEMAPHORE_ANSIBLE_FORCE_INSTALL_REQUIREMENTS Or in config file e.g. ansible_force_install_requirements

[UglyAnimal]

+1

[Leandro4002]

Hello, I have found a workaround:

At the beginning of your playbook, just run this task:

    - name: Make sure that the latest version of the requirements are installed
      ansible.builtin.command: ansible-galaxy install -r requirements.yml --force
      changed_when: false
      delegate_to: localhost

The ansible-galaxy command does a git clone in a temporary directory of your repo defined in requirements.yml. Then it extracts the collection to /home/semaphore/.ansible/collections/ansible_collections/yourNamespace/yourCollection (you can see the collection path with ansible-galaxy collection list) with a clean structure for an ansible collection (FILES.json, "roles" folder, etc).

Now when ansible runs a task "include_role" for example, it will use what is inside this directory. But, it doesn't know that between the start of the playbook and the execution of the "include_role" task, files have been updated with ansible-galaxy

[rome-legacy]

probably related to this https://github.com/semaphoreui/semaphore/issues/3056