Cluster setup failure: "Another app is currently holding the yum lock"

[JohnTortugo]

I'm trying to follow a basic example of creating a cluster using flintrock but I'm strugling to get past the error below. Seems like some previous step is still installing something in the nodes and the setup process just bailed out? Any help will be much appreciated.

moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
  "cipher": algorithms.TripleDES,
/Users/dcsl/.local/pipx/venvs/flintrock/lib/python3.13/site-packages/paramiko/transport.py:259: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from cryptography.hazmat.primitives.ciphers.algorithms in 48.0.0.
  "class": algorithms.TripleDES,
2025-07-30 21:10:33,957 - flintrock.flintrock - WARNING - Spark download source appears to point to a file, not a directory. Flintrock will not try to determine the correct file to download based on the Spark and Hadoop versions.
2025-07-30 21:10:33,958 - flintrock.flintrock - INFO  - Launching 1 master and 2 slaves...
2025-07-30 21:10:48,896 - flintrock.ec2       - DEBUG - 3 instances not in state 'running': 'i-098e94562d0ceeb66', 'i-0907a0f89c694e285', 'i-07a5950c3723d145a', ...
2025-07-30 21:10:52,616 - flintrock.ec2       - DEBUG - 1 instances not in state 'running': 'i-0907a0f89c694e285', ...
2025-07-30 21:10:57,378 - flintrock.ssh       - INFO  - [54.242.207.133] SSH online.
2025-07-30 21:10:58,521 - flintrock.core      - INFO  - [54.242.207.133] Configuring ephemeral storage...
2025-07-30 21:10:59,560 - flintrock.ssh       - DEBUG - [54.196.174.114] SSH timeout.
2025-07-30 21:10:59,560 - flintrock.ssh       - DEBUG - [54.145.220.172] SSH timeout.
2025-07-30 21:11:05,423 - flintrock.ssh       - INFO  - [54.196.174.114] SSH online.
2025-07-30 21:11:05,426 - flintrock.ssh       - INFO  - [54.145.220.172] SSH online.
2025-07-30 21:11:06,583 - flintrock.core      - INFO  - [54.196.174.114] Configuring ephemeral storage...
2025-07-30 21:11:06,603 - flintrock.core      - INFO  - [54.145.220.172] Configuring ephemeral storage...
2025-07-30 21:11:35,847 - flintrock.core      - INFO  - [54.242.207.133] Installing Adoptium OpenJDK Java 25...
2025-07-30 21:11:47,170 - flintrock.core      - INFO  - [54.145.220.172] Installing Adoptium OpenJDK Java 25...
2025-07-30 21:11:53,844 - flintrock.core      - INFO  - [54.196.174.114] Installing Adoptium OpenJDK Java 25...
Do you want to terminate the 3 instances created by this operation? [Y/n]: n
[54.196.174.114] Existing lock /var/run/yum.pid: another copy is running as pid 8049.
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory :  52 M RSS (286 MB VSZ)
    Started: Thu Jul 31 04:11:52 2025 - 00:03 ago
    State  : Sleeping, pid: 8049
Existing lock /var/run/yum.pid: another copy is running as pid 8263.
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory :  33 M RSS (325 MB VSZ)
    Started: Thu Jul 31 04:11:56 2025 - 00:01 ago
    State  : Sleeping, pid: 8263
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory : 206 M RSS (498 MB VSZ)
    Started: Thu Jul 31 04:11:56 2025 - 00:03 ago
    State  : Running, pid: 8263
Another app is currently holding the yum lock; waiting for it to exit...
  The other application is: yum
    Memory : 220 M RSS (511 MB VSZ)
    Started: Thu Jul 31 04:11:56 2025 - 00:05 ago
    State  : Running, pid: 8263
Error: Nothing to do

Config used:

services:
  spark:
    version: 3.3.0
    download-source: "https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz"
  hdfs:
    version: 3.3.4
    download-source: "https://archive.apache.org/dist/hadoop/core/hadoop-3.3.4/"
provider: ec2
providers:
  ec2:
    key-name: name-of-key
    identity-file: path-to-key
    instance-type: c5d.9xlarge
    region: us-east-1
    ami: ami-0cadefa35b0684c01
    user: ec2-user
    security-groups:
      - default
    instance-profile-name: EMR_EC2_DefaultRole
    min-root-ebs-size-gb: 20
    tenancy: default  # default | dedicated
    ebs-optimized: yes  # yes | no
    instance-initiated-shutdown-behavior: terminate  # terminate | stop

launch:
  num-slaves: 2
  install-hdfs: True
  install-spark: True
  java-version: 25

debug: true

[nchammas]

Could you share the full Flintrock config, version, and command line invocation you are using? That way I can try to replicate the problem.

[JohnTortugo]

Thank you for getting back to me. I updated my original post with the config; unfortunately, the AMI that I'm using is "private".

[nchammas]

If you try the default AMI and Java 11 or 17 does it work or do you have the same problem?

[JohnTortugo]

It fails saying that the AMI doesn't exist in the region.

[nchammas]

You'll want to use us-east-1 like shown in the config template.

If you can't use that region, I suggest trying the default Amazon Linux 2 AMI for whatever region you normally use. The goal is to eliminate some variables to narrow down the source of the problem. In your case, the main variables are the private AMI and the Java version.