run-on-aws.sh does not work as documented

[serverhorror]

I've tried running ods-devenv/scripts/run-on-aws.sh as described from the master branch and from the 3.x branch.

Both cases exit with a message that no such image is availabe.

To Reproduce Steps to reproduce the behavior:

1. Get a fresh clone
2. Commit is 4771056e7c88fd15ca445e925273aa6c33a345f9
3. Start a container python:3
4. Install AWS CLI v2 as per https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html#cliv2-linux-install
5. Run the run-on-aws.sh command

vscode ➜ /workspaces/ods-core/ods-devenv (master ✗) $ pwd
/workspaces/ods-core/ods-devenv
vscode ➜ /workspaces/ods-core/ods-devenv (master ✗) $ ./scripts/run-on-aws.sh 
--key-name not given, setting to default edp_dublin_keypair.
Neither --security-group-id nor --iam-instance-profile specified. Using default sg-006935bec03a154a1
You are in startup mode using ODS in a box image null.
It looks like we have no AMI image ready for the git ref master!
You may want to specify a branch name with --target-git-ref some-branch-name
Stopping script execution now.

Expected behavior

• Access Account
• Build image in my account
• Deploy Stack via script
• Output access instructions

Screenshots

Affected version (please complete the following information):

• OpenShift: n/a
• OpenDevStack master, 3.x

Log Output (ensure to remove any confidential information like tokens, project names, etc.

vscode ➜ /workspaces/ods-core (master ✗) $ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        .devcontainer/
        ods-devenv/.gitignore
        ods-devenv/awscliv2.zip

nothing added to commit but untracked files present (use "git add" to track)
vscode ➜ /workspaces/ods-core (master ✗) $ aws sts get-caller-identity
{
    "UserId": "A...Q",
    "Account": "6...1",
    "Arn": "arn:aws:iam::6...1:none/of-your-business"
}
vscode ➜ /workspaces/ods-core (master ✗) $ ./scripts/run-on-aws.sh 
bash: ./scripts/run-on-aws.sh: No such file or directory
vscode ➜ /workspaces/ods-core (master ✗) $ cd ods-devenv/
vscode ➜ /workspaces/ods-core/ods-devenv (master ✗) $ ./scripts/run-on-aws.sh 
--key-name not given, setting to default edp_dublin_keypair.
Neither --security-group-id nor --iam-instance-profile specified. Using default sg-006935bec03a154a1
You are in startup mode using ODS in a box image null.
It looks like we have no AMI image ready for the git ref master!
You may want to specify a branch name with --target-git-ref some-branch-name
Stopping script execution now.
vscode ➜ /workspaces/ods-core/ods-devenv (master ✗) $ ```

[clemensutschig]

@s2oBCN @serverhorror I guess the AMI is not public, that's why you can't see it ... :( https://github.com/opendevstack/ods-core/blob/master/ods-devenv/packer/CentOS2ODSBox.json

@stitakis I don't see this tagging into a specific account?! ... hmmm

I guess what's missing is (from : https://www.packer.io/docs/builders/amazon/ebs)

ami_groups ([]string) - A list of groups that have access to launch the resulting AMI(s). By default no groups have permission to launch the AMI. all will make the AMI publicly accessible.

[serverhorror]

There are a few other places that rely on non public AMIs, this is just a note to try and get this to work quickly.

• https://github.com/opendevstack/ods-core/blob/4771056e7c88fd15ca445e925273aa6c33a345f9/ods-devenv/packer/create_ods_box_image.sh#L185-L188

  A nicer filter might be this:

  aws ec2 describe-images \
    --owners 125523088429 \
    --filters \
      "Name=name,Values=CentOS 7*" \
      "Name=architecture,Values=x86_64" \
    --query 'Images[?contains(Name, `CentOS 7`)].{Name:Name,ImageId:ImageId,Architecture:Architecture,CreationDate:CreationDate}|sort_by(@, &CreationDate)[-1]'
  

  The documentation about the owner ID can be found her: https://wiki.centos.org/Cloud/AWS

• https://github.com/opendevstack/ods-core/blob/4771056e7c88fd15ca445e925273aa6c33a345f9/ods-devenv/packer/create_ods_box_image.sh#L112-L116

• https://github.com/opendevstack/ods-core/blob/4771056e7c88fd15ca445e925273aa6c33a345f9/ods-devenv/packer/create_ods_box_image.sh#L185-L188

[clemensutschig]

@serverhorror did you also try a build of the AMI - and that failed as well?

[serverhorror]

I'm working my way thru the scripts.

It seems a bit unexpected to have ISO and VMWare expectations in there rather than just starting from the official CentOS AWS images.

I'll make sure that works before opening a PR

[serverhorror]

Am I missing something obvious? That's the only place I found where the openshift user is created.

It seems that one has to have VMWare to build a base CentOS box that is suitable to build the ODS AMI.

• https://github.com/opendevstack/ods-core/blob/4771056e7c88fd15ca445e925273aa6c33a345f9/ods-devenv/packer/kickstart.cfg#L33
• https://github.com/opendevstack/ods-core/blob/4771056e7c88fd15ca445e925273aa6c33a345f9/ods-devenv/packer/CentOS_BaseImage_VMWare.json#L13-L16

[clemensutschig]

@serverhorror can you try with Master.. the latest image should now be public.. ;-) i added it hard to Master... a we finally got it stable again

[clemensutschig]

@serverhorror - can you try again / master and 4.x should be available :)

[tbugfinder]

ods-devenv was removed, so this could be closed.