ec2-image-builder-roadmap icon indicating copy to clipboard operation
ec2-image-builder-roadmap copied to clipboard

Ability to escape instances of double curly braces in file content

Open karlkatzke opened this issue 8 months ago • 4 comments

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request Many languages and programs use the double curly braces syntax to denote variables, and there is no way to place files for these other languages (e.g., I can't run docker ps --filter {{ .ID }}) using AWSTOE and Image Builder right now, because there is no documented way to escape a curly brace.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? I saw the response by @austoonz in #76 but I think he misinterpreted the request and closed that ticket prematurely.

name: ConfigureBashRC
description: Configure .bashrc and some user utilities.
schemaVersion: 1.0
phases: 
  - name: build
    steps: 
      - name: AppendToProfile
        action: AppendFile
        inputs: 
          - path: /etc/profile
            content: |
              alias ll='ls -lah'
              alias dexecjob1='docker exec -it -u ec2-user \`docker ps --filter "name=job1-prod" --format "{{ .ID }}" | head -n 1\` bash';

The important part here is that I want the content of the file rendered exactly like that - I DO NOT want AWSTOE to try to render an internal constant or parameter with the name .ID

This would work great if I could escape variables or pass a parameter as an input to not try to render variables in the content. Again, I specifically want {{ .ID }} to get appended to the path in the inputs and do not want it to get substituted.

Are you currently working around this issue? Frankly, really stupid things. I'm writing the file to /tmp, and using sed on it twice to add two sets of curly braces. Something else that works is creating a constant with a set of curly braces.

karlkatzke avatar Jun 13 '25 16:06 karlkatzke

@karlkatzke I wanted to note that I do understand the issue, and what I provided in #76 was a workaround to the problem. I'll provide a similar workaround here for reference so you don't need to write the file and use sed to fix it.

Perhaps I closed that too quickly noting a workaround rather than a true solution. I'll leave this one open until there is an actual solution for escaping the braces.


The workaround for reference of anyone else that finds this.

# The file starts with some text outlining what should be changed.
[ec2-user@ip-172-31-45-144 ~]$ cat testfile
# This is a test file to demonstrate appending content using `AppendFile` with curly braces honored.

Per https://github.com/aws/ec2-image-builder-roadmap/issues/114, this file should contain a line with `"{{ .ID }}"` after invoking the component.


# A similar example but using `constants` to inject the desired `{{ .ID }}` value into the file.
[ec2-user@ip-172-31-45-144 ~]$ cat appendfile.yml
name: ConfigureBashRC
description: Configure .bashrc and some user utilities.
schemaVersion: 1.0
constants:
  - MY_VAR:
      type: string
      value: '{{ .ID }}'
phases:
  - name: build
    steps:
      - name: AppendToProfile
        action: AppendFile
        inputs:
          - path: ./testfile
            content: |
              alias ll='ls -lah'
              alias dexecjob1='docker exec -it -u ec2-user \`docker ps --filter "name=job1-prod" --format "{{ MY_VAR }}" | head -n 1\` bash';

# Invoking the component
[ec2-user@ip-172-31-45-144 ~]$ awstoe run -d appendfile.yml
{
    "executionId": "edbdacb5-487e-11f0-8bf3-0effdfbc1c4b",
    "status": "success",
    "failedStepCount": 0,
    "executedStepCount": 1,
    "ignoredFailedStepCount": 0,
    "skippedStepCount": 0,
    "failureMessage": "",
    "logUrl": "/home/ec2-user/TOE_2025-06-13_17-50-44_UTC-0_edbdacb5-487e-11f0-8bf3-0effdfbc1c4b"
}

# The file at the end, noting `--format "{{ .ID }}"` does exist in it.
[ec2-user@ip-172-31-45-144 ~]$ cat testfile
# This is a test file to demonstrate appending content using `AppendFile` with curly braces honored.

Per https://github.com/aws/ec2-image-builder-roadmap/issues/114, this file should contain a line with `"{{ .ID }}"` after invoking the component.

alias ll='ls -lah'
alias dexecjob1='docker exec -it -u ec2-user \`docker ps --filter "name=job1-prod" --format "{{ .ID }}" | head -n 1\` bash';

austoonz avatar Jun 13 '25 17:06 austoonz

@austoonz But your workaround in #76 didn't work; it still replaced the value of {{ MY_VAR }} with MY_VARIABLE_VALUE without any curly braces. We need the ability to have curly braces available at the run-time of the placed script, not just at image build-time. There are probably other workarounds as well like copying a file containing the code that requires curly braces, but it requires provisioning S3 buckets and building a CI/CD pipeline that will place updated code in those buckets, or running a build system for ansible, or something else extreme. I personally interpreted your response as not a workaround, but as a "WONTFIX" ...

The above example was simplified; I have several dozen of these double-bracket-surrounded values in aliases that we wish to make available to users that need to shell into our ec2 systems. It's also the first time I haven't been able to easily work around this issue by doing something else. Like the other workarounds I mentioned, the only virtue of the workaround is that it exists, but workarounds do not scale and may create other undesirable complexity.

karlkatzke avatar Jun 13 '25 19:06 karlkatzke

The above example of using a constant should work and keep the intended curly braces in the modified file, however yes, this is a workaround and totally agree that it doesn't scale well.

We do have an internal backlog/feature request to solve this, however as of now this work is not currently prioritized.

I'll leave this issue open as the external feature request for others to see, +1, etc.

austoonz avatar Jun 13 '25 19:06 austoonz

If you'd opensource the agent then we could have put up a fix a long time ago

jjshoe avatar Aug 13 '25 19:08 jjshoe