astro-cli icon indicating copy to clipboard operation
astro-cli copied to clipboard

Variables/Connections/Pools specified in `airflow_settings.yaml` can't be imported or exported with Podman

Open joaomsa opened this issue 4 months ago • 0 comments

Describe the bug Values in airflow_settings.yaml are not respected when using podman as the container binary

When trying to use the astro dev object import or astro dev object export commands while running with podman the commands fail without displaying any error messages.

For the first command, although astro indicates that variables have been successfully imported, the airflow command was never actually run. For the latter we have a fatal error.

These astro commands do not respect the container.binary: podman option specified in the .astro/config.yaml file and instead hardcode the docker executable.

https://github.com/astronomer/astro-cli/blob/main/docker/docker.go#L21

which podman

/opt/podman/bin/podman

which docker

docker not found

cat .astro/config.yaml

project:
    name: test-bug
container:
    binary: podman

cat airflow_settings.yaml

airflow:
    variables:
        - variable_name: hello
          variable_value: world

astro dev object import

Added Variable: hello

Airflow objects created from settings file

astro dev object export

panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
github.com/astronomer/astro-cli/settings.ExportPools({0x140001159c0?, 0x15?})
        github.com/astronomer/astro-cli/settings/settings.go:571 +0x748
github.com/astronomer/astro-cli/settings.Export({0x140001159c0, 0x40}, {0x102f1869a?, 0x0?}, 0x2, 0x1, 0x1, 0x1)
        github.com/astronomer/astro-cli/settings/settings.go:435 +0x7c
github.com/astronomer/astro-cli/airflow.(*DockerCompose).ExportSettings(0x14000262000, {0x102f1869a, 0x15}, {0x102f014ad, 0x4}, 0x0?, 0x0?, 0x0?, 0x0)
        github.com/astronomer/astro-cli/airflow/docker.go:1156 +0x150
github.com/astronomer/astro-cli/cmd.airflowSettingsExport(0x14000550f08, {0x103f0ed60?, 0x4?, 0x102f015fd?})
        github.com/astronomer/astro-cli/cmd/airflow.go:930 +0x108
github.com/spf13/cobra.(*Command).execute(0x14000550f08, {0x103f0ed60, 0x0, 0x0})
        github.com/spf13/[email protected]/command.go:940 +0x648
github.com/spf13/cobra.(*Command).ExecuteC(0x140001e6908)
        github.com/spf13/[email protected]/command.go:1068 +0x320
github.com/spf13/cobra.(*Command).Execute(0x1034f2160?)
        github.com/spf13/[email protected]/command.go:992 +0x1c
main.main()
        github.com/astronomer/astro-cli/main.go:21 +0x38

astro dev run variables export -

Running: airflow variables export -
[2024-09-26T09:43:56.575+0000] {plugin.py:32} WARNING - Astro managed secrets backend is disabled
{}

What CLI Version did you experience this bug?

astro version

Astro CLI Version: 1.29.0

This CLI bug is related to which Astronomer Platform?

  • [x] Astro
  • [ ] Software
  • [ ] None/Unknown

What Operating System is the above CLI installed on?

🪜 Steps To Reproduce

Attempt to specify variables/connections/pools in airflow_settings.yaml without the docker client installed (using an alternative binary like podman)

📸 Screenshots

joaomsa avatar Sep 26 '24 09:09 joaomsa