astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

DuckDB integration does not use published wheel on Astro runtime

Open jeremybeard opened this issue 2 years ago • 0 comments

Describe the bug When building an Astro runtime image with the Astro SDK DuckDB submodule the published DuckDB wheel is not used and instead it is compiled locally, which takes many minutes and also requires installing the gcc compiler on the image.

Version

  • Astro SDK: 1.6.1
  • Astro runtime: 8.4.0
  • OS: macOS

To Reproduce Steps to reproduce the behavior:

  1. Create a new Astro CLI project with astro dev init
  2. Modify requirements.txt to have the line astro-sdk-python[duckdb]==1.6.1
  3. Modify packages.txt to have the line g++ (gcc is required to build the wheel)
  4. Start the Airflow instance with astro dev start
  5. See that it takes ~5 minutes to build the DuckDB wheel

Expected behavior The published DuckDB wheels to be used instead of building it locally.

jeremybeard avatar Jun 08 '23 13:06 jeremybeard