elementary icon indicating copy to clipboard operation
elementary copied to clipboard

edr 'dbt deps' step running into Windows file path length limits

Open JennaFlan2426 opened this issue 1 year ago • 0 comments

Describe the bug edr commands such as 'edr report' fail with a FileNotFound error regarding "dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql". (dbt-data-reliability-0.14.0 folder name may change with different Elementary version numbers)

Failure occurs during the 'dbt deps' preparatory step when EDR attempts to install required packages within the folder structure of the Elementary Python module. When I run "dbt deps --project-dir " the install of dbt-utils package is successful (therefore I can confirm write permissions to the directory structure) but install of elementary package fails during untar of the above file.

Complete path from Python environment folder is "\Lib\site-packages\elementary\monitor\dbt_project\dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql" which is 204 characters long and risks exceeding Windows 256 character filepath limits. Therefore, untar of the elementary package in the desired location fails with a misleading "File Not Found" error (the problem actually is untar can't WRITE the file because the path is too long).

Possible cause of https://github.com/elementary-data/elementary/issues/1261 which was closed as a "local environment" issue. I wish to leave this as at least an information breadcrumb for other users... and also ask the team to consider filepath depth in future naming choices for those of us stuck with Windows. Thanks!

To Reproduce Steps to reproduce the behavior:

  1. Create a Python environment with a Windows file path greater than 52 characters, such as "C:\Users\JenniferFlanagan\GitHub\analytics-dbt-project.dbtenv"
  2. pip install dbt and elementary
  3. Navigate to <environment_path>\Lib\site-packages\elementary\monitor\dbt_project\
  4. Attempt to run dbt deps (edr commands do this as a preparatory step)
  5. Logs should indicate that untar of the elementary package fails on file "\dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql" with a "File Not Found" error
  6. To confirm, retry these steps using a shorter environment path such as "C:\Users\JenniferFlanagan.testenv" (34 characters; long path will total 238), dbt deps will run without error

Expected behavior EDR commands succeed on dbt deps step

Environment (please complete the following information):

  • Elementary CLI (edr) version: 0.14.0
  • Elementary dbt package version: 0.14.1
  • dbt version: 1.7.9 (also tested after 1.8.5 upgrade)
  • Data warehouse: Postgres
  • Infrastructure details: Windows OS

Additional context Running 'dbt deps' manually from the <environment_path>\Lib\site-packages\elementary\monitor\dbt_project\ also resolved a less-obvious "failed to run dbt command" error encountered while trying to get 'edr report' to succeed.

Would you be willing to contribute a fix for this issue? Willing, not sure about able.

JennaFlan2426 avatar Aug 23 '24 16:08 JennaFlan2426