preLaunchTask 'Compile English translations' fails with OSError "Cannot call rmtree on a symbolic link"

[srescio]

The problem

Launching the Home Assistant debug session with or without pip is stopped due to errors in preLaunchTask 'Compile English translations'

it fails almost 90% of times, randomically on different components each time, claiming the rmtree is being invoked not on a real download dir but on a symlink

https://github.com/user-attachments/assets/c81a6e42-1216-466c-a7c2-3fd36a7c9a68

What version of Home Assistant Core has the issue?

2024.10.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

Compile English translations task

Link to integration documentation on our website

https://github.com/home-assistant/core/blob/eac930ad7faba2f8d7bc40c899662ec6e04f8bb2/script/translations/develop.py#L108-L117

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/workspaces/core/script/translations/__main__.py", line 30, in <module>
    sys.exit(main())
             ^^^^^^
  File "/workspaces/core/script/translations/__main__.py", line 25, in main
    return module.run()
           ^^^^^^^^^^^^
  File "/workspaces/core/script/translations/develop.py", line 128, in run
    run_single(translations, flattened_translations, integration)
  File "/workspaces/core/script/translations/develop.py", line 109, in run_single
    rmtree(str(download.DOWNLOAD_DIR))
  File "/usr/local/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/usr/local/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/usr/local/lib/python3.12/shutil.py", line 680, in _rmtree_safe_fd
    raise OSError("Cannot call rmtree on a symbolic link")
OSError: [Errno None] None: '/workspaces/core/build/translations-download'

Additional information

artificially slowing down the speed of the run_single function by placing a sleep command at line https://github.com/home-assistant/core/blob/eac930ad7faba2f8d7bc40c899662ec6e04f8bb2/script/translations/develop.py#L107 makes the issue stop, also changing the strategy by not going in error if the directory already exist, and relying on the .write_text method to override the json file that has alway the same filename also makes the issue disappear, this was attempted in PR https://github.com/home-assistant/core/pull/127675

[srescio]

worth noting that the directory being destroyed at line 109 and immediately recreated at line 111 is always the same path, it does not change per component, also it always contains the same filename en.json but with different content each time additionally the dir check is always true, including the last component where it fails, so the error is actually masking the real root cause:

https://github.com/user-attachments/assets/0b59a727-2b1d-44a6-979a-8355737b69c4

python3 -m script.translations develop --all 

Generating translations for buienradar
/workspaces/core/build/translations-download True
Generating translations for smlight
/workspaces/core/build/translations-download True
Generating translations for tesla_wall_connector
/workspaces/core/build/translations-download True
Generating translations for syncthing
/workspaces/core/build/translations-download True
Generating translations for blink
/workspaces/core/build/translations-download True
Generating translations for deako
/workspaces/core/build/translations-download True
Generating translations for demo
/workspaces/core/build/translations-download True
Generating translations for deconz
/workspaces/core/build/translations-download True
Generating translations for statistics
/workspaces/core/build/translations-download True
Generating translations for tedee
/workspaces/core/build/translations-download True
Generating translations for soundtouch
/workspaces/core/build/translations-download True
Generating translations for aseko_pool_live
/workspaces/core/build/translations-download True
Generating translations for zeversolar
/workspaces/core/build/translations-download True
Generating translations for philips_js
/workspaces/core/build/translations-download True
Generating translations for min_max
/workspaces/core/build/translations-download True
Generating translations for nextcloud
/workspaces/core/build/translations-download True
Generating translations for ambient_station
/workspaces/core/build/translations-download True
Generating translations for siren
/workspaces/core/build/translations-download True
Generating translations for geonetnz_quakes
/workspaces/core/build/translations-download True
Generating translations for ads
/workspaces/core/build/translations-download True
Generating translations for here_travel_time
/workspaces/core/build/translations-download True
Generating translations for geo_json_events
/workspaces/core/build/translations-download True
Generating translations for powerwall
/workspaces/core/build/translations-download True
Generating translations for flipr
/workspaces/core/build/translations-download True
Generating translations for brother
/workspaces/core/build/translations-download True
Generating translations for oncue
/workspaces/core/build/translations-download True
Generating translations for hydrawise
/workspaces/core/build/translations-download True
Generating translations for foscam
/workspaces/core/build/translations-download True
Generating translations for fully_kiosk
/workspaces/core/build/translations-download True
Generating translations for renault
/workspaces/core/build/translations-download True
Generating translations for notify
/workspaces/core/build/translations-download True
Generating translations for huisbaasje
/workspaces/core/build/translations-download True
Generating translations for meteo_france
/workspaces/core/build/translations-download True
Generating translations for forecast_solar
/workspaces/core/build/translations-download True
Generating translations for apsystems
/workspaces/core/build/translations-download True
Generating translations for honeywell
/workspaces/core/build/translations-download True
Generating translations for pvoutput
/workspaces/core/build/translations-download True
Generating translations for nissan_leaf
/workspaces/core/build/translations-download True
Generating translations for izone
/workspaces/core/build/translations-download True
Generating translations for starlink
/workspaces/core/build/translations-download True
Generating translations for dynalite
/workspaces/core/build/translations-download True
Generating translations for ffmpeg
/workspaces/core/build/translations-download True
Generating translations for control4
/workspaces/core/build/translations-download True
Generating translations for locative
/workspaces/core/build/translations-download True
Generating translations for iotty
/workspaces/core/build/translations-download True
Generating translations for smartthings
/workspaces/core/build/translations-download True
Generating translations for starline
/workspaces/core/build/translations-download True
Generating translations for utility_meter
/workspaces/core/build/translations-download True
Generating translations for snooz
/workspaces/core/build/translations-download True
Generating translations for dominos
/workspaces/core/build/translations-download True
Generating translations for sun
/workspaces/core/build/translations-download True
Generating translations for twentemilieu
/workspaces/core/build/translations-download True
Generating translations for cpuspeed
/workspaces/core/build/translations-download True
Generating translations for simulated
/workspaces/core/build/translations-download True
Generating translations for meteoclimatic
/workspaces/core/build/translations-download True
Generating translations for holiday
/workspaces/core/build/translations-download True
Generating translations for persistent_notification
/workspaces/core/build/translations-download True
Generating translations for ruuvi_gateway
/workspaces/core/build/translations-download True
Generating translations for motionblinds_ble
/workspaces/core/build/translations-download True
Generating translations for derivative
/workspaces/core/build/translations-download True
Generating translations for dexcom
/workspaces/core/build/translations-download True
Generating translations for justnimbus
/workspaces/core/build/translations-download True
Generating translations for energenie_power_sockets
/workspaces/core/build/translations-download True
Generating translations for tod
/workspaces/core/build/translations-download True
Generating translations for intellifire
/workspaces/core/build/translations-download True
Generating translations for flume
/workspaces/core/build/translations-download True
Generating translations for verisure
/workspaces/core/build/translations-download True
Generating translations for dsmr_reader
/workspaces/core/build/translations-download True
Generating translations for daikin
/workspaces/core/build/translations-download True
Generating translations for schedule
/workspaces/core/build/translations-download True
Generating translations for lacrosse_view
/workspaces/core/build/translations-download True
Generating translations for ping
/workspaces/core/build/translations-download True
Generating translations for vallox
/workspaces/core/build/translations-download True
Generating translations for nws
/workspaces/core/build/translations-download True
Generating translations for ld2410_ble
/workspaces/core/build/translations-download True
Generating translations for vera
/workspaces/core/build/translations-download True
Generating translations for konnected
/workspaces/core/build/translations-download True
Generating translations for mobile_app
/workspaces/core/build/translations-download True
Generating translations for yolink
/workspaces/core/build/translations-download True
Generating translations for lcn
/workspaces/core/build/translations-download True
Generating translations for litterrobot
/workspaces/core/build/translations-download True
Generating translations for garages_amsterdam
/workspaces/core/build/translations-download True
Generating translations for dlink
/workspaces/core/build/translations-download True
Generating translations for fyta
/workspaces/core/build/translations-download True
Generating translations for goodwe
/workspaces/core/build/translations-download True
Generating translations for switchbee
/workspaces/core/build/translations-download True
Generating translations for homekit
/workspaces/core/build/translations-download True
Generating translations for coolmaster
/workspaces/core/build/translations-download True
Generating translations for frontier_silicon
/workspaces/core/build/translations-download True
Generating translations for home_connect
/workspaces/core/build/translations-download True
Generating translations for imap
/workspaces/core/build/translations-download True
Generating translations for discord
/workspaces/core/build/translations-download True
Generating translations for rest_command
/workspaces/core/build/translations-download True
Generating translations for progettihwsw
/workspaces/core/build/translations-download True
Generating translations for calendar
/workspaces/core/build/translations-download True
Generating translations for opower
/workspaces/core/build/translations-download True
Generating translations for time_date
/workspaces/core/build/translations-download True
Generating translations for v2c
/workspaces/core/build/translations-download True
Generating translations for panel_iframe
/workspaces/core/build/translations-download True
Generating translations for hyperion
/workspaces/core/build/translations-download True
Generating translations for rituals_perfume_genie
/workspaces/core/build/translations-download True
Generating translations for iotawatt
/workspaces/core/build/translations-download True
Generating translations for wake_on_lan
/workspaces/core/build/translations-download True
Generating translations for tankerkoenig
/workspaces/core/build/translations-download True
Generating translations for jewish_calendar
/workspaces/core/build/translations-download True
Generating translations for ccm15
/workspaces/core/build/translations-download True
Generating translations for logger
/workspaces/core/build/translations-download True
Generating translations for lamarzocco
/workspaces/core/build/translations-download True
Generating translations for dsmr
/workspaces/core/build/translations-download True
Generating translations for local_file
/workspaces/core/build/translations-download True
Generating translations for elgato
/workspaces/core/build/translations-download True
Generating translations for history_stats
/workspaces/core/build/translations-download True
Generating translations for hvv_departures
/workspaces/core/build/translations-download True
Generating translations for nibe_heatpump
/workspaces/core/build/translations-download True
Generating translations for vulcan
/workspaces/core/build/translations-download True
Generating translations for google_sheets
/workspaces/core/build/translations-download True
Generating translations for waze_travel_time
/workspaces/core/build/translations-download True
Generating translations for islamic_prayer_times
/workspaces/core/build/translations-download True
Generating translations for fireservicerota
/workspaces/core/build/translations-download True
Generating translations for opengarage
/workspaces/core/build/translations-download True
Generating translations for html5
/workspaces/core/build/translations-download True
Generating translations for airzone
/workspaces/core/build/translations-download True
Generating translations for iammeter
/workspaces/core/build/translations-download True
Generating translations for flick_electric
/workspaces/core/build/translations-download True
Generating translations for wallbox
/workspaces/core/build/translations-download True
Generating translations for frontend
/workspaces/core/build/translations-download True
Generating translations for kostal_plenticore
/workspaces/core/build/translations-download True
Generating translations for system_health
/workspaces/core/build/translations-download True
Generating translations for jellyfin
/workspaces/core/build/translations-download True
Generating translations for telegram_bot
/workspaces/core/build/translations-download True
Generating translations for homeassistant
/workspaces/core/build/translations-download True
Generating translations for aquacell
/workspaces/core/build/translations-download True
Generating translations for mystrom
/workspaces/core/build/translations-download True
Generating translations for moat
/workspaces/core/build/translations-download True
Generating translations for waqi
/workspaces/core/build/translations-download True
Generating translations for twinkly
/workspaces/core/build/translations-download True
Generating translations for kegtron
/workspaces/core/build/translations-download True
Generating translations for sma
/workspaces/core/build/translations-download True
Generating translations for lastfm
/workspaces/core/build/translations-download True
Generating translations for duckdns
/workspaces/core/build/translations-download True
Generating translations for vizio
/workspaces/core/build/translations-download True
Generating translations for improv_ble
/workspaces/core/build/translations-download True
Generating translations for freedompro
/workspaces/core/build/translations-download True
Generating translations for update
/workspaces/core/build/translations-download True
Generating translations for lametric
/workspaces/core/build/translations-download True
Generating translations for assist_pipeline
/workspaces/core/build/translations-download True
Generating translations for logbook
/workspaces/core/build/translations-download True
Generating translations for duke_energy
/workspaces/core/build/translations-download True
Generating translations for adguard
/workspaces/core/build/translations-download True
Generating translations for broadlink
/workspaces/core/build/translations-download True
Generating translations for canary
/workspaces/core/build/translations-download True
Generating translations for system_bridge
/workspaces/core/build/translations-download True
Generating translations for srp_energy
/workspaces/core/build/translations-download True
Generating translations for lutron_caseta
/workspaces/core/build/translations-download True
Generating translations for zoneminder
/workspaces/core/build/translations-download True
Generating translations for websocket_api
/workspaces/core/build/translations-download True
Generating translations for melcloud
/workspaces/core/build/translations-download True
Generating translations for dialogflow
/workspaces/core/build/translations-download True
Generating translations for duotecno
/workspaces/core/build/translations-download True
Generating translations for radio_browser
/workspaces/core/build/translations-download True
Generating translations for elkm1
/workspaces/core/build/translations-download True
Generating translations for assist_satellite
/workspaces/core/build/translations-download True
Generating translations for amcrest
/workspaces/core/build/translations-download True
Generating translations for plant
/workspaces/core/build/translations-download True
Generating translations for local_ip
/workspaces/core/build/translations-download True
Generating translations for ifttt
/workspaces/core/build/translations-download True
Generating translations for mazda
/workspaces/core/build/translations-download True
Generating translations for qbittorrent
/workspaces/core/build/translations-download True
Generating translations for google_tasks
/workspaces/core/build/translations-download True
Generating translations for universal
/workspaces/core/build/translations-download True
Generating translations for mastodon
/workspaces/core/build/translations-download True
Generating translations for file
/workspaces/core/build/translations-download True
Generating translations for cert_expiry
/workspaces/core/build/translations-download True
Generating translations for crownstone
/workspaces/core/build/translations-download True
Generating translations for simplisafe
/workspaces/core/build/translations-download True
Generating translations for led_ble
/workspaces/core/build/translations-download True
Generating translations for xbox
/workspaces/core/build/translations-download True
Generating translations for debugpy
/workspaces/core/build/translations-download True
Generating translations for dnsip
/workspaces/core/build/translations-download True
Generating translations for sms
/workspaces/core/build/translations-download True
Generating translations for trafikverket_train
/workspaces/core/build/translations-download True
Generating translations for apple_tv
/workspaces/core/build/translations-download True
Generating translations for dte_energy_bridge
/workspaces/core/build/translations-download True
Generating translations for poolsense
/workspaces/core/build/translations-download True
Generating translations for workday
/workspaces/core/build/translations-download True
Generating translations for trafikverket_ferry
/workspaces/core/build/translations-download True
Generating translations for nfandroidtv
/workspaces/core/build/translations-download True
Generating translations for rainbird
/workspaces/core/build/translations-download True
Generating translations for iss
/workspaces/core/build/translations-download True
Generating translations for nice_go
/workspaces/core/build/translations-download True
Generating translations for wmspro
/workspaces/core/build/translations-download True
Generating translations for wled
/workspaces/core/build/translations-download True
Generating translations for energy
/workspaces/core/build/translations-download True
Generating translations for auth
/workspaces/core/build/translations-download True
Generating translations for rympro
/workspaces/core/build/translations-download True
Generating translations for simplepush
/workspaces/core/build/translations-download True
Generating translations for wolflink
/workspaces/core/build/translations-download True
Generating translations for skybell
/workspaces/core/build/translations-download True
Generating translations for flexit_bacnet
/workspaces/core/build/translations-download True
Generating translations for group
/workspaces/core/build/translations-download True
Generating translations for nyt_games
/workspaces/core/build/translations-download True
Generating translations for accuweather
/workspaces/core/build/translations-download True
Generating translations for volumio
/workspaces/core/build/translations-download True
Generating translations for monarch_money
/workspaces/core/build/translations-download True
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/workspaces/core/script/translations/__main__.py", line 30, in <module>
    sys.exit(main())
             ^^^^^^
  File "/workspaces/core/script/translations/__main__.py", line 25, in main
    return module.run()
           ^^^^^^^^^^^^
  File "/workspaces/core/script/translations/develop.py", line 130, in run
    run_single(translations, flattened_translations, integration)
  File "/workspaces/core/script/translations/develop.py", line 111, in run_single
    rmtree(str(download.DOWNLOAD_DIR))
  File "/usr/local/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/usr/local/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/usr/local/lib/python3.12/shutil.py", line 680, in _rmtree_safe_fd
    raise OSError("Cannot call rmtree on a symbolic link")
OSError: [Errno None] None: '/workspaces/core/build/translations-download'

if the intention of the code 108-117 is to ensure that the directory exists before attempting to write the file, then having mkdir not error if the directory is already there avoids this scenario likely created by the too quick delete/create operations also the .write_text method at line 113 is going to overwrite the file if already present anyway so the individual integration translations will be found when write_integration_translations at line 117 runs

[srescio]

run with a sleep command, its always overwriting the same file in the same dir over and over, given the write_text does the overwrite it feels like an overkill? unless this strategy of recreating each time the dir accomplishes something not explicit that I am missing?

https://github.com/user-attachments/assets/2e807936-a77d-4067-86b6-1e201dcbe15b

[srescio]

@home-assistant/core fyi (i see the team in codeowners however the handler is not suggested from chat so not sure tagging is working 😅)

[srescio]

also @frenck as we had a first discussion on the topic, you mentioned that the dir may be not there and clean up reason although i miss how these are currently addressed by the code in question?

[tudborg]

The exception message isn't particularly helpful.
This is where the exception is raised:

https://github.com/python/cpython/blob/0c43d60e7d0edb2fd8f996091d96ba4e8350e72a/Lib/shutil.py#L674-L676

Underlying issue seems to be that the inode of DOWNLOAD_DIR actually changes during rmtree.

I set up rmtree like this to verify

lstat_before = os.lstat(DOWNLOAD_DIR)

def _onexc(func, path, exc_info):
    print(f"onexc: #{func} #{path} {exc_info}")
    lstat_after = os.lstat(DOWNLOAD_DIR)
    print("before:", lstat_before, "\n", "after:", lstat_after)

rmtree(str(DOWNLOAD_DIR), onexc=_onexc)

And I get something like this back

onexc: #<function islink at 0xffff97b0c4a0> #/workspaces/home-assistant-core/build/translations-download [Errno None] None: '/workspaces/home-assistant-core/build/translations-download'
before: os.stat_result(st_mode=16877, st_ino=41960, st_dev=37, st_nlink=3, st_uid=1000, st_gid=1000, st_size=96, st_atime=1728566909, st_mtime=1728566909, st_ctime=1728566909) 
 after: os.stat_result(st_mode=16877, st_ino=41962, st_dev=37, st_nlink=3, st_uid=1000, st_gid=1000, st_size=96, st_atime=1728566909, st_mtime=1728566909, st_ctime=1728566909)

I have no idea why this happens. Issue with the FS when using devcontainers maybe? I'm on MacOS 14.6.1 using vscode 1.94.0 and docker desktop 4.34.0

Anyway, if the goal of the code is to clean up the existing language files, doing

    if download.DOWNLOAD_DIR.is_dir():
        # instead of `rmtree(str(download.DOWNLOAD_DIR))`:
        for lang_file in download.DOWNLOAD_DIR.glob("*.json"):
            lang_file.unlink()
    else:
        download.DOWNLOAD_DIR.mkdir(parents=True)

would do the trick and solves the issue.

[srescio]

@frenck what do you think of the above? if thats ok @tudborg could raise a PR with it

[srescio]

@frenck ping for this topic 👀

[srescio]

@frenck ping again, tested now, @tudborg fix works good for me, any concern with implementing his suggestion? see video attached 📹 fyi @MartinHjelmare do you know who else could oversee/approve this change? I could not find an handler to tag relevant owners

https://github.com/user-attachments/assets/d503c637-dd50-4c33-b800-0da93976bf4b

[tudborg]

@srescio feel free to PR my change. I have no ownership feeling over 5 lines of code :)

It will be much easier for @frenck if the code is just ready to be 👍/👎

[srescio]

@frenck pls 🙇🏻‍♂️

[MartinHjelmare]

Everyone, please stop pinging people. Be patient. There's an open PR. Someone will review it eventually.