skypilot
skypilot copied to clipboard
[Setup] `sky_workdir` Structure
Description
Related to #388. When Tianjun tried to set his environment variables and setup his work directory, these factors depended on the work directory being named nce_spede
.
This, however, conflicts with how Sky treats a work directory. If a work directory is ~/my_workdirectory/*
, then on remote, Sky would move the contents to ~/sky_workdir/*
. This kills the information fo the work directory name my_workdirectory
.
So far, this affected all our users. According to Romil,
From kevin's doc: "Needed prompt to find his files in ‘sky_workdir’ when he ssh’d into his vm."
From Daniel's doc: "sky_workdir being named sky_workdir prevents people from custom installing their packages (e.g. pip install -e) #388 “This is not what I expected to happen, when I say my workdir is qm9 I expect the workdir to be named qm9 on the cluster, not in sky_workdir”"
Fix
There was a good discussion in #388. Continuing off of that discussion, some possible suggestions were
- Do
~/sky_workdir/my_workdirectory
- Symlink the absolute path
/Users/michaelluo/my_workdirectory
to~/sky_workdir
on remote