skypilot icon indicating copy to clipboard operation
skypilot copied to clipboard

[Setup] `sky_workdir` Structure

Open michaelzhiluo opened this issue 2 years ago • 0 comments

Description

Related to #388. When Tianjun tried to set his environment variables and setup his work directory, these factors depended on the work directory being named nce_spede.

This, however, conflicts with how Sky treats a work directory. If a work directory is ~/my_workdirectory/*, then on remote, Sky would move the contents to ~/sky_workdir/*. This kills the information fo the work directory name my_workdirectory.

So far, this affected all our users. According to Romil,

From kevin's doc: "Needed prompt to find his files in ‘sky_workdir’ when he ssh’d into his vm."

From Daniel's doc: "sky_workdir being named sky_workdir prevents people from custom installing their packages (e.g. pip install -e) #388 “This is not what I expected to happen, when I say my workdir is qm9 I expect the workdir to be named qm9 on the cluster, not in sky_workdir”"

Fix

There was a good discussion in #388. Continuing off of that discussion, some possible suggestions were

  1. Do ~/sky_workdir/my_workdirectory
  2. Symlink the absolute path /Users/michaelluo/my_workdirectory to ~/sky_workdir on remote

michaelzhiluo avatar Mar 25 '22 18:03 michaelzhiluo