Permissions for multiuser install

[nwolthuis5]

We have a marvin install on our linux system at Haverford which runs a Jupyter Hub. We have encountered permissions issues with some of the files in the standard directory structure (e.g. the log files in /tmp/sdss_access and also download of the drpall). Is there a way to make these default to be made with rwx permissons for all users on our system?

[karenlmasters]

Just to comment on this that I helped Nathan write this issue. I'm pretty sure we can come up with a local work around, but might be something affecting others installing for multiusers.

[joelbrownstein]

Hi Karen and Nathan,

The rsync that does the download copies the file permissions of the file, so for public data we make sure it is world readable. However for the proprietary mangawork, it is the opposite, and downloaded files cannot be world readable by default.

Cheers, Joel

[karenlmasters]

Hey Joel. Yeah I see where you're coming from.... we'll have to figure out a local solution then. Although we're also having trouble with Marvin log files which go into sdss_access. Could those be world readable, or do they contain proprietary data in some examples?

[joelbrownstein]

Hi Karen,

The permissions of the sdss_access log files would actually be determined by the user that ran the download via their default umask. Can you ask what is the response from the user typing umask at the command line? If it’s too restrictive (e.g. 0027) then ask them to use umask 0022, instead, which can be made permanently the default by putting that in their bashrc, for e.g.

Let me know if that works, so we can add that to the documentation.

Cheers, Joel

[karenlmasters]

Hmm. We all have umask 0022. This is definitely a permissions issues on our system though.... Also have trouble downloading the drpall file through Marvin..... I'll make sure we all have the directory structure set up correctly.

[havok2063]

Is it a permissions issue only when downloading the files using sdss_access or also for general file access? If it's only the former, another option may be, if you have the space, is to download all files for the release you want ahead of time and place them in a local SAS structure. If Marvin finds the files already it won't use sdss_access to download them.

There is no .netrc setup needed for public access. Are you trying to access data from a DR or an MPL?

[karenlmasters]

We get an error on drpall download. We can go around it (download it and put it in the expected place), but I thought it would be nice to figure out the error.

[karenlmasters]

`/packages/python3.6.8/lib/python3.6/site-packages/marvin/utils/general/general.py:925: MarvinUserWarning: drpall file not found. Downloading it. warnings.warn('drpall file not found. Downloading it.', MarvinUserWarning)

MarvinError Traceback (most recent call last) in ----> 1 maps = Maps(mangaid='1-455684',mode='remote') 2 ha = maps['emline_gflux_ha_6564'] 3 ha.plot() 4 plt.show()

/packages/python3.6.8/lib/python3.6/site-packages/marvin/tools/maps.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download, nsa_source, bintype, template, template_kin) 97 mangaid=mangaid, plateifu=plateifu, 98 mode=mode, data=data, release=release, ---> 99 drpall=drpall, download=download) 100 101 NSAMixIn.init(self, nsa_source=nsa_source)

/packages/python3.6.8/lib/python3.6/site-packages/marvin/tools/core.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download) 116 MMAMixIn.init(self, input=input, filename=filename, mangaid=mangaid, 117 plateifu=plateifu, mode=mode, data=data, release=release, --> 118 download=download) 119 120 # drop breadcrumb

/packages/python3.6.8/lib/python3.6/site-packages/marvin/tools/mixins/mma.py in init(self, input, filename, mangaid, plateifu, mode, data, release, drpall, download, ignore_db) 118 self._forcedownload = download if download is not None else config.download 119 --> 120 self._determine_inputs(input) 121 122 assert self.mode in ['auto', 'local', 'remote']

/packages/python3.6.8/lib/python3.6/site-packages/marvin/tools/mixins/mma.py in _determine_inputs(self, input) 187 self.plateifu = mangaid2plateifu(self.mangaid, 188 drpall=self._drpall, --> 189 drpver=self._drpver) 190 191 elif self._plate:

/packages/python3.6.8/lib/python3.6/site-packages/marvin/utils/general/general.py in mangaid2plateifu(mangaid, mode, drpall, drpver) 417 raise MarvinError( 418 'mangaid2plateifu was not able to find a plate-ifu for ' --> 419 'mangaid={0} either local or remotely.'.format(mangaid)) 420 421 `

[havok2063]

What release are you trying to download the drpall file for? I will see if I can recreate the error.

[karenlmasters]

This is the public one. DR15. I think it might be a system issue here - in that one student reports he can download, but one other (and me) cannot. If I do the above after putting the drpall in the expected place it works (but only with mode='remote').

[havok2063]

I see. I just did a quick test, trying to recreate the issue with the above code, and it does indeed work for me, downloading the drpall file for DR15 and loading the maps. Let me know what your solution is. It would be good to document the issues you're having, and solutions, for others who want to set up Marvin in a Jupyter hub environment.

[karenlmasters]

Thanks. So for you did you install marvin and you are running marvin? this is a system installation (i.e. by a sysadmin) and user running.... could that make a difference?

[havok2063]

I was running this locally on my own machine. Yeah, this sounds like an issue between sysadmin and users permissions. Marvin essentially uses two directories. One is /tmp/sdss_access which is where sdss_access stores its log files. The other is the entire (sub)directory structure defined by the $SAS_BASE_DIR environment variable, which Marvin will set to ~/sas/ if the envvar is undefined. You can redefine the SAS envvar to wherever.

If this was installed by a sysadmin in a certain way/place, then it might be the case these directories aren't read/writeable by the users trying to use marvin to access these directories. You may need to give all users the correct permissions, or point the marvin directories in a more accessible place, or install marvin as some common user with the correct permissions and have all users use marvin as that common user. You could create a marvin group with the right permissions and add all users to that group.