visit
visit copied to clipboard
visit-dav
MetaData server error on MacOS Monterey
Describe the bug
Launching VisIt from the applications directory on MacOS Monterey with M1 Chip, displays the message: "The MetaData server running on localhost could not get the file list for the current directory".
Helpful additional information
- Did VisIt crash: yes
- Did you get wrong results: no results
To Reproduce
Steps to reproduce the behavior. For example:
- Go to Applications and launch Visit.
- Click on Open and try to open a file from user space
- See error
Expected behavior
VisIt to open a file
Attachments
- Can you reproduce the bug on our test data?
- If so, which file?
- If not, access to your data would be very helpful. If the data set is small, please zip and upload it as an attachment. If it is large, please provide instructions for how we can contact you to obtain this data.
- Please attach a (zipped) sessionfile to easily reproduce (In the viewer, use File->Save Session).
- Please attach any screenshots relevant to understanding the issue.
Additional context
We have tried this with VisIt 3.1.4, 3.3.0 and 3.3.1.
Out of 30 or so people today all but 1 were able to get VisIt to work by launching it from the terminal as was suggested in a closed issue from 2020. This did not fix the problem with the final user. I see the issue from 2020 is closed, but the problem seems to be back. I did not have this issue a few months ago on Mac, but now it presents a major issue to our users here (terminal use is a bit much to ask from some of them, and that is if it works).
@jameskress sorry you are running into this. As you mention, its a known issue. And, we have been trying various approaches to correct...yet to no avail.
Given that 29/30 people you tested have a workaround by launching from command-line, how can this be classified as a major issue?
BTW...the one case where it didn't work probably involves additional security settings or preferences impacting behavior in that case. Or, older vs. newer macOS versions. Do you by chance happen to know the range of macOS versions involved in the 30 users you tested?
Hi Mark, You are right, major is a bad phrase. It is a major pain. We work a lot with non computational folks so the idea of a command line is not something they know about or are used to. So from that perspective it is problematic if they cannot open the software as a regular program. The final person ended up just leaving the workshop after we spent 15 or so minutes trying to set different permissions and use different versions.
I believe that that person was running the previous version of macOS, prior to Monterey, as there system looked to be one of the oldest in the room. But I do not have a good poll on what the range was. My guess is that most were newer as the systems they give students/researchers here get replaced at a regular interval.
We also tried allowing "Full disk access" which was something I saw in another post, to no avail. I am not a mac guy by any means, and was running VisIt fine until recently (several new upgrades installed I believe), so the issue surprised me recently.
I just wanted to see if there were any updates or fixes I missed. But If the solution is just to wait we can do that and not clog up your issue queue, since I did not see any recent comments on this issue I was hoping there was a magic cure.
I am working hard to get to the bottom of this. A challenge is that VisIt is by and large developed on Linux. Yes, we have phenomenal team members each working to make it also work on Windows and macOS. But, on macOS anyways, we don't develop it under Xcode. And, we try not to use very much macOS specific coding (or Windows specific coding for that matter) for obvious reasons.
Over the years, Apple continues to "improve" its security measures and more and more this means if you don't develop code in the "Apple way", its going to be a PITA to make it work. See for example, https://developer.apple.com/forums/thread/663889 which uses several of the same technologies VisIt does (Qt, Python, bash and multiple intercommunicating exectuables).
I am recording here what information I am able to gather with other tools (currently neither lldb nor Xcode will attach to process launched from finder (e.g. by double-clicking the icon of the installed process)...
- From
console, I was able to find this error...error 11:07:46.852926-0700 kernel Sandbox: mdserver(29828) System Policy: deny(1) file-read-data /Users/miller86/Documents
I just tried running Gimp 2.8.20 on my macOS by double clicking the app bundle icon (e.g. not from command line). And, when I try to descend into Documents, Desktop or Downloads it first presents a dialog asking to give the app permission to do this. I said Ok to one and Don't allow to another and from then on, it allows me to go into Desktop but not Downloads. So, this one-time question process is what is missing from VisIt. I also see this for gimp on console...
default 11:34:28.312007-0700 tccd Prompting for access to kTCCServiceSystemPolicyDownloadsFolder from /Applications/MacPorts/GIMP.app/Contents/MacOS/GIMP
Here is where the GIMP team discussed a lot of this, https://gitlab.gnome.org/GNOME/gimp/-/issues/3710
After I did the above with Gimp, I went looking in prefs to see what changed...
Somehow, we need to get VisIt to have those check boxes too instead of Full Disk Access which was a prev. 10.14 approach.
Ok, so I just tried LLNL's Hopper application too. And, like Gimp, it first asked permission to descend into Documents and once I gave it permission, it appears in the list of Files and Folders with the Documents folder checked.
So, we've got to get to the bottom of why VisIt is not asking for permission when first descending into these. I don't think its a Info.plist thing.
I think we may need to explicitly ask up front (when we first start visit), but I am not sure how to suggest this. GUI, Viewer, and engine are all different exes that need this permission. But the are all part of the same app bundle -- they should inherit settings, but I am not sure the share "Allow" when a user clicks in real time.
And, com.apple.security.inherit
That said, there appears to be a special mode Sandbox in which an app can run that possibly changes the security conditions. I noticed the consol log error messges above from mdserver indicate Sanbox.
Finally, we have only VisIt in .../Contents/MacOS and this ref suggests we need to have the other tools VisIt uses there also in order for them to behave within the security context.
We used to used to have com.apple.security.files.user-selected.read-write but that didn't solve it.
are there extra flags we can use to get detailed security output to debug thing like this? I tried looking in the macOS console in the past, but I didn't see anything.
are there extra flags we can use to get detailed security output to debug thing like this?
Possibly. Am looking/reading as much as I can. I feel like we came this party through the side door and are now trying to get formal tickets and our names on the attendee list 😉. I am thinking we might need to do some minimal dev directly in Xcode to really resolve.
I have tried attaching lldb to processes launched after double clicking VisIt icon but it won't attach under those conditions (yes I sudo'd).
BTW, I dunno but I wonder if the inherting feature of entitlements needs to be stated on the inheritor not the one from which to inherit.
FYI, GIMP is not even a codesigned app but nonetheless does not have this issue...
% codesign --display --entitlements :- /Applications/MacPorts/GIMP.app
/Applications/MacPorts/GIMP.app: code object is not signed at all
And for Hopper...
% codesign --display --entitlements :- /Applications/Hopper.app
Executable=/Applications/Hopper.app/Contents/MacOS/Hopper
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.cs.allow-jit</key>
<true/>
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
<key>com.apple.security.cs.allow-dyld-environment-variables</key>
<true/>
<key>com.apple.security.cs.debugger</key>
<true/>
</dict>
</plist>
FYI...used dtruss to trace mdserver and don't get very much info (see below).
I do see that open_nocancel() for Documents returns -1.
/Applications/VisIt.app/Contents/Resources/3.2.2/darwin-x86_64/bin/mdserver -host 127.0.0.1 -port 5601 -key 17ec90689256c77e14cf
3640 60158 71671 0 4:21PM ttys002 0:00.00 grep mdserver
[scratlantis:/Applications/Hopper.app/Contents] miller86% dtruss -p 59744
dtrace: system integrity protection is on, some features will not be available
dtrace: failed to initialize dtrace: DTrace requires additional privileges
[scratlantis:/Applications/Hopper.app/Contents] miller86% sudo dtruss -p 59744
Enter PIN for 'Certificate For PIV Authentication (MARK MILLER (Affiliate))':
dtrace: system integrity protection is on, some features will not be available
SYSCALL(args) = return
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 26 0
chdir("/Users/miller86\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86\0", 0x1100004, 0xFFFFFFFFE6248B70) = 6 0
fstatfs64(0x6, 0x7FFEE6248218, 0x0) = 0 0
getdirentries64(0x6, 0x7F858F824E00, 0x2000) = 7328 0
close_nocancel(0x6) = 0 0
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x21) = 33 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x3BE) = 958 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 36 0
chdir("/Users/miller86/Downloads\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86/Downloads\0", 0x1100004, 0xFFFFFFFFE6248B70) = -1 1
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x2B) = 43 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
sendto(0x5, 0x7FFEE62485E0, 0x1D) = 29 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 26 0
chdir("/Users/miller86\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86\0", 0x1100004, 0xFFFFFFFFE6248B70) = 6 0
fstatfs64(0x6, 0x7FFEE6248218, 0x0) = 0 0
getdirentries64(0x6, 0x7F858F015E00, 0x2000) = 7328 0
close_nocancel(0x6) = 0 0
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x21) = 33 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x3BE) = 958 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 26 0
chdir("/Users/miller86\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86\0", 0x1100004, 0xFFFFFFFFE6248B70) = 6 0
fstatfs64(0x6, 0x7FFEE6248218, 0x0) = 0 0
getdirentries64(0x6, 0x7F859180DE00, 0x2000) = 7328 0
close_nocancel(0x6) = 0 0
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x21) = 33 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x3BE) = 958 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 34 0
chdir("/Users/miller86/Desktop\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86/Desktop\0", 0x1100004, 0xFFFFFFFFE6248B70) = -1 1
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x29) = 41 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
sendto(0x5, 0x7FFEE62485E0, 0x1D) = 29 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 26 0
chdir("/Users/miller86\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86\0", 0x1100004, 0xFFFFFFFFE6248B70) = 6 0
fstatfs64(0x6, 0x7FFEE6248218, 0x0) = 0 0
getdirentries64(0x6, 0x7F8591016000, 0x2000) = 7328 0
close_nocancel(0x6) = 0 0
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 0
sendto(0x5, 0x7FFEE6248660, 0x21) = 33 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x400) = 1024 0
sendto(0x5, 0x7FFEE62485F0, 0x3BE) = 958 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 33 0
chdir("/Users/miller86/.visit\0", 0x0, 0x0) = 0 0
open_nocancel("/Users/miller86/.visit\0", 0x1100004, 0xFFFFFFFFE6248B70) = 6 0
fstatfs64(0x6, 0x7FFEE6248218, 0x0) = 0 0
getdirentries64(0x6, 0x7F8591018800, 0x2000) = 1384 ??$??0
close_nocancel(0x6) = 0 0
sendto(0x5, 0x7FFEE62486F0, 0xE) = 14 /Users/miller860
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 ancel0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 9 ??$??0
sendto(0x5, 0x7FFEE6248660, 0x28) = 40 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 )0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x4, 0x7FFEE62488C0, 0x400) = 16 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
getuid(0x0, 0x0, 0x0) = 3640 0
getgroups(0x64, 0x7FFEE6248740, 0x0) = 14 0
sendto(0x5, 0x7FFEE62485F0, 0x286) = 646 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
select(0x5, 0x7FFEE6248C70, 0x0, 0x0, 0x0) = 1 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
recvfrom(0x0, 0x7FFEE62488C0, 0x400) = 14 0
sendto(0x3, 0x7FFEE62486C0, 0xE) = 14 0
setitimer(0x0, 0x7FFEE6248CD0, 0x7FFEE6248CF0) = 0 0
What is interesting is that the chdir() to Documents (and friends) does seem to succeed. So, I think its the attempt to readdir() that where its failing.
I don't know how it happened but for the 3.3.1 release after launching from the command-line, it asked for permission to access Documents and I gave it. Then, I tried running 3.3.1 from icon (on dmg mount, not from /Applications) and that still failed but I was able to see more stuff in console logs...
default 18:36:12.340877-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.341840-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.342350-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.342612-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
error 18:36:12.345576-0700 tccd IDENTITY_ATTRIBUTION: Failed to copy signing info for 76797, responsible for /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver: #-67062: Error Domain=NSOSStatusErrorDomain Code=-67062 "(null)"
default 18:36:12.346833-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.348124-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.348542-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.348788-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
error 18:36:12.351715-0700 tccd IDENTITY_ATTRIBUTION: Failed to copy signing info for 76797, responsible for /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver: #-67062: Error Domain=NSOSStatusErrorDomain Code=-67062 "(null)"
default 18:36:12.352926-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:12.354331-0700 tccd -[TCCDAccessIdentity staticCode]: static code for: identifier /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver, type: 1: 0x7fd479d28a50 at /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
default 18:36:12.361397-0700 tccd Prompting for access to kTCCServiceSystemPolicyDocumentsFolder from /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
default 18:36:47.019634-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.020550-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.021452-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.021897-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.024589-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.026384-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.026890-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.027173-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.029573-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:47.032468-0700 tccd -[TCCDAccessIdentity staticCode]: static code for: identifier /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver, type: 1: 0x7fd47c422c40 at /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
default 18:36:47.037216-0700 tccd Prompting for access to kTCCServiceSystemPolicyDesktopFolder from /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
default 18:36:53.729902-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.730654-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.731095-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.731300-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.733247-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.734540-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.735033-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.735272-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.737178-0700 tccd SecTaskCopyDebugDescription: mdserver[76797]/0#-1 LF=0
default 18:36:53.738862-0700 tccd -[TCCDAccessIdentity staticCode]: static code for: identifier /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver, type: 1: 0x7fd479c3f310 at /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
default 18:36:53.742945-0700 tccd Prompting for access to kTCCServiceSystemPolicyDownloadsFolder from /Users/miller86/visit/visit/33rc/release/build-mb-3.3.1-darwin-10.15-x86_64-release/build.release/exe/mdserver
So, installing VisIt to /Applications didn't change behavior (from above) comment.
But, I did capture this more detailed system log message...
Sandbox: mdserver(78538) System Policy: deny(1) file-read-data /Users/miller86/Documents
Violation: System Policy: deny(1) file-read-data /Users/miller86/Documents
Process: mdserver [78538]
Path: /Applications/VisIt.app/Contents/Resources/3.3.1/darwin-x86_64/bin/mdserver
Load Address: 0x10e6e1000
Identifier: mdserver
Version: ??? (???)
Code Type: x86_64 (Native)
Parent Process: python3.7 [78527]
Responsible: /bin/sh [/Applications/VisIt.app/Contents/MacOS/VisIt]
User ID: 36XX
Date/Time: 2022-10-10 18:49:22.973 PDT
OS Version: Mac OS X 10.15.7 (19H2026)
Report Version: 8
MetaData: {
"errno":1,
"signing-id":"mdserver",
"build":"Mac OS X 10.15.7 (19H2026)",
"responsible-process-hosted-path":"\/Applications\/VisIt.app\/Contents\/MacOS\/VisIt",
"pid":78538,
"path":"\/Users\/miller86\/Documents",
"operation":"file-read-data",
"matched-user-intent-extension":false,
"mount-rdev":16777220,
"process-path":"\/Applications\/VisIt.app\/Contents\/Resources\/3.3.1\/darwin-x86_64\/bin\/mdserver",
"rdev":0,
"profile":"platform",
"apple-internal":false,
"responsible-process-path":"\/bin\/sh",
"process":"mdserver",
"profile-flags":0,
"user-approval":"kTCCServiceSystemPolicyDocumentsFolder",
"action":"deny",
"matched-extension":false,
"normalized_target":["Users","miller86","Documents"],
"storage-class":"kTCCServiceSystemPolicyDocumentsFolder",
"responsible-process-user-uuid":"AF69A7B8-A00B-4653-8E2D-3328FDA675FE",
"vnode-type":"DIRECTORY",
"flags":5,
"uid":36XX,
"team-id":"A827VH86QR",
"hardware":"Mac",
"target":"\/Users\/miller86\/Documents",
"platform_binary":"no",
"responsible-process-uid":36XX,
"summary":"deny(1) file-read-data \/Users\/miller86\/Documents",
"platform-policy":true,
"hardlinked":false,
"file-flags":0,
"primary-filter":"path",
"platform-binary":false,
"primary-filter-value":"\/Users\/miller86\/Documents"
}
Thread 0 (id: 66589492):
0 libsystem_kernel.dylib 0x00007fff6f779826 __open_nocancel + 10
1 mdserver 0x000000010e6f77e1 MDServerConnection::ReadFileList() + 257
2 mdserver 0x000000010e6f76a7 MDServerConnection::ChangeDirectory(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 87
3 mdserver 0x000000010e6efc89 ChangeDirectoryRPCExecutor::Update(Subject*) + 41
4 libvisitcommon.dylib 0x0000000115cdcce8 Subject::Notify() + 168
5 libvisitcommon.dylib 0x0000000115bad6b2 AttributeSubject::Notify() + 18
6 libvisitcommon.dylib 0x0000000115d169fd Xfer::Process() + 445
7 mdserver 0x000000010e6f44fc MDServerConnection::ProcessInput() + 44
8 mdserver 0x000000010e6f2604 MDServerApplication::Execute() + 980
9 mdserver 0x000000010e70244c MDServerMain(int, char**) + 284
10 libdyld.dylib 0x00007fff6f637cc9 start + 1
11 mdserver 0x0000000000000007
Binary Images:
0x10e6e1000 - 0x10e705fff mdserver (0) <c36cbbdf-7e54-365c-8de3-a0bd626ce464> /Applications/VisIt.app/Contents/Resources/3.3.1/darwin-x86_64/bin/mdserver
0x115a01000 - 0x115d7cff7 libvisitcommon.dylib (0) <47e57871-6dca-3080-a3ad-589e20990faf> /Applications/VisIt.app/Contents/Resources/3.3.1/darwin-x86_64/lib/libvisitcommon.dylib
0x7fff6f61d000 - 0x7fff6f653fff libdyld.dylib (750.7) <ab99c9ee-7127-3451-89ab-339f8f2cee61> /usr/lib/system/libdyld.dylib
0x7fff6f778000 - 0x7fff6f7a4ff7 libsystem_kernel.dylib (6153.141.66) <ff081f3a-f653-3f8f-9e64-9f34792eedb3> /usr/lib/system/libsystem_kernel.dylib
Ok, so I am able to get desired behavior by adding /bin/sh to Security and Permissions for Full Disk Access. But, thats not the best solution.
Here is why I think /bin/sh is the key thing to add. It is the main process that launches VisIt.
I think to correct, we want an actuall Mach-O executable that starts VisIt. This should be easy and I am going to try making such with an installed VisIt and see if that fixes the issue.
% file /Applications/VisIt.app/Contents/MacOS/VisIt
/Applications/VisIt.app/Contents/MacOS/VisIt: POSIX shell script text executable, ASCII text
Whereas...
% file /Applications/MacPorts/GIMP.app/Contents/MacOS/GIMP
/Applications/MacPorts/GIMP.app/Contents/MacOS/GIMP: Mach-O 64-bit executable x86_64
% file /Applications/Hopper.app/Contents/MacOS/Hopper
/Applications/Hopper.app/Contents/MacOS/Hopper: Mach-O 64-bit executable x86_64
% file /Applications/MacPorts/MacVim.app/Contents/MacOS/MacVim
/Applications/MacPorts/MacVim.app/Contents/MacOS/MacVim: Mach-O 64-bit executable x86_64
This is awesome info Mark! I appreciate everything you are doing. I only started on Mac with this new position, and don't use it for dev, so this is new to me as well.
I also saw the popup on 3.3.1, and I think others in the workshop got it on 3.3.0. They had never installed a previous VisIt, so no old settings / permissions were sitting around. But that did not solve it.
Cool find on /bin/sh. That absolutely supports your idea that Mac now wants executables vs the scripts. Too bad they keep making the security policy more of a pain to follow. I am certain this was working a month or two ago when I set everything up...
Another supporting point for your theory is from the cinema_scope application from https://cinemascience.github.io/downloads.html. This is a fairly basic repo and the application had access to Documents and Desktop but not full disk access.
KW-23567:MacOS kressjm$ file cinema_scope cinema_scope: Mach-O 64-bit executable x86_64
Ok, after all that hunting around...I have the solution.
The solution is for the executable that the VisIt icon launches, /Applications/VisIt.app/Contents/MacOS/VisIt, to be an actual Mach-0 binary and not a shell script.
When I cd to /Applications/VisIt.app/Contents/MacOS and create foo.c as...
#include <stdlib.h>
int main(int argc, char **argv)
{
system("/Applications/VisIt.app/Contents/Resources/bin/visit");
return 0;
}
and then compile it gcc foo.c -o foo and then do the following
-
cp VisIt VisIt.sh -
rm VisIt -
cp foo VisIt.exe -
ln -s VisIt.exe VisIt
And then double click on the VisIt icon to start VisIt, I get the desired behavior. When user tries to descend into Documents, Downloads or Desktop, the user is given a dialog box to give permission and if permission is granted, VisIt can then browser files in those folders and I wind up seeing the following in Security and Privacy preferences...
We will have this implemented for next release. Thanks @jameskress for pressing for a solution!
In fixing this, there are some related things...
- #2303
- #4324
- #10287
which I am aim to fix with change to address this issue.
In addition, what about supporting multiple different versions of VisIt installed on macOS? Can we do that already. I always select replace when installing VisIt but there is an option to keep both and I wonder if after doing that, VisIt works? IMHE, it doesn't maybe because of the space in pathnames mentioned in the above issues.
@biagas and @cyrush ... I need to modify this CMake logic...
https://github.com/visit-dav/visit/blob/07a62609aa290bd15608fdeee95f5280e3db2f50/src/CMakeLists.txt#L2348-L2356
and change the referenced file (currently a shell script) for CPACK_BUNDLE_STARTUP_COMMAND to be a compiled C program in somewhat similar fashion to what we do on Windows here...
https://github.com/visit-dav/visit/blob/07a62609aa290bd15608fdeee95f5280e3db2f50/src/bin/CMakeLists.txt#L149-L155
The compiled C program is vanilla C with no dependencies which I think will need to live in src/bin as well. I may call it LaunchVisItFromAppBundle.c just to maintain historical association with shell code.
So, I think I will need to make src/bin/LaunchVisItFromAppBundle.c so I think I need to ADD_EXECUTABLE() it in src/bin/CMakeLists.txt. Anything else?
@markcmiller86 Thanks for this! This will be awesome to have.
My observations regarding the multiple versions question above: I too was wondering about multiple versions as I like to keep them around on Linux and Windows, but never clicked "keep both" on Mac. Trying today it looks like it causes issues, even launching from command line.
It creates a new path in applications, in my case:
VisIt 2.app
Running this install from command line failed with:
/opt/homebrew/opt/[email protected]/bin/python3.9: can't find '__main__' module in '/Applications'
So sounds like this is the issue that you already linked above about spaces in path names.
Running the original install works fine from the command line. Also, the app name appears in the "Applications" list, but does not show as an icon in applications. So apart from the name now being very descriptive as an application name it fails to launch or show up.
It creates a new path in applications, in my case:
VisIt 2.appRunning this install from command line failed with:/opt/homebrew/opt/[email protected]/bin/python3.9: can't find '__main__' module in '/Applications'So sounds like this is the issue that you already linked above about spaces in path names.
In investigating this further, it looks like dealing with spaces in pathnames needs to be handled on a case by case basis in shell, python and C/C++ code involved in bootstrapping launch of VisIt. In shell, for example, it requires auditing all the places where a path which may include spaces is expanded and ensuring its properly quoted. In C/C++ code, it involves ensuring the spaces are properly escaped. I think python may handle it fine but haven't looked.
I mention because considering all the places in VisIt where pathname is handled, I worry that ensuring correct operation in the presence of a space in the path name of where VisIt is installed cannot be readily and fully tested.
I think I might have a better solution and that is to have VisIt installed with version specific numbering. So, instead of /Applications/VisIt you'd have /Applications/VisIt-3.2.2 and /Applications/VisIt-3.3.0, etc. Re-installing the same version would result in being asked to keep or replace and if you choose keep, you'd get that space (which you should be able to manually adjust and I think we'll ensure this is documented) but installing a new version while keeping an older version would not introduce spaces in the installation name.
I would have never guessed that the shell script vs the binary exec was the cause of this. Especially since other perms seem to work (like asking for network access). During my hunting in the dark I never was close at all.
Thanks so much Mark for figuring this out !!!!