PassengerAgent System Policy: deny mach-priv-task-port

[jamgold]

Hello,

Running Phusion Passenger 5.0.21 on OSX 10.11.1 El Capitan and the System log is full of error messages about PassengerAgent

12/5/15 10:01:06.000 kernel[0]: Sandbox: PassengerAgent(44038) System Policy: deny(1) mach-priv-task-port 44046

expanding the error shows following. Anybody know how to fix this?

PassengerAgent(44038) System Policy: deny mach-priv-task-port

Process:         PassengerAgent [44038]
Path:            /usr/local/Cellar/passenger/5.0.21/libexec/buildout/support-binaries/PassengerAgent
Load Address:    0x10bdd7000
Identifier:      PassengerAgent
Version:         ??? (???)
Code Type:       x86_64 (Native)
Parent Process:  PassengerAgent [44036]

Date/Time:       2015-12-05 09:53:26.014 -0800
OS Version:      Mac OS X 10.11.1 (15B42)
Report Version:  8

4367774.zip

[jamgold]

Upgraded to 5.0.22 but the problem persists.

kernel[0]: Sandbox: PassengerAgent(72384) System Policy: deny(1) mach-priv-task-port 6987

[jamgold]

One more update on this: I have deployed two nodejs apps via passenger

• ghost
• meteor app

only the meteor app exhibits the problem with PassengerAgent.

[jamgold]

I am now on 5.0.30 and this is still happening. Nobody else seeing this?

[CamJN]

Could you provide a bit more info about how you are running passenger, and your meteor app? Or better yet can you provide a minimal meteor app that reproduces the issue?

notes for myself: This seems to be related to SIP restrictions on using task_for_pid.

htop experienced a similar issue: https://github.com/hishamhm/htop/issues/449. monit too: https://bitbucket.org/tildeslash/monit/issues/266 & https://bitbucket.org/tildeslash/monit/commits/2fbf24563b88

SIP reference: https://developer.apple.com/library/watchos/documentation/Security/Conceptual/System_Integrity_Protection_Guide/System_Integrity_Protection_Guide.pdf

[jamgold]

Thank you for responding. I am pretty sure this is related to SIP. Will provide more information and a minimal app to reproduce.

[OnixGH]

@jamgold any update?

[jamgold]

Sorry @OnixGH, totally dropped the ball. I am still seeing the log entries but everything else works, so I stopped looking at the logs. Not sure how to exactly provide a minimal app to reproduce without shutting down my apps running under Passenger right now.

[jamgold]

Quick update. For every kernel[0]: Sandbox: PassengerAgent(473) System Policy: allow(0) mach-priv-task-port message a log file is being created in /var/log/asl/AUX....

On my Mac quite a lot of files accumulated. In /var/log/asl/AUX.2017.07.06 alone 10819, and over all the AUX.2017 directories 77747 files.

I have attached the content of one of those files

42467564.txt

[CamJN]

So that confirms that the source of the logs is the call to task_for_pid in measureRealMemory in ProcessMetricsCollector.h. However according to Apple (and consistent with my experience) you should be able to call task_for_pid on your own processes. So perhaps SIP is mistakenly protecting the /usr/local/* sub-directories on your mac?

You can try and check this with the following command:

ls -@Rl /usr/local | fgrep -i rootless

If there's any output then SIP is confused.

[jamgold]

Ran that command and no output

[CamJN]

Weird, I wonder why SIP thinks it needs to protect the process then.

[CamJN]

Does the line including /usr/local start with a * in /System/Library/Sandbox/rootless.conf?

[jamgold]

yes, here is the entry

* /usr/local

my webroot is on a different disk, can that matter?

[CamJN]

It shouldn't matter, no. We are measuring the memory use of Passenger when we make that call, so it's the passenger process that's being interfered with by SIP.

[jamgold]

the /usr directory has com.apple.rootless

 l -@d /usr drwxr-xr-x@ 13 root wheel 442 Nov 30 2016 /usr com.apple.FinderInfo 32 com.apple.rootless 0

[CamJN]

Yes, /usr is expected to be protected only /usr/local is provided to the user.

[CamJN]

Oh I just realized something, do you have SIP disabled? You can check with csrutil status.

[CamJN]

You used to get logs like: kernel[0]: Sandbox: PassengerAgent(72384) System Policy: deny(1) mach-priv-task-port 6987 which would imply that SIP was preventing the call but your latest log says: kernel[0]: Sandbox: PassengerAgent(473) System Policy: allow(0) mach-priv-task-port which is logged every time task_for_pid is called if SIP is disabled.

see: https://github.com/hishamhm/htop/issues/449#issuecomment-231619390

[jamgold]

Damn.  csrutil status System Integrity Protection status: disabled.

Let me enable SIP and report back

[jamgold]

Unfortunately back to deny every 5 seconds

7/7/17 14:25:15.000 kernel[0]: Sandbox: PassengerAgent(385) System Policy: deny(1) mach-priv-task-port

[jamgold]

One more thing, I run Phusion Passenger via nginx (don't know if that matters)

[CamJN]

How do you start nginx? Is it manual or launchd or something else?

[jamgold]

Launched by launchd. Using Homebrew version of nginx

[CamJN]

And is it started as a LaunchDaemon at boot or as a LaunchAgent on login?

[jamgold]

LaunchDaemon at boot via /Library/LaunchDaemons/homebrew.mxcl.nginx.plist

[CamJN]

Unfortunately I still cannot reproduce when running Nginx & Passenger that way. I'll keep thinking about this, but it's quite a riddle at this point.

[jamgold]

I am very grateful you haven't given up yet :)

[CamJN]

Ok so one more thing to check as to why SIP thinks it needs to protect PassengerAgent: xxd /usr/local/opt/passenger/libexec/buildout/support-binaries/PassengerAgent | fgrep __RESTRICT

I'm not confident that will reveal anything but it's something we haven't tried yet.

[jamgold]

It does not find __RESTRICT in the output of xxd

[CamJN]

I forgot to ask you to check the launchd plist for rootless attrs, so please do that:

ls -Ol@ /Library/LaunchDaemons/homebrew.mxcl.nginx.plist

When you say your webroot is on another disk, is it network mounted or connected via usb/firewire/thunderbolt/etc? It shouldn't matter but we've already covered all the sane explanations.

Does the disk that your webroot is on contain another osx install?

Can you also check the meteor app:

ls -@Rl /Volumes/webroot_Volume/path/to/app | fgrep -i rootless

Are you running the Meteor app as a bundled meteor-application or in dev mode? How was meteor installed? And where? Same for node.

Thanks.