buildstep
buildstep copied to clipboard
progrium
heroku-buildpack-php newrelic segfaults?
Using the latest version of the official Heroku buildpack for PHP and setting the correct environment settings for New Relic (and thus loading the New Relic extension) the resulting app fails to boot / run. And simply dies with a segfault... ?
Since i assume Heroku to still work with New Relic enabled, i suspect it must be something and was redirected here by my colleagues. Any tips?
to reproduce:
# get the heroku PHP example app and store it in the /tmp
git clone https://github.com/heroku/php-getting-started.git /tmp/app
# set the buildpack just to be sure
echo "export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-php.git#v66" > /tmp/app/.env
# clean up git just to be sure
rm -rfR /tmp/app/.git
# re add a single remote Dokku(alt) host
cd /tmp/app
git remote add dokku dokku@dokku-host:php-examle
# remove the .gitconfig to enable the push of .env file
rm .gitconfig
# add everything + commit
git add . && git commit -a -m "initial commit"
# push it to the dokku(alt) host
git push --set-upstream app master:master
This will deploy without newrelic as can be tested. This works so far ;)
now set the correct config to enable the loading of the newrelic .so (as described here)
dokku config:set php-examle NEW_RELIC_APP_NAME="test-php-app" NEW_RELIC_LICENSE_KEY="insert your licensekey here"
dokku rebuild php-example
Now hitting the php-example will report something like:
13:56:55 web.1 | started with pid 13
13:56:56 web.1 | DOCUMENT_ROOT changed to 'web/'
13:56:56 web.1 | 4 processes at 128MB memory limit.
13:56:56 web.1 | Starting php-fpm...
13:56:56 | exited with code 1
13:56:56 system | sending SIGTERM to all processes
13:56:58 web.1 | Going down, terminating child processes...
13:56:58 web.1 | exited with code 0
In other words: crash!
Nothing obvious except maybe in that new relic seems to be described as run as a separate daemon. Not sure how that interacts with everything else. Curious if you can find better ways to get debug information from new relic or the buildpack.
On Mon, Mar 23, 2015 at 9:06 AM, Robin Speekenbrink < [email protected]> wrote:
Using the latest version of the official Heroku buildpack for PHP and setting the correct environment settings for New Relic (and thus loading the New Relic extension) the resulting app fails to boot / run. And simply dies with a segfault... ?
Since i assume Heroku to still work with New Relic enabled, i suspect it must be something and was redirected here by my colleagues. Any tips?
to reproduce:
get the heroku PHP example app and store it in the /tmp
git clone https://github.com/heroku/php-getting-started.git /tmp/app# set the buildpack just to be sureecho "export BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-php.git#v66" > /tmp/app/.env# clean up git just to be sure rm -rfR /tmp/app/.git# re add a single remote Dokku(alt) hostcd /tmp/app git remote add dokku dokku@dokku-host:php-examle# remove the .gitconfig to enable the push of .env file rm .gitconfig # add everything + commit git add . && git commit -a -m "initial commit"# push it to the dokku(alt) host git push --set-upstream app master:master
This will deploy without newrelic as can be tested. This works so far ;)
now set the correct config to enable the loading of the newrelic .so (as described here https://github.com/heroku/heroku-buildpack-php/blob/master/bin/util/newrelic.sh )
dokku config:set php-examle NEW_RELIC_APP_NAME="test-php-app" NEW_RELIC_LICENSE_KEY="insert your licensekey here" dokku rebuild php-example
Now hitting the php-example will report something like:
13:56:55 web.1 | started with pid 13 13:56:56 web.1 | DOCUMENT_ROOT changed to 'web/' 13:56:56 web.1 | 4 processes at 128MB memory limit. 13:56:56 web.1 | Starting php-fpm... 13:56:56 | exited with code 1 13:56:56 system | sending SIGTERM to all processes 13:56:58 web.1 | Going down, terminating child processes... 13:56:58 web.1 | exited with code 0
In other words: crash!
— Reply to this email directly or view it on GitHub https://github.com/progrium/buildstep/issues/143.
Jeff Lindsay http://progrium.com
Jeff,
Haven't been able to get more debug info than this, but you're right that the newrelic agent does launch an extra process which pushes the gathered data to NR itself. But that extra process shouldnt kill the app should it?
(and how would Heroku do this if the buildpack is theirs?)
thanks!
I'm experiencing the same problem.
if NEW_RELIC_LICENSE_KEY is set the buildpack installs the newrelic extension, however it crushes.
This a stack trace I can obtain:
2015-05-12 17:48:41.782 (97 97) warning: daemon connect(fd=7 uds=/tmp/.newrelic.sock) returned -1 errno=ENOENT. Failed to connect to the newrelic-daemon. Please make sure that there is a properly configured newrelic-daemon running. For additional assistance, please see: https://newrelic.com/docs/php/newrelic-daemon-startup-modes
Any hints on this?
I'm trying to get to the bottom of this, and I've submitted this issue: https://github.com/heroku/heroku-buildpack-php/issues/113 on the heroku-php-buildpack repo. I don't know if it is buildpack specific, or just dokku specific.
I've given up on dokku+NR... The launching of the extra daemon by NR inside the container (since that extra daemon will push the data to NR periodically) crashes the container i just gave up ;)
Yeah in theory there shouldn't be a problem. I just don't know what it's doing because I can't get an idea from glancing at it.
On Thu, May 21, 2015 at 8:04 AM, Robin Speekenbrink < [email protected]> wrote:
I've given up on dokku+NR... The launching of the extra daemon by NR inside the container (since that extra daemon will push the data to NR periodically) crashes the container i just gave up ;)
— Reply to this email directly or view it on GitHub https://github.com/progrium/buildstep/issues/143#issuecomment-104268158.
Jeff Lindsay http://progrium.com
@fruitl00p I ran into similar problems you are describing. I don't think the new-relic daemon segfaults. I can actually start it manually, when attached to the container with:
$ /app/.heroku/php/bin/newrelic-daemon -d verbosedebug -f -l /tmp/heroku.ext-newrelic.newrelic-daemon.5001.log
I am only guessing...
- new relic daemon does not have write permissions for /tmp/heroku.ext-newrelic.newrelic-daemon.5000.log?
seem the logfile gets touched during the build-process as root here? https://github.com/heroku/heroku-buildpack-php/blob/f2b37ffa3aac00ccc6cb11405325298f40904028/bin/util/newrelic.sh#L29
It seems on heroku processes run as root (fpm, as well as the new-relic daemon). According to their docs (https://devcenter.heroku.com/articles/runtime-principles)
Superficially, it is convenient to think of each of your app’s dynos as managing a single root process
it seems dokku runs as root during build and executes the process as non root later on (which is a good thing :+1: )
@progrium how to ensure the new-relic logfile created during build (root-owned) is writable by the new-relic daemon (which is started trough php-fpm - non-root).
Should I do a PR against heroku/heroku-buildpack-php so the logfile is made world writable during build?
have tried chmod 777 on the logfile, new relic daemon is phoning home just fine with the modified buildpack forked from heroku/heroku-buildpack-php https://github.com/wesrc/heroku-buildpack-php-1/commit/b28a37ea76bf09a8ac84dd2a28e8cf10c2672f19. /cc @dzuelke
Has this been added in a pull request? Can we use the solution proposed by @lenada ?
I haven't seen an updated solution just yet @mattions maybe @progrium has news?
You should be using herokuish now if possible, not buildstep. Herokuish should be a bit more compatible with heroku buildpacks.
I´m using herokuish, so is not a problem only with buildstep, @lenada buildpack works perfect. I understand that a pr is not possible because is only happening for us using dokku/herokuish/buidstep, and not on heroku.
