Segmentation fault

[christian-fei]

Hello, first of all thanks for this well done image.

I'm facing the following issue, a bit of context first:

The image works like a charm on my local machine (vagrant), but on the CI (circleci) it fails with "Segmentation fault":

Status: Downloaded newer image for msaraiva/elixir-dev:latest
 ---> 222954f834bb
Step 1 : ADD mix.exs /app/mix.exs
 ---> 59306404aad1
Step 2 : WORKDIR /app
 ---> Running in a4a02abe52f6
 ---> 0b757798a367
Step 3 : RUN yes | mix local.hex
 ---> Running in bf1aa4ff8da1
Segmentation fault (core dumped)
The command '/bin/sh -c yes | mix local.hex' returned a non-zero code: 139

I wanted to ask if you have any clues on how to solve this. Thanks

[msaraiva]

Hi Christian, have you tried to remove all base images, including alpine:3.2 and then pull msaraiva/elixir-dev:latest again?

[christian-fei]

hmm, thank you, i will try this approach, thank you!

[msaraiva]

@christian-fei Are you still having trouble with your CI server? In case you're not aware of, I'm waiting for the new v3.3 of Alpine Linux (which has already been released) to be available at docker hub to update the Erlang/Elixir images. You can see details here: https://github.com/msaraiva/alpine-erlang/issues/2. After this change, I don't expect any kind of trouble like this to happen anymore.

[christian-fei]

Awesome!! Thanks for the info. Didn't have time to make the changes, at this point I will wait for the alpine 3.3 release and your integration with it.

Thanks!

[christian-fei]

Hello, tried to remove the base images and build it as first, but still not luck. Do you have any news for the update to v3.3? Thanks

[msaraiva]

@christian-fei just got an update on that:

The v3.3 release candidate 1 is out and v3.3 repositories are available for testing. We will branch 3.3-stable (and release 3.3.0) within a week or two.

You can see more details here: https://github.com/gliderlabs/docker-alpine/issues/103#issuecomment-160549247

[christian-fei]

Awesome! thanks for getting back

[msaraiva]

@christian-fei the images have been updated. Can you confirm that this issue is gone?

[christian-fei]

Hello! Tried the build again on circleci with the latest image msaraiva/elixir-dev:1.2.0 but got the same exact issues as before regarding "segmentation fault". This makes me think that there's a problem on the circleci side.

For more info you can checkout this build for example

Thanks for your time and support!

[msaraiva]

Error removing intermediate container 1f7ff031c7c5: Cannot destroy container 1f7ff031c7c5f5872ac957d972fd520f3b767d2e1d706d9400fb0c643d7445c1: Driver btrfs failed to remove root filesystem 1f7ff031c7c5f5872ac957d972fd520f3b767d2e1d706d9400fb0c643d7445c1: Failed to destroy btrfs snapshot: operation not permitted

@christian-fei it looks like docker is having trouble to remove some of the old containers. Can you see those containers when you run docker ps -a? Are you able to remove them manually using docker rm <container_id>?

[christian-fei]

@msaraiva there are no docker containers running nor images locally... very strange. see this build

Found these issues, but I spent already a lot of time on it, and it seems they are never going to fix it:

• https://github.com/docker/compose/issues/1046
• https://discuss.circleci.com/t/docker-error-removing-intermediate-container/70

[msaraiva]

@christian-fei

1. I think you can get rid of the "Error removing intermediate container" if you add --rm=false to your docker build command: docker build --rm=false -t .... (But I don't think this will fix the segmentation fault problem).
2. Also try to add RUN mix local.hex --force before RUN mix deps.get in your Dockerfile.

Let's see if we can get any further.

[christian-fei]

yep, exactly as you thought: the segmentation fault problem didn't go away. but it's a step forward, so thank you very much! :)

unfortunately, i inspected the build output on the build container of circleci, and the only logs i could retrieve contained:

Segmentation fault (core dumped)

and nothing else..

---

It seems the command mis deps.get (--force) causes the issue, will investigate further

[shadabahmed]

Getting the same issue with Alpine 3.3. It builds fine on local and even on Ubuntu 14.04 VM, but always fails with segfault on CirceCI. I tried adding strace to mix hex.local command:

...
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 64 entries */, 2048)   = 2048
getdents64(4, /* 46 entries */, 2048)   = 1472
getdents64(4, /* 0 entries */, 2048)    = 0
close(4)                                = 0
getdents64(3, /* 0 entries */, 2048)    = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)
The command '/bin/sh -c cd /opt/src/bravos && strace mix local.hex --force' returned a non-zero code: 139

[appplemac]

I work at CircleCI. The Failed to destroy btrfs snapshot: operation not permitted messages are expected—we currently disallow removing intermediate containers. Sorry about this.

Please let me know if I can help with anything context-wise.

[christian-fei]

Hello, yep saw the discussion. Does that mean that every project that docker is essentially not supported? Because it worked before on a personal project (that uses docker), but then I had to switch to travis for this reason.

[appplemac]

@christian-fei The only thing you cannot do on CircleCI right now is remove intermediate containers. The rest of Docker functionality is available.

If your personal project relies on deleting intermediate containers during the build, then yes, that would not be supported.

Please reach out to sayhi (at) circleci (dot) com if you are having trouble setting up your Docker project.

I would still like to help with the segfault here.

[jaebinyo]

@appplemac I'm seeing the same segmentation fault issue on CircleCI. Any update on this issue?

[jaebinyo]

I just installed erlang on alpine 3.3 in CircleCI, erl command itself causes segmentation fault. Maybe 32, 64bit issue?

[appplemac]

@jaebinyo We still don’t allow removing intermediate containers. Sorry about that.

[joshwlewis]

@jaebinyo @christian-fei, were either of you able to solve or work around this? I'm seeing the same segfault with Alpine 3.3 and OTP 18.2.1 on CircleCI only...

[jaebinyo]

@joshwlewis No.. I gave up on that. I'm using ubuntu base image on CircleCI because of that issue.

[christian-fei]

@joshwlewis i gave up too, I didn't find a workaround for this issue on CircleCI :(

[joshwlewis]

@jaebinyo @christian-fei Circle's docker 1.10 beta resolved the issue for me. Probably worth a try if you are still hung up on this.

[jaebinyo]

@joshwlewis Yup! It worked with that trick! Thanks a lot for the info. Now I can use alpine-erlang on CircleCI. Thanks Josh!

[christian-fei]

@joshwlewis i'm going to try this right away! Thx :)