Weird inode eater studio

[lnxchk]

I don't know how helpful this will be, but it's certainly weird.

OS: Ubuntu 16.04.2 LTS (in AWS) Hab: hab 0.26.1/20170715033447

Application: https://github.com/thommay/container_sched_backend

I was working with this rust app because it's a neat enough demo with low overhead. I ran it with hab start.

The machine I'm working on is in AWS, and I don't remember if I stopped the studio or if ssh just timed out; it's been over a week since I worked on this. I logged into the machine today and it was completely out of inodes.

root@ip-172-31-6-208:/tmp# df -i
Filesystem      Inodes   IUsed  IFree IUse% Mounted on
udev            503716     381 503335    1% /dev
tmpfs           505651     487 505164    1% /run
/dev/xvda1     3072000 3071924     76  100% /
tmpfs           505651       1 505650    1% /dev/shm
tmpfs           505651       4 505647    1% /run/lock
tmpfs           505651      16 505635    1% /sys/fs/cgroup
tmpfs           505651       4 505647    1% /run/user/1000
tmpfs           505651       1 505650    1% /hab/studios/home--ubuntu--aar-habitat--habitat/run
tmpfs           505651       1 505650    1% /hab/studios/home--ubuntu--container_sched_backend--habitat/run
tmpfs           505651       1 505650    1% /hab/studios/home--ubuntu--container_sched_backend/run

I went digging around, looked in /hab/studios:

root@ip-172-31-6-208:/hab/studios# ls
home--ubuntu--aar-habitat--habitat  home--ubuntu--container_sched_backend  home--ubuntu--container_sched_backend--habitat
root@ip-172-31-6-208:/hab/studios# find . | wc
3169700 3169954 218673174

On Ian's advice in slack, I cd'd to the project directory and ran hab studio rm on the container_sched_backend studio, freeing up 91% of inodes:

ubuntu@ip-172-31-6-208:~$ df -i
Filesystem      Inodes  IUsed   IFree IUse% Mounted on
udev            503716    381  503335    1% /dev
tmpfs           505651    496  505155    1% /run
/dev/xvda1     3072000 252185 2819815    9% /
tmpfs           505651      1  505650    1% /dev/shm
tmpfs           505651      4  505647    1% /run/lock
tmpfs           505651     16  505635    1% /sys/fs/cgroup
tmpfs           505651      4  505647    1% /run/user/1000
tmpfs           505651      1  505650    1% /hab/studios/home--ubuntu--aar-habitat--habitat/run
tmpfs           505651      1  505650    1% /hab/studios/home--ubuntu--container_sched_backend--habitat/run

I didn't see anything else particularly weird on the system. There were some rust pipes in /tmp, but they weren't current.

[adamhjk]

I saw this as well, but didn't have time to chase it down.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale[bot]]

This issue has been automatically closed after being stale for 400 days. We still value your input and contribution. Please re-open the issue if desired and leave a comment with details.