weave-tc icon indicating copy to clipboard operation
weave-tc copied to clipboard

Published 20 hours ago • verified publisher icon Quentin-M

Amazon EC2

Open RainistJordy opened this issue 6 years ago • 2 comments
trafficstars

Hello Quentin,

Do you perhaps have any experience deploying this solution to AWS? We used kops to deploy everything and have been running into this issue. Now we launched this weave-tc as a DaemonSet in kube-system, and everything seems fine inside of the weave-tc pods. In other pods however the problem persists. We tried mounting /usr/lib/tc and /sbin/tc from the host machine to the same locations inside of the container.

Any ideas?

Best Regards,

Jordy

RainistJordy avatar Jul 24 '19 09:07 RainistJordy

Hi there,

We do currently deploy this solution on AWS w/ weave, which solves the problem pretty much entirely. Did you verify that the tc & iptables rules are in place on the hosts? Do you notice the iptables rule marking packets successfully? Do you notice insert_failed increase when the packet loss occurs?

Quentin-M avatar Jul 25 '19 20:07 Quentin-M

I've figured out that the DNS issues are only resolved when hostNetwork is true on other pods. Whenever it is not, it doesn't work. I'm using Weave 2.4.0, Kernel 4.9 and Kubernetes 1.10. As long as hostNetwork is true obviously iptables is fine and packet marking is okay.

I do have a new AMI ready to go with Linux 5.1.19 and Weave 2.5.2 which solves it too, it's just a little riskier to upgrade to.

RainistJordy avatar Jul 26 '19 02:07 RainistJordy