docs
docs copied to clipboard
docker
Update limits on overlay networks
A while ago there were issues with loadbalancer 1) on large overlay networks, so a limit in overlay size was added to documentation in 2) This issue has since been resolved, and according to 3) the limitations are no longer necessary.
The documentation should be updated accordingly
- https://github.com/moby/moby/issues/30820
- https://github.com/docker/docker.github.io/pull/5208
- https://github.com/moby/moby/pull/37372#issuecomment-414391171
@thaJeztah Is this true that a /24 is no longer required for an overlay network?
If so let me know and I can work on a PR for the docs.
The part of the docs in question:
You should create overlay networks with /24 blocks (the default), which limits you to 256 IP addresses, when you create networks using the default VIP-based endpoint-mode. This recommendation addresses limitations with swarm mode.
https://docs.docker.com/engine/reference/commandline/network_create/#overlay-network-limitations
It would be good if the documentation included what version of docker this changed.
There hasn't been any activity on this issue for a long time.
If the problem is still relevant, mark the issue as fresh with a /remove-lifecycle stale comment.
If not, this issue will be closed in 14 days. This helps our maintainers focus on the active issues.
Prevent issues from auto-closing with a /lifecycle frozen comment.
/lifecycle stale
There hasn't been any activity on this issue for a long time.
If the problem is still relevant, mark the issue as fresh with a /remove-lifecycle stale comment.
If not, this issue will be closed in 14 days. This helps our maintainers focus on the active issues.
Prevent issues from auto-closing with a /lifecycle frozen comment.
/lifecycle stale
@akerouanton @dvdksn Is any of this covered in the networking rewrite? (Sorry still haven't found time to look at the PR 🙈)
We have not updated this bit yet, no. Sounds like we can just remove the Overlay network limitations🔗 section then?
@dvdksn I don't think so, because the overlay network is actually using a bridge interface internally to connect containers co-located on a same host. As such, the limitations I asked you to add to the bridge doc page also apply to the overlay driver.
So if somebody creates a /16 overlay network, what problems could they be facing?
I guess nothing, until you hit the limit of 1024 interfaces. That's hard-coded in the kernel. ref
But I will let @akerouanton correct me
i only have 12 /24 overlay networks with encryption enabled - one manager and two worker nodes- when i first boot up the boxes- i'm able to spin up containers until i hit a limit and then any subsequent containers- even when i removed them- won't start up- all containers are stuck at a "ready" state or new:
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS vkri0f9lwtzi docs_code.1 collabora/code:latest Running New 44 minutes ago
ID NAME MODE REPLICAS IMAGE PORTS 68uaoz7y3oiz docs_code replicated 0/1 collabora/code:latest
- if i reboot the servers then im able to spin up the containers - ive tried to cleanup any lingering containers but am not able to free up any resources start these up again- im really leaning towards this being a network limitation issue although i don't think im pushin the envelope very much- i am running traefik in the front but even if these 5 webapps are all on the traefik network- there shouldn't be a probelm. its good to note- on some other hardware i was getting a network allocation OOM error. im deparate to fix this- do i need to change to /16 segments or try with the dnsrr? im pretty new to docker swarm at scale and really need some help- i have people that want to push to production very shortly and i can't even spin up all our containers or confidently restart them! Im grateful for any help friends :-)
