AzureDatabricksBestPractices
AzureDatabricksBestPractices copied to clipboard
Azure
IP address requirements clarification
The IP address requirements section seems confusing:
Each cluster node requires
- 1 Public IP and 2 Private IPs
- For a desired cluster size of X: number of Public IPs = X, number of Private IPs = 4X; how is this 4X; per above, isn't it 2X?
Also when I provision a cluster with 2 nodes, it appears to use 4 IP addresses only - 2 public and 2 private. Per above, it should've used 2 public IPs and 4 private IPs or 8 private IPs, which doesn't seem to be the case.
Please clarify. Thanks.
Hi @rajesh-ca Each VM has 2 NICs, one public and one private. The public NIC has both public and private IP (1pu + 1pr) The private NIC has only private IP (1pr) So, for each VM, you should have 1 public IP and 2 private IPs.
For your second question, in the second image, you are only seeing the private IP of the default NIC, which is the public I think. If you go to the networking part of any VM of the cluster, you can see both NICs.
As for the part "number of Private IPs = 4X", I'm afraid that I'm as lost as you, I though it was 2x. Maybe it's a bug, but I'm not sure.
Thanks @jdocampo for clarifying. I see that now within the UI. Maybe the wording in the guide should contain what you've written - it's much easier to understand and visualize that.
According to the article: The 4X requirement for Private IPs is due to the fact that for each deployment: Half of address space is reserved for future use The other half is equally divided into the two subnets: private and public
Given that the private & public subnets masks must be equal, per the article, this seems to be a waste of IP addresses provisioned in the public subnet.
The information from the different sources is confusing. https://docs.microsoft.com/en-gb/azure/databricks/administration-guide/cloud-configurations/azure/vnet-inject#requirements states that with /26 subnets, you can have a maximum of 64 nodes active at a time. This is already wrong since such subnets have only 62 usable IPs. In any case it contradicts the assertion in this repo, that we need 2x more IP addresses in the private subnet.
Added issue https://github.com/MicrosoftDocs/azure-docs/issues/57577
According to virtual network faq, 5 addresses are reserved by Azure, so that would make 59, not 62 usable IP's.
