Discussion: Static IP address for Hyper-V

[bbrala]

I was looking into setting a static ip address for the machine's but im not entirely sure how we would go about that.

There are a few different ways to setup your network switch and would we implement different ways to set the ip address based on the network configuration?

Currently Vagrant doesnt manage the switches in any way, should we even try and start doing that.

When it is a extenal switch you basically part of the same network as the host, which usually means there is DHCP and a static IP is limited to your subnet. Also there is the possibility of collisions.

An internal switch has possiblities, but would require the user to share his internet on his adapter manually to get internet on the machine.

Another possibility is doing an internal switch and create a nat network, which would give some flexiblity, but i know that docker does this also and would probably conflict with that since windows can only handle one NAT network.

Anyways, i'm not entirely sure what the best way to go about it is.

Perhaps you have some insight @PatrickLang :)

[PatrickLang]

Yeah, I've been thinking about this. My goal is to get @StefanScherer 's windows10-docker-swarm working on Hyper-V.

Things that I know need to change:

1. Actually implement networks. Here's some notes on what needs to change https://github.com/PatrickLang/vagrant/commit/d26e3fe0b5857acaf7e7dd5f27484fb11de8e1d2.

• I will probably submit a doc update to explain Hyper-V networking support as it is today

2. Support creating a private_network
3. Set private IPs using one of:

• Hyper-V's built in support
• Using the guest plugin capability cap/configure_networks

Questions:

Do you have any opinions on 3? It seems like the guest plugins support setting IPs for more operating systems than the Hyper-V Integration Components, but I have been struggling to find docs and examples of how to use them.

If a VM was exported with multiple NICs - should those settings (including MAC & IP) be preserved when importing it, or should the network configuration in the Vagrantfile take precedence? As I implement 1, I haven't decided if I should keep or delete existing NICs. When I'm trying to build up a lab with precise network settings I think I would rather create new NICs with the exact settings instead of trying to reuse the old ones.

[bbrala]

I've looked into how Vagrant does things right now and from what i understand, it indeed has implementations for setting the network for the guests. I would probably look at how the implementation for VirtualBox is and use that for Hyper-V. This will keep the basic code for setting the network in the guest the same which i feel is a pro.

https://github.com/mitchellh/vagrant/blob/9c299a2a357fcf87f356bb9d56e18a037a53d138/plugins/providers/virtualbox/action/network.rb#L132

I think there is a lot of logic there that can be reused for the Hyper-V logic. We only would need to be very carefull in what different options there are on the machines of the users. I really think we can screw up the networking of the users if we do too much.

I've even had moments when i was manually setting up my network for my VM that i screwed up my network settings after enabling or disabling Hyper-V networks. We don't want that to happen to an unsuspecting user.

Regarding the deletion of NIC's, i feel you should be carefull deleting nics. Perhaps make it an option because unless you make every little think configurable in the config file you might not be able to conserve a specific setup a box might need. I really think rebuilding the NIC's from scratch always will become a problem in the future, and for most people they just need the VM as close to how it was exported as possible, since that was how the software was set up for them.

When i implemented support for VMCX i started of with rebuilding the VM from the settings of the old one, but that really ends up making things harder and more code.

[s0lucien]

I have achieved this using https://sourceforge.net/projects/dhcpserver/ and some powershell scripts that manage it. I am not sure whether it's the right path to go , or whether it's fine to add it as a dependency but it works pretty well without any bugs. If you are interested, i could spend some time adding it into a fork.

The advantage is that it can be used for any Win/Linux box, since they all ask for an IP address via DHCP

What it does is :

1. create internal switch (using a naming scheme which involves the machine name, so it is possible to keep track of it)
2. write the info in the OpenDHCP INI file (hyper-V will give the VM some MAC address, it is possible to retrieve it at run-time using powershell. Using the MAC address it is possible to let the DHCP server know it's supposed to assign a specific IP address to it)
3. start the dhcp server on the host endpoint of this internal switch (it runs as a service)

[bbrala]

I'm not sure adding another dependency is really the way to go.

From what i understand in the new Windows 10 update there has been an update to NAT networking, this allows for multiple NAT networks to exist, which makes the whole static IP problem quite a lot easier since we can control the NAT network as we see fit.

https://blogs.technet.microsoft.com/virtualization/2017/04/13/whats-new-in-hyper-v-for-the-windows-10-creators-update/

Multiple NAT networks and IP pinning

NAT networking is vital to both Docker and Visual Studio’s UWP device emulators. When we released Windows Containers, developers discovered number of networking differences between containers on Linux and containers on Windows. Additionally, introducing another common developer tool that uses NAT networking presented new challenges for our networking stack.

In the Creators Update, there are two significant improvements to NAT:

Developers can now use for multiple NAT networks (internal prefixes) on a single host. That means VMs, containers, emulators, et. al. can all take advantage of NAT functionality from a single host. Developers are also able to build and test their applications with industry-standard tooling directly from the container host using an overlay network driver (provided by the Virtual Filtering Platform (VFP) Hyper-V switch extension) as well as having direct access to the container using the Host IP and exposed port.

I think if we can use this to create an easy quick internal network for Vagrant to use for the machine. It should check for overlap in NAT networks though, since that might cause problems. If a NAT already exists which would serve that IP then just use that. Otherwise create a network.

Cleaning up should be a breeze if we use well defined names for the network.

The only catch is you need a recent version of Windows 10. But i feel that this should not really be an issue that stops Vagrant from having this.

[chrisroberts]

The only catch is you need a recent version of Windows 10. But i feel that this should not really be an issue that stops Vagrant from having this.

I think the important part to this is ensuring graceful fallbacks (even if it's just explanatory error messages) for when the feature is not available. I'm very much in agreement with the approach though and prefer it over adding more dependencies.

[bbrala]

@chrisroberts ok, fair enough, i will try and find some time to get something up and running. I agree on the fallbacks ofcourse.

@PatrickLang I've run through your linked commit, is there any reason why you want support for multiple networks, i haven't really seen machines that need multiple networks really.

[PatrickLang]

@bbrala multiple networks are very helpful when working with Windows Active Directory where I need to have a tightly integrated Kerberos, DNS, and DHCP setup spanning multiple VMs.

I typically have 1 network that's "public" relying on DHCP & routing to the public internet such as public wifi. The other network is private just for the machines using the AD infrastructure.

Here's an example of a doc I'm doing at work targeted to Azure, but I'd like to fully automate it with Packer+Vagrant so others can run it on a Win10 laptop.

[GuyPaddock]

Also useful, as @bbrala pointed out, is that HyperV has support for a NAT (with port forwarding) under the hood. Here's how to use it in Windows 10: http://www.thomasmaurer.ch/2016/05/set-up-a-hyper-v-virtual-switch-using-a-nat-network/ https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/setup-nat-network

[bbrala]

@PatrickLang Hmm, i'll think about how to go about doing it for multiple network interfaces, but perhaps this should be built in 2 steps. First one interface, then add the possibility for a second one. Although i could make sure we use a hash of the settings or something to identify the network interface. Not sure yet.

Time is a little short at the moment, but I will have a go at this eventually :)

[bbrala]

Walking through the docs and code for networking i noticed multiple networks is supported in VirtualBox, think it would be weird to change the interface much for HyperV so we need multiple networks right away.

Other than that i'm still a little torn in how smart we would make the code. I'm not entirely sure if for example Vagrant with virtualbox does anything to fix/find collisions in private networks. If you make 2 overlapping networks that would be created in VirtualBox for example, but i guess that isn't that hard to just test.

[PatrickLang]

I forked the NAT topic to a new issue. I think there may be a quick option to implement that first while we think about multiple nic and static ip here.

[bbrala]

@mwhooker; you did the implementation of HyperV for packer. Do you have any insight on this chicken/egg issue regarding static IP's?

[mwhooker]

@bbrala regretfully, I only merged the PR. @taliesins is the brains behind the endeavor. Maybe he knows?

[bbrala]

Ah sorry, it has been a while, i remembered incorrectly, sorry about that :)

[taliesins]

@bbrala two potential approaches are:

1. Scripts set ip address of nics. No changes to Packer.
2. Assign mac address to nics, and then use dhcp with ip reservation (this is the one I prefer, as it means base box will not need to be modified). Changes required to Packer builders.

We then need to change Packer to add an option to specify the management ip address to use instead of it always polling the hypervisor api for the ip address.

[anlx-sw]

to contribute to this discussion - i played around with vagrant and hyper-v lately.

since windows server 2012 hyper-v allows to set the ip of guests from the host: http://www.ravichaganti.com/blog/set-or-inject-guest-network-configuration-from-hyper-v-host-windows-server-2012/

i tested this script method with a linux guest on my hyper-v and it worked as proposed.

there also seems to be a workaround to set the vswitch used mentioned in this discussion https://github.com/mitchellh/vagrant/issues/7915 config.vm.network "public_network", bridge: "VirtualSwitchName"

it is possible to create a NAT-Vswitch via Powershell: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/setup-nat-network

so i think all pieces to automatically create a nat-vswitch (or use a pre-created one) and apply the config to the vagrant guest would be there. so the same things which are done with virtualbox could be done with hyper-v.

or am i missing any limitations on the hyper-v side?

[bbrala]

The script you posted doesn't work for me in Windows 10 unfortunately. We did have a look at setting ip's through hyperv but OS support is a bit limited regarding that. Perhaps we need to accept that fact for now.

[carlpett]

This Hyper-V post contains an interesting piece about new functionality ahead: link. In the comments they mention that this new "special" virtual switch will do DHCP.

[bbrala]

Good catch, that will make things a lot easier. Patience i guess

[rogermartensson]

Is there any progress in this? I would like to get setting at static IP working since I need to do this manually today. DHCP is not a blocking issue for me.

I already have a NAT-enabled vSwitch (via Docker) and I think that vagrant should error-out if the vSwitch IP and VM-IP is not in the same subnet (if possible).

Creation of NAT-enabled vSwitches are also documented on blogs around the world. So if no NAT-enabled vSwitch can be found and used then maybe it should be created. Windows 10 does not support more than one NAT-enabled vSwitch but I think I saw something about it getting support for it in the future? (then this possibility should be checked before creation/use)

[bbrala]

The new Fall Creators update might help us getting this to work. There now is a Default switch available which is always present. This switch would enable us to connect to the VM regardless of current configuration on the host. This means we could actually connect and set the IP address on the selected switch without a problem.

This would mean some refactoring though, since this would only work on machines with Windows 10 Fall creators update. Also i'm not entirely sure yet how we would implement using that interface as a managing interface for the VM. I would need to look into how the current VirtualBox implementation does this. From what i remember it always requires a NAT switch for exactly that reason.

The update is pretty fresh, i haven't had time to look into this yet but i feel we could get this implemented using the new default switch.

[rogermartensson]

VirtualBox implementation requires two NICs. First one is host-only and the second is used for communicating with "the world". VirtualBox can also use an Internal Network that is ever more strict than host-only. Is hyperv set up the same way?

Host-only sounds really like a hyper-v internal network with the infamous exception of missing DHCP. The reason "vagrant up" works at the moment seems to be that we get an ipv6-address that is used during setup. (ubuntu vms in my case)

So if first nic is management only maybe create a vSwitch with Internal Network if one doesn't exists? or maybe create a vagrant internal network that is always used with or without the Default Switch?

The public network is then later used to connect to whatever network that is needed. May it be internal, private och external. If Default Switch exists and no config says otherwise, then nic 2 is connected to it. If no default switch is found, nic2 is in "unconfigured" state and Vagrantfile needs be be updated with right vSwitch to use.

Since we have no DHCP you Have to insert both private and public ip# in Vagrantfile.

My ruby is almost non-existent but my powershell is most acceptable. Since I would like this to get better I will try to test whatever you come up with. :)

[gurry]

Has any work been done on this guys?

[PatrickLang]

I haven't done anything yet. I was hoping the NAT changes in the Windows 10 & Windows Server 1709 release would make this easier but not quite yet.

Here's my current proposal:

I think I may be able to make this work for a second NIC. The first NIC would still use an external switch or the Windows built-in nat on "Default Switch". This would give the VM outbound connectivity. The second NIC could use a static IP on a private network. This would be good for labs where you want the VMs networked to each other with static IPs.

Would that be helpful for your case?

[gurry]

Yeah. This would work for me.

[StefanScherer]

The second NIC is used in the docker-swarm scenario for VMware as well. The Vagrant VMware plugin cannot handle setting a fixed IP address in Windows VM's, but my workaround to do this in a provision script https://github.com/StefanScherer/windows10-docker-swarm/blob/master/Vagrantfile#L46 scripts/fix-second-network.ps1 helps at least for Workstation and Fusion.

[jurosh]

Sorry to join discussion without additional tech info, but would like to kindly ask if there is any progress, or plans? As more of you noted, there is way how to make it work (and be fast) with two NICs like is in VirtualBox so curious if this is might be priority in near future. For me this is (currently) the most annoying thing while doing development on Windows, cause I need to always manually update hosts file and few other configs with automatic IP from hyperv.

[bbrala]

I'm not sure there is people working on this right now. At least, i'm not. Perhaps i can help by sharing how we handle this at the moment. Since we also need to update the hostfiles we use a plugin to do that for us.

The plugin is vagrant-hostmanager.

In our vagrant file:

if Vagrant.has_plugin?("vagrant-hostmanager")

    require_relative "lib/project_hosts"
    config.hostmanager.enabled = false
    config.hostmanager.manage_host = true
    config.hostmanager.manage_guest = true
    config.hostmanager.ignore_private_ip = false
    config.hostmanager.include_offline = true
    config.hostmanager.aliases = PROJECT_HOSTS

    config.vm.provision :hostmanager, run: 'always'
  end

The lib/project_hosts file, which reads all directories above the vagrant repository to add those to our hostfile.

projects_dir = File.expand_path('../..', File.dirname(__FILE__))

# Get list of all directories in folder above vagrant project repository.
paths = Dir.entries(projects_dir).select {|entry| File.directory? File.join(projects_dir,entry) and !(entry.start_with?(".")) }

PROJECT_HOSTS = Array.new
paths.each do |path|
  PROJECT_HOSTS.push("#{path}.dev.tld")
end

require 'json'
extra_hosts_json = File.read('provision/configuration/hosts.json')
extra_hosts = JSON.parse(extra_hosts_json)

PROJECT_HOSTS.push(*extra_hosts)

And provision/configuration/hosts.json for some hardcoded uri's

[
  "testing.weirddomain.dev.tld",
  "dev.tld"
]

Perhaps this helps.

[chrisroberts]

Hi everyone,

Just wanted to drop a note that I'm going to be picking up a handful of Hyper-V related tasks over the next few weeks and this will be one of them.

Cheers!

[bbrala]

That is awesome to hear.