Use .To4() on serverId
The server id should be four octets, per RFC1533, so call .To4 on it to ensure we have a v4 address and not a mapped v4 address.
I wasn't entirely sure if you wanted to keep the API, .To4() can return nil if serverId is a v6 address, in which case the ReplyPacket should probably fail. Not using .To4() means you can end up sending a 16 octet long ServerIdentifier, which doesn't work with all DHCP clients (at least the ISC DHCP client is unhappy with it).
I'm going to think about this some more. If IPv4 nochange. If mapped v4 corrects a potential bug. If IPv6 goes from invalid serverId (too long) to invalid serverId (zero length).
However behaviour would be inconsistent between ReplyPacket and .AddOption which could make a bug harder to find.