DNS entry problems on a DC with attached iSCSI

We’ve recently set up a new DC with some iSCSI storage attached- single site, so the DC is providing DC & file storage.

In DNS, the LAN and iSCSI entries were showing up which was annoying, as pinging generally didn’t work because it would try to find the iSCSI network.

In short, you don’t need to mess around with any network connections or fiddle about in the registry- use method 2 from

https://support.microsoft.com/en-gb/kb/275554

Just stop the DNS on the DC listening on any port other than the LAN- including an IPv6 interfaces (in our case at any rate).

Advertisements

Windows Server 2012 Hyper-V clustering basics.

Right. First of thanks to Microsoft for making clustering way easier. However, there’s very little info about Server 2012- most of what you can easily find is all about Server 2008/ 2008 R2, which is (obviously) quite outdated.

Anyway, start at the start. I’m not saying any of this is right- or recommended, by the book etc- but it works:

  • When thinking about clustering Hyper-V, don’t get swayed by the Hyper-V side. You can’t just start clustering Hyper-V virtual machines. You need to start by clustering the Hyper-V hosts;
  • To do this successfully, you need at least 4 networks: CSV (Cluster Shared Volume), Live Migration, LAN, & iSCSI (ideally paired against different network cards for resilience);
  • You’ll need to install both Hyper-V & clustering roles on each host, AND the associated powershell cmdlet sets (for ease of use);
  • This is important: I’m basing this on EqualLogic SANs with the Dell Host Integration Tools kit. This seems to be important, because (I’m guessing) if you use Microsoft’s own iSCSI software, they recommend using multiple iSCSI subnets. After completely rebuilding the network infrastructure to accommodate this, it turns out the HIT kit really doesn’t like this set up so I’ve reverted back to a single iSCSI VLAN + subnet. Complete pain but at least it’s working again. With the HIT kit installed, run iscsicpl.exe and switch to the “Dell EqualLogic MPIO” tab. You should have 4 connections against every disk, unless the host you’re looking at doesn’t own the “Cluster Group” group in which case you’ll have 2 connections for the quorum disk. This will increase to 4 connections as soon as that host owns the Quorum. You’ll only ever have 4 connections to the quorum on the host that owns the cluster group;
  • To install clustering, PowerShell “Install-WindowsFeature Failover-Clustering” and “Install-WindowsFeature RSAT-Clustering-PowerShell” (I know you can chain these together);
  • To create the cluster, open up “Failover Cluster Manager”, right-click “Failover Cluster Manager” and choose “Create Cluster…”;
  • Follow the wizard through, for the time being only tick and assign an IP address to your preferred CSV/ Cluster and LAN networks;
  • Do not tick “Add all available storage to the cluster”;
  • Run the validation tool- hopefully everything is green-ticked;
  • Finish creating the cluster;
  • Once your cluster is created, right-click on “Disks” and choose “Add Storage”. At the next screen, pick just your Quorum disk;
  • Once your Quorum disk shows up as “Available Storage”, right-click the cluster itself , choose “More Actions”, pick “Configure Cluster Quorum Settings…” and then at the next screen choose “Add or change the quorum witness” then choose your Quorum disk;
  • Now you can add additional iSCSI LUNs as disks, right-click on them and “add to Cluster Shared Volumes” (or something similar);
  • At this point rename all the disks and networks to something useful, otherwise you’ll never know what you’re pointing at;
  • To configure the networks, make sure that:
  • Your preferred CSV/ cluster network is accessible only to the cluster, not clients;
  • Your preferred LAN network is accessible to the cluster AND clients;
  • Your preferred LiveMigration network is accessible only to the cluster, not clients;
  • Your preferred iSCSI network isn’t available to the cluster at all;
  • At this point, right-click on the “Networks” group and choose “Live Migration Settings…”;
  • De-select all networks apart from your preferred LiveMigration network”
  • The networks are assigned-allegedly- according to this article. My experience is that this is right. With a bit of luck, if you now PowerShell “Get-ClusterNetwork -cluster XXXX | Sort-Object Metric | FT Name, Metric, AutoMetric” you’ll see that the CSV/ Cluster network has the lowest metric, followed by the LiveMigration network, then iSCSI, then LAN.
  • This is the point at which you can start creating Hyper-V machines on those shared LUNs. So… instead of creating the machine on C:, or E: you create it on \\host\c$\ClusterStorage\VolumeX. This looks like local storage, but of course it’s just a mount point to an iSCSI LUN. Which is why it’s so easy to move virtual machines, because they don’t exist on any host;
  • If you need your VMs to move with the cluster nodes, you need to add each VM as a resilient VM inside the clustering tool. This then renders the Hyper-V management tools ineffective against the chosen machines as they are now a cluster resource, not a Hyper-V machine;
  • Also… you may get a lot of errors (event ID 1196) about failing to register the DNS name on some networks. I don’t know this for definite, but if you look at ipconfig all the private IP addresses have DNS servers set pointing to presumably imaginary IPv6 DNS addresses. I’m going to remove all the DNS servers from private ranges and see what happens.
  • Okay… I think (hope) the way around the 1196 errors is to use “Set-DnsClient -InterfaceAlias X -RegisterThisConnectionsAddress $False. I’m hoping this stops it trying to register with the false DNS servers, as there appears to be no way of setting DNS servers to be blank.

Live VHDX extension struggle

Given we had a server with a CIFS share that was fast approaching its limit (on the file server, not just the share) I built a Hyper-v machine around two iSCSI shares: one held the Windows guest, the other held the CIFS data. Now I knew the data was going to be about 4TB so created the CIFS share at that size, but because we’ve decided to use VHDX as the “visible” storage medium (rather than pass-through, purely for portability) I thought I’d start off with a 1TB VHDX file and then expand it “on the fly” to see how easy it was. I started RoboCopying all the data on the above CIFS share to the new VHDX, knowing it would start failing to copy at the 1TB mark.

And the answer was incredibly easy, if only I’d connected the 1TB VHDX to the guest’s SCSI connector first time. Basically, it’s impossible to disconnect an IDE drive while the VM is live, but no such problem with SCSI (should have realised this).

Step 1 was to use DISKPART on the guest to set disk 1 offline, then on the host ran Remove-VMHardDiskDrive -VMName xxxx -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 (no path for Remove-… as it’s only removing a disk from the guest). Next, I used a Resize-VHD script (possibly pointlessly as it was a dynamic VHDX, but it worked…or at least didn’t fail), used Add-VMHardDiskDrive -VMName xxxx -ControllerType SCSI -ControllerNumber 0 -ControllerLocation 0 -Path X:\xxxx.vhdx. I could then use DISKPART on the guest to switch the disk back online, extend it with server manager (on another machine- the storage server is Server Core) to- again deliberately- just 3.5TB to put limits on the copy process for experimentation. As soon as the extend had finished, the RoboCopy process kicked back in. All that’s left now is for the Windows DeDuplication to sort out the copied data and we should be ready to migrate over (I’m not sure how much it can compress the data given that it’s all unique images, which would already be considerably compressed through the jpg format etc. Currently running at 2% deduplication..)

iSCSI #01

Think I’ve finally nailed this. With help from various sites, I’ve now got two commands which seem to work after running the iscsicli addtarget command. It’s really important to count how many *s and spaces there are- so the first example starts off T<space>*<space> etc.

iscsicli PersistentloginTarget <IQN> T * * * * 0 2 0 0 * * * u p 1 * *

In order, 0 = set security to none, 2 = enable multipath, 0 = sets header digest, 0 = sets data digest, u is your CHAP username, p is your chap password, 1 = set authentication type to CHAP. I think you can ignore the *’s, they do things but not sure how important they are.

iscsicli loginTarget iqn * * * <INI> <INIPRT> * * * * * * * u p 1 * *

<INI> = initiator name which can be found with iscsicli listinitiators, <INIPRT> = I think should be 2, ensures you know which NIC port is connecting, and u, p and 1 as above.