Windows Server 2012 Hyper-V clustering basics.

Right. First of thanks to Microsoft for making clustering way easier. However, there’s very little info about Server 2012- most of what you can easily find is all about Server 2008/ 2008 R2, which is (obviously) quite outdated.

Anyway, start at the start. I’m not saying any of this is right- or recommended, by the book etc- but it works:

  • When thinking about clustering Hyper-V, don’t get swayed by the Hyper-V side. You can’t just start clustering Hyper-V virtual machines. You need to start by clustering the Hyper-V hosts;
  • To do this successfully, you need at least 4 networks: CSV (Cluster Shared Volume), Live Migration, LAN, & iSCSI (ideally paired against different network cards for resilience);
  • You’ll need to install both Hyper-V & clustering roles on each host, AND the associated powershell cmdlet sets (for ease of use);
  • This is important: I’m basing this on EqualLogic SANs with the Dell Host Integration Tools kit. This seems to be important, because (I’m guessing) if you use Microsoft’s own iSCSI software, they recommend using multiple iSCSI subnets. After completely rebuilding the network infrastructure to accommodate this, it turns out the HIT kit really doesn’t like this set up so I’ve reverted back to a single iSCSI VLAN + subnet. Complete pain but at least it’s working again. With the HIT kit installed, run iscsicpl.exe and switch to the “Dell EqualLogic MPIO” tab. You should have 4 connections against every disk, unless the host you’re looking at doesn’t own the “Cluster Group” group in which case you’ll have 2 connections for the quorum disk. This will increase to 4 connections as soon as that host owns the Quorum. You’ll only ever have 4 connections to the quorum on the host that owns the cluster group;
  • To install clustering, PowerShell “Install-WindowsFeature Failover-Clustering” and “Install-WindowsFeature RSAT-Clustering-PowerShell” (I know you can chain these together);
  • To create the cluster, open up “Failover Cluster Manager”, right-click “Failover Cluster Manager” and choose “Create Cluster…”;
  • Follow the wizard through, for the time being only tick and assign an IP address to your preferred CSV/ Cluster and LAN networks;
  • Do not tick “Add all available storage to the cluster”;
  • Run the validation tool- hopefully everything is green-ticked;
  • Finish creating the cluster;
  • Once your cluster is created, right-click on “Disks” and choose “Add Storage”. At the next screen, pick just your Quorum disk;
  • Once your Quorum disk shows up as “Available Storage”, right-click the cluster itself , choose “More Actions”, pick “Configure Cluster Quorum Settings…” and then at the next screen choose “Add or change the quorum witness” then choose your Quorum disk;
  • Now you can add additional iSCSI LUNs as disks, right-click on them and “add to Cluster Shared Volumes” (or something similar);
  • At this point rename all the disks and networks to something useful, otherwise you’ll never know what you’re pointing at;
  • To configure the networks, make sure that:
  • Your preferred CSV/ cluster network is accessible only to the cluster, not clients;
  • Your preferred LAN network is accessible to the cluster AND clients;
  • Your preferred LiveMigration network is accessible only to the cluster, not clients;
  • Your preferred iSCSI network isn’t available to the cluster at all;
  • At this point, right-click on the “Networks” group and choose “Live Migration Settings…”;
  • De-select all networks apart from your preferred LiveMigration network”
  • The networks are assigned-allegedly- according to this article. My experience is that this is right. With a bit of luck, if you now PowerShell “Get-ClusterNetwork -cluster XXXX | Sort-Object Metric | FT Name, Metric, AutoMetric” you’ll see that the CSV/ Cluster network has the lowest metric, followed by the LiveMigration network, then iSCSI, then LAN.
  • This is the point at which you can start creating Hyper-V machines on those shared LUNs. So… instead of creating the machine on C:, or E: you create it on \\host\c$\ClusterStorage\VolumeX. This looks like local storage, but of course it’s just a mount point to an iSCSI LUN. Which is why it’s so easy to move virtual machines, because they don’t exist on any host;
  • If you need your VMs to move with the cluster nodes, you need to add each VM as a resilient VM inside the clustering tool. This then renders the Hyper-V management tools ineffective against the chosen machines as they are now a cluster resource, not a Hyper-V machine;
  • Also… you may get a lot of errors (event ID 1196) about failing to register the DNS name on some networks. I don’t know this for definite, but if you look at ipconfig all the private IP addresses have DNS servers set pointing to presumably imaginary IPv6 DNS addresses. I’m going to remove all the DNS servers from private ranges and see what happens.
  • Okay… I think (hope) the way around the 1196 errors is to use “Set-DnsClient -InterfaceAlias X -RegisterThisConnectionsAddress $False. I’m hoping this stops it trying to register with the false DNS servers, as there appears to be no way of setting DNS servers to be blank.

KB2837618 and KB2837643

I don’t quite know what’s going on yet, but it looks like Microsoft have had another outing with the above updates.

We know the security update 28387618 kills Outlook 2013; what’s unclear is the part 2837643 plays. It looks like, somehow, 2837643 is the “master” update which somehow also installs 2837618; ?. We’ve also had a lot of people complaining about Out-Of-Office not working which I’ve also found, whether this is anything to do with these 2 updates is unknown as yet.

I’ve just built a new W7 PC with Office 2013 and am patching it now (minus these two). Then it’ll be a case of  applying different combinations of the two above updates manually to see what happens.

UPDATE 22/11/2013. It’s not looking good for these updates. My W7 test machine has Office, but not the above two updates. I can open up Out-Of-Office on this one, simultaneously try it on my W8 machine (which does have the updates) and it doesn’t open on the W8 one. So… it’s looking likely that these updates can (a) cause Outlook to crash and (b) prevent access to Out-Of-Office.

Backup Exec 2010 R3 SP3 on Windows Server 2012 Core using Windows deduplication

Hmmm… it’s looking like Symantec have a problem with Server 2012 Core with Windows deduplication turned on.

We’d been advised that the above release of BE 2010 would at least let us back up our 2012 boxes, so after installation quickly installed the agent on our 2 2012 boxes: 1 built GUI, 1 built Core.

The GUI one seems fine, backups are taking normalish amounts of time and restores work. The Core box, however, implies that a full backup o four data (4TB?) would basically never finish, but then the job finishes really quickly. And… the restores don’t work. Or at least not fully, which is why we back things up to be honest.

3 calls to Symantec later and there’s still not much to report from them, but doing some testing I found that:

  • Build a Hyper-V test server with deduplication against a 40GB disk, got some test data on it (90MB image files) and set deduplication off. Brilliant- 94% deduplication;
  • Installed BE agent, ran backup, ran restore- nothing, just corrupt files;
  • Installed the full GUI role, re-ran backups and restores, nothing again;
  • Uninstalled BE agent, rebooted, reinstalled BE agent, nothing again;
  • Rebuilt server as Server 2012 GUI (from start), installed BE agent;
  • Ran backups/ restores, worked! despite some strange VSS errors;

I can only think that BE just really doesn’t like Core. I tried a small (2.7GB) backup of some of our production data and the results were really random- some of the restored Word documents worked, some didn’t. Same for Excel files. I couldn’t get any PDFs to open, whereas JPGs only seemed to fail if generated as user content, not as embedded software files.

Currently no idea- although I suppose it’d be advisable not to use the above BE version to back up any production data that sits on a Windows Server Core DeDuplication volume for now.

UPDATE 15NOV2013: it looks like Backup Exec (or at least our BE environment) has a serious problem with Server Core- but at least it’s easy to fix at least. The first time the BE agent gets pushed out, it DOESN’T install the BE 2010 R3 SP3 update- which is why the backups are corrupt, because support for Server 2012 only came in with SP3 so the agent remains on 13.0.5204.0. Re-installing the agent however does bring the agent up to 13.0.5204.1225, and magically the backups start to work. Symantec tech support are trying to recreate this is in a lab environment, but it maybe it’s just our environment this affects. As before, Server 2012 GUI seems to have no issues. This is tediously repeatable… I’ve just done it again with another brand-new, unpatched Core machine and the same thing has happened.