Bye, bye Service Console

This isn’t just another article about vSphere 5. It’s not my aim simply to rattle off a list of new and improved features. There are probably a plethora of those posts out there already, some better than others – use Google to find them. Smile

I was inspired to write this after I saw white paper linked to on Twitter about the differences between ESX and ESXi written by Global Knowledge. Actually it was the responses to that article that prompted me.

Of all of the many changes announced today, it is the departure of the Service Console that is perhaps the most significant in my view. It may not be a new, super-whizzy feature and many people are already using ESXi and might be thinking “so what”. For me, removing the COS / SC / Service Console is significant for two reasons.

Firstly, no longer can the unenlightened refer to vSphere as “Linux based” or “Unix based”. ESX never really was that and it wound me up more than it should have done when people got it wrong. I have a strong Microsoft background (although I can hold my own when it comes to Unix / Linux OSs) but in some companies that seemed to exclude me from touching VMware infrastructure even though I knew all about it. Ok, that’s more of a pet peeve than a significant reason.

My other one though is that to me this signifies the direction in which VMware are taking virtualisation. It may even be more accurate to say that dropping the Service Console marks the completion of a transition or a journey. ESX was an excellent way for enterprises to make efficient use of powerful hardware amongst many other benefits but there was always a glass ceiling there.

Some say that ESXi isn’t as flexible because it doesn’t have a command line. Perhaps they don’t realise that it really doesn’t need one. Occasionally, when things go wrong, SSH access is useful and ESXi does provide that. But for day to day usage there are better and more efficient ways to manage a Virtual Infrastructure (PowerCLI, vCenter or one of the many VMware and 3rd party products) and dropping the Service Console is both recognition and reinforcement of that. ESXi turns the hypervisor into commodity or utility platform, it’s not a management interface in its own right. ESXi is a foundation stone for building a dynamic, automated infrastructure so forget about the Service Console now.

In some ways I’ll be sorry to see it go but ESX was a means to an end and it has reached its end.


VMware Version Numbers

I’ve been invited onto the beta of the next version of VMware Workstation. There’s nothing much unusual about that. I have a licensed copy of it and have participated before. What’s different this time is the version number.

VMware are (at present) calling this next version “VMware Workstation 2011”. (I don’t think I’m breaking the EULA or anything by mentioning this.) This would, if they stick with it, represent a change from previous versions (4.x, 5x, 6.x and 7.x to name but a few). This will be a marketing tactic. I’m not about to try and second guess what it might be. I’m more interested in whether this be applied only to the “consumer” products (e.g. Workstation, Fusion, Player etc) or if future versions of the “enterprise” products will adopt a similar convention.

Imagine that your next datacenter upgrade is from vSphere 4.1 to vSphere 2011. Will it happen? I couldn’t say for sure. VMware have more or less aligned their version numbers in the recent past. Most of the products are 4.1 or similar. Could this be in preparation for a mass change to version 2011?

On the other hand, there are many rumours that the next version of vSphere will be 5. The next version of SRM will 5 also according to many of the same rumours. Assuming that they’re true, don’t expect ESX 2011.


ESX and ESXi AD Integration

If, like me, you make your ESX / ESXi server passwords nice and complex you end up having to dig them out of a password safe every time you want to connect directly to one of them. Or you have an SSH connection manager of some sort perhaps. Even then, there will come a time when you want to connect directly and that 16 character, random, mixed case password just isn’t memorable enough for you to use it.

Luckily if you’re running vSphere 4.1 or later you can configure your hosts to use AD authentication. Hooray!

Obviously there are security implications to doing this. Each environment is different and any risks should be considered before implementing this.

So, let’s deal with the pre-requisites first. There are three of those:

  1. Time synchronisation – Your ESX / ESXi hosts must be synchronised to a time source and they should be in sync with the domain controllers in your AD domain. The authentication mechanisms in AD are very sensitive to time differences. Actually, that’s a delicate way to put it. It won’t work if the time is wrong.
  2. Name resolution – The ESX / ESXi hosts will use DNS to locate domain controllers for whichever domain you configure them to use. Therefore each host must have a working DNS configuration.
  3. An AD group – Sadly there is a limitation here. AD users that you wish to grant administrative access of your hosts to have to be a specific group in AD called “ESX Admins”. This is not obvious in the documentation however.

The same document then talks you through configuring each host. It’s fairly simple.

Just find the “Authentication Services” option on the “Configuration” tab for each host. By default it will look like this:

Click on the properties link to edit the “Authentication Services Settings”. In the windows that opens, select “Active Directory” as the service type. Then enter the FQDN of your AD domain into the domain field and click the “Join Domain” button.

Finally you just need to enter the credentials of an account permitted to join the ESX host to the AD domain.

Once the task in vCenter completes (it can take a little while), just refresh the “Authentication Services” page and you’ll be able to see that the host is now joined to the AD domain.

All good unless you have a lot of hosts to work through. In which case, you might want to check out LucD’s very handy PowerCLI script to join hosts to an AD domain.


PowerCLI: Adding New PortGroups

I’m sure that someone somewhere has written a script exactly like this in the past but I wanted to write my own for a number of reasons. While I’ll probably never be a PowerCLI hero, it really doesn’t hurt to keep in practice and hone your skills.

Let’s start with what I want to accomplish. I’m working with an ESX 3.5 web hosting environment and there’s a new project in the pipelines. A brace of new servers are required and they’ll be on a new VLAN. The VLAN has been created and configured on the various switches that the ESX hosts connect to but now of course a corresponding PortGroup is required. (Actually two are needed – complex project.) Now we’re not talking about a huge number of hosts here. It would probably only take 10 minutes to do it by hand using the VI client. It’ll take me longer to write this post! However, it is something that happens relatively often in this environment so it’s worth taking the time to write a script. Continue Reading


VMTools Version Mismatch

Before I get started, I just want to say that I don’t know for sure if this is an issue that is causing problems for some VMs that I have been looking at or not. I’d welcome any feedback, comment or help anyone has to offer on the topic. I thought that I’d share this just in case it helps anyone at any point.

The environment that I’ve been working in is a shared web hosting platform built on ESX 3.5 and vCenter 2.5. Recently the focus of effort has been on data centre migrations, service migrations and office moves. Business as usual (BAU) activities have taken a back seat because of resource constraints. That includes upgrading the virtual infrastructure to the latest version. Currently vCenter is at Update 4 (build 147633) and needs to be brought up to date following the release of Update 5 a few weeks ago. The ESX hosts are slightly further downlevel however. Two thirds of them are on Update 3 (build 123630) with a couple of extra patches applied to bring them up to build 143128. The other third is on Update 2 (build 110268).

Windows Server 2008 R2 has recently started to be used for some of the VMs and this is not officially supported until Update 5. Some of these VMs have had minor issues, things like random slowness and occasional failures to remove snapshots following their scheduled backups. While looking into these issues I discovered quite a variety of VMTools versions being used.

Ideally all ESX hosts in a cluster should be on the same update level. The version of VMTools that a VM uses should ideally match the lowest build version of ESX that is being used in a cluster as the VM could potentially run on any host in the cluster. On one VM vCenter said that the VMTools version was “OK” but I wanted to run a manual “Install / Upgrade” on the VMTools anyway just to run a repair and make sure that the drivers were installed correctly. When I tried to do this, the installer popped back an error stating that a newer version was already installed.

That got me thinking, which versions are installed on all the VMs? Now I could have used the excellent RVTools or something like that or I could craft my own brief PowerCLI script. I couldn’t resist the latter…

[ps]$vmhosts = get-vmhost | sort Name

$myCol = @()
foreach ($vmhost in $vmhosts)
$vmhostdetail = $vmhost | get-view
$versionmajor = $vmhostdetail.config.product.version
$versionbuild = $vmhostdetail.config.product.build
$vms = $vmhost | get-vm | sort Name
foreach ($vm in $vms)
$myObj = "" | Select Host,Version,Build,Guest,ToolsVersion
$vmdetail = $vm | get-view
$vmtoolsversion = $vmdetail.config.tools.toolsversion
$myObj.Host = $vmhost.Name
$myObj.Version = $versionmajor
$myObj.Build = $versionbuild
$myObj.Guest = $vm.Name
$myObj.ToolsVersion = $vmtoolsversion
$myCol += $myObj

$myCol | Export-Csv -NoTypeInformation "C:VMToolsVersions.csv"
ii "C:VMToolsVersions.csv"[/ps]

Running this script produces a CSV file with a line for each VM show the host, host version and build number, guest name and VMtools version for each VM. I’ve simplified the output that I got to remove the host names and guest names and also I’ve removed duplicate combinations of version numbers. That just leaves the following:


This shows the different combinations of ESX hosts versions and VMTools versions being used. What’s very interesting are the VMs running VMTools version 164009. That is the build number for vSphere 4.0.0!

I know how this has come about. Rather than using templates, the preference here is to use a Windows ISO file that has been customised to be a practically unattended install that also includes a number of automatically installed applications. One of these is the VMTools installation. As there is a vSphere infrastructure here also the ISO has been modified to include the latest tools version so that one ISO fits all. But does it? The VMs are running butis the version of VMTools causing problems? That’s the bit that I don’t know the answer to yet. If anyone does know, please feel free to comment.

For now the way forward is going to be creating multiple ISO files or removing the VMTools installation from the ISO altogether. That way new builds will get their VMTools installed from the ESX server that they’re running on. An upgrade to Update 5 is also called for and with next week looking to be a bit quieter I think that it’ll be planned then to be executed early in the new year.


ESX 3.5 U5

I mentioned ESX 3.5 Update 5 only yesterday in my post about VMtools on Windows 2008 R2. Little did I know that 16 hours later I’d be writing about it again to say that it had been released!

The update can be downloaded from VMware’s website as usual. Shamelessly copied from the release notes, here’s what you can expect to have changed:

Enablement of Intel Xeon Processor 3400 Series – Support for the Intel Xeon processor 3400 series has been added. Support includes Enhanced VMotion capabilities. For additional information on previous processor families supported by Enhanced VMotion, see Enhanced VMotion Compatibility (EVC) processor support (KB 1003212).

Driver Update for Broadcom bnx2 Network Controller – The driver for bnx2 controllers has been upgraded to version 1.6.9. This driver supports bootcode upgrade on bnx2 chipsets and requires bmapilnx and lnxfwnx2 tools upgrade from Broadcom. This driver also adds support for Network Controller – Sideband Interface (NC-SI) for SOL (serial over LAN) applicable to Broadcom NetXtreme 5709 and 5716 chipsets.

Driver Update for LSI SCSI and SAS Controllers – The driver for LSI SCSI and SAS controllers is updated to version 2.06.74. This version of the driver is required to provide a better support for shared SAS environments.

Newly Supported Guest Operating Systems – Support for the following guest operating systems has been added specifically for this release:

For more complete information about supported guests included in this release, see the VMware Compatibility Guide: http://www.vmware.com/resources/compatibility/search.php?deviceCategory=software.

  • Windows 7 Enterprise (32-bit and 64-bit)
  • Windows 7 Ultimate (32-bit and 64-bit)
  • Windows 7 Professional (32-bit and 64-bit)
  • Windows 7 Home Premium (32-bit and 64-bit)
  • Windows 2008 R2 Standard Edition (64-bit)
  • Windows 2008 R2 Enterprise Edition (64-bit)
  • Windows 2008 R2 Datacenter Edition (64-bit)
  • Windows 2008 R2 Web Server (64-bit)
  • Ubuntu Desktop 9.04 (32-bit and 64-bit)
  • Ubuntu Server 9.04 (32-bit and 64-bit)

Naturally you’ll need to upgrade vCenter to Update 5 to gain some of these benefits. The release notes for that mention only one significant enhancement:

Support for High Consolidation in VMware HA Clusters – VirtualCenter 2.5 Update 5 includes significant performance and scalability improvements to VMware HA. Use VirtualCenter 2.5 Update 5 for environments with more than 35 virtual machines per host in an HA cluster.
For information on the ESX Server host settings required for this scalability improvement, see ESX Server host settings required for environments with up to 80 virtual machines per host in an HA Cluster (KB 1012002).

I think that there is a good chance that Update 5 may be the last major update that the 3.5 line of products receives. Or at least it will be for some time. I’ll have some upgrades to do as a result of this release but I’m pushing for upgrades to vSphere like crazy. You know it makes sense.


Installing VMtools on 2008 Server R2 Core

Have you ever wondered how to install VMtools on a Windows 2008 Server R2 Core guest running on an ESX 3.5 host? “Hang on,” you say, “R2 is officially supported on ESX 3.5!” And you’d be right, for the moment at least. ESX 3.5 U4 does indeed not support R2 officially yet although VMware support hinted that U5 is due at some point. Still, the customer is king and mine wanted an R2 VM.

First things first, the WoW64 Execution Layer needs to be installed and the server rebooted to allow R2 to run 32 bit applications.

[text]Start /w ocsetup ServerCore-WOW64

shutdown /r /t 0[/text]

Once the reboot is complete and you’ve logged in again, select to Install the VM Tools for the VM. Nothing will actually happen except that ESX will mount the VMtools iso on the VM. To install the tools you’ll need to navigate to the drive where the iso has been mounted. In the example below it is the D: drive.

Installing the tools is then simply a case of silently executing the installer. This will automatically reboot the server afterwards.

[text]msiexec /i "VMware Tools.msi" /passive[/text]


That’s it, you’re done.


Windows VM MAC Address Change

Sometimes moving a server into a Virtual Infrastructure is not straightforward. Consider a server that has software on it that is licensed based on a MAC address for instance. MAC addresses start with a vendor ID and the rest of the address is made up with hexadecimal numbers to make each MAC address unique. VMware’s vendor ID is 00:50:56, all VM NICs will have MAC addresses starting with it.

So, suppose you have a piece of software installed on a Windows server that is licensed to a MAC address of 00:0B:CD:6D:17:D9. How do you P2V that server and still have the software work? Continue Reading


HA Agent on ESX-HOST in cluster CLUSTER-NAME has an error

If I wanted to, this could be a very big post all about configuring HA correctly. But I don’t want to reinvent the wheel. Instead I just want to share my experiences with this error:

HA Agent on ESX-HOST in cluster CLUSTER-NAME has an error

Odds are that you will eventually see this one pop up in vCenter for one of your ESX 3.x hosts. If you’re not sure what it means, well the translation basically is that the host displaying the error could fail and your VMs running on it probably won’t get started up automatically on another host. Essentially HA is broken on the host. Continue Reading