Automated NIC names in Linux on VMware vSphere

A colleague of mine was working with a customer recently on some changes to their automated VM provisioning process (they’re not vRA customers… yet). He got stuck trying to get around a particular challenge with the automatic naming of network interfaces in certain Linux distributions.

The customer in question is using vRealize Orchestrator (vRO) to create (not clone) their Virtual Machines from a JSON structure that is supplied by an external system. In that structure there are definitions for the hardware, OS network identity (name, IP etc) and OS installation sources (ISO file for installation and floppy image for a ks.cfg (KickStart) file).

Once the JSON object is provided to the vRO workflow, the VM is created, booted and automatically starts to install and configure itself.

Customer’s Simple VM

Simple VMs have a number of disks defined (for root, opt, var, swap etc partitions) that are attached to a single ParaVirtual SCSI adapter. The VM is also equipped with a single VMXNET3 network adapter.

In this configuration, there is no problem. The installation of the OS runs through to completion and the VM is handed off to Puppet and eventually goes in to service.

Customer’s Complex VM

For the provision of Linux-based Oracle servers however, the customer wanted to be able to specify not only extra disks and partitions, but extra SCSI controllers too. Continue Reading

0

Extensibility course in vRA / vRO (Feb 2018)

A new vRA / vRO Extensibility course is scheduled to be held in the UK in February 2018. It will provide detailed information on how to leverage the power of vRealize Automation and vRealize Orchestrator.

I should point out that it’s a beta course, so it may not be 100% polished and pristine. However, the course is focussed on version 7.3 of both products (the latest at the time of writing) so it will be bang up-to-date and help you learn the “art of the possible”.

Extensibility course outline and details

The course runs for a full week from Monday 19th February at VMware’s UK office in Staines. If you’re new to vRA and vRO, or you want to take your automation beyond simple virtual machine deployment then this course will likely benefit you. If you don’t manage to get on it, I’m sure there will be further opportunities throughout the year.

Extensibility Topics Covered

The topics covered by the course include:

  • The basics of vRO workflows
  • Design considerations for vRO workflows
  • Development of vRO workflows
  • Extending vRA using the Event Broker
  • Using vRO workflows with vRA’s Event Broker
  • Using vRO workflows to provide XaaS catalog items and resource actions
  • Working with REST APIs using vRO

You can register for the course via VMware’s myLearn website.

0

Automating vSphere VM disk zeroing with vRA7 and vRO

A long time ago, on a project far back in time, the team that I was part of was given a requirement to zero the disks of VMs before they were deleted by vRA / vRO (or vCAC and vCO as they were called back then). One of my colleagues on the project, Jonathan Medd, devised an approach for doing this using an “experimental” PowerCLI feature and wrote it up on his blog.

Fast forward nearly two and a half years and I’m looking at an upgrade for this platform and wondering if there’s a way to accomplish the same task in vRO rather than by breaking out to PowerCLI. Don’t get me wrong, I love PowerCLI. But fewer parts would mean that there’s less to go wrong. How to do it then…

Disk zeroing in PowerCLI

This is still listed as an experimental feature in the PowerCLI documentation for vSphere 6.5. The Set-HardDisk cmdlet has the -ZeroOut option and it would still be used exactly as Jonathan describes it in his article.

PowerCLI documentation for Set-HardDisk

PowerCLI documentation for Set-HardDisk

Disk zeroing in vRO

I’m not sure when it was added (i.e. which version), but back in 2014 we couldn’t find equivalent functionality in vRO. I did a quick search of the vCenter plugin methods in my v7.2 appliance and couldn’t see it there either. It turns out though that I was having a bad typing day. Burke Azbill pointed me to the right place (thank you):

vRO API help for zeroFillVirtualDisk_Task

vRO API help for zeroFillVirtualDisk_Task

So, “zeroFillVirtualDisk_Task” is a method called from VcVirtualDiskManager. All we need to give it is a datastore path and a VcDataCenter object and it’ll do the rest.

Getting a datastore path is relatively straight forward. Using vSphere’s Managed Object Browser (MOB), I can pick a VM object and navigate down to the config (1) and the hardware (2), get the disk devices (3) and look at their backing (4). The fileName attribute gives me the datastore path that I need.

Example VM configuration information in the vSphere MOB

Example VM configuration information in the vSphere MOB

Obtaining a VcDataCenter object, if I have the VM object already is a doddle too. There’s a vCenter plugin action that will do it for me based on the information that I have available to me in the MOB above. Taking the datastore attribute, which is a reference to a datastore object, I can pass it to the action below and get the VcDataCenter back.

Library vRO action getDatacenterForDatastore

Library vRO action getDatacenterForDatastore

Putting it together

Starting with a vCenter VM (vm in the script below) object in vRO then, zeroing all of the VM’s disks can be achieved as follows:

Deconstructing the code:

Line 1 – Gets the vCenter plugin connection for the vCenter that owns the VM called vm.
Line 2 – Gets the VCVirtualDiskManager object that the zeroFillVirtualDisk_Task method is a member of.
Lines 3 to 5 – Getting the config, hardware and devices for the VM. This could be done as one line.
Line 9 – Starts a loop to run the following code for each device.
Line 10 – Checks to see if the current device is a Virtual Disk.
Line 11 – Gets the Virtual Disk filename.
Line 12 – Gets the VcDataCenter object using the Virtual Disk datastore as an input.
Line 14 – Instructs vCenter to zero the Virtual Disk.

Considerations for vRA

There are a few considerations aside from some minor tweaks to make the code more efficient or robust before we look at adding this as a workflow subscription in vRA.

  • Firstly, on some storage devices (my home lab included), zeroing the disks causes a thin provisioned disk to expand to its full size. If the disks are large, you can expect the task to take some time and / or consume a lot of storage space.
  • Secondly, if there are snapshots running on the VM, they must be removed first.
  • Thirdly, the VM must be powered off before this will run. If the workflow subscription takes place at the right time, this shouldn’t be a problem however.
  • Finally, as I’ve already mentioned, this process could take some time to complete. The zeroFillVirtualDisk_Task method returns a vCenter task reference. Ideally that should be monitored for completion rather than just firing and forgetting. If the VM has multiple disks, that’s multiple tasks.

Adding it to the vRA Event Broker

Taking the original script above and factoring in the considerations too, it’s possible to create a fairly simple workflow that can be added as an Event Broker subscription in vRA. I’ll post it up here soon.

0

How to change the vRA 7.2 “All Services” icon using vRO

This post was inspired by Ryan Kelly’s recent post of a similar title in which he revealed how to change the “All Services” icon in the vRA 7.2 Catalog tab. Shortly after seeing that, I noticed that Ricky El-Qasem had also posted on the subject and created a small utility to accomplish the same task.

Great posts both of them. But I wanted to take a slightly different route and accomplish the same task in vRO which, as I’m sure you know, comes bundled with vRA. I must admit that I’m not totally satisfied with the result, maybe about 95%, but I’ll explain why later.

Selecting a source image

For my initial testing, I thought I’d use the image on Ryan’s post as it clearly worked for him. However, the image in his post is JPEG formatted and contains a border. That would clearly look a bit naff so I converted it to a PNG file, removed the border and the white background.

For guidance, I’d recommend a PNG file for these icons so that if they’re not square the image background can set to be transparent, allowing the theme from vRA to show through. It just looks better.

The process

As vRO allows images to be imported and stored within it, I thought I would try this approach. I have previously used such Resource Elements to attach images to HTML emails sent by vRO (as well as to store templates for text / emails, JSON objects and XML).

 

From that point, it should simply have been a case of converting the image to Base64 text within vRO and executing a REST update to vRA to replace the image. However, the methods available for extracting data from a Resource Element convert the image to a string, which munges it a bit. The resulting Base64 string is therefore corrupt and you end up with a broken image displaying in vRA:

Doh!

I tried various ways to get around the issue but so far I haven’t been able to, hence my minor dissatisfaction. What this means is that like the method that Ryan outlined, you still have to convert the image to Base64 outside of vRO. To finish off the workflow that I wanted to create, I did just that using an online tool and stored the Base64 string in a different resource element in vRO.

In the RAW output for the image shown, only the actual Base64 text is required and can be placed in to a text file and added to vRO:

 

This changed the process in the workflow slightly. In some ways it simplified it. Let’s look at that now…

Looks fairly simple, doesn’t it? Actually it is. No messing around with tokens!

Line 2 – This returns the vCACCAFE:VCACHost connection for the default tenant in vRA (vsphere.local)
Line 4 – Creates a REST connection to the vRA Catalog Icon Service
Lines 7 & 8 – Creates a reference to the default tenant organisation in vRA
Lines 11 to 16 – Constructs a new icon object using the Base64 image data as well as the ID of the image we’re replacing
Line 19 – Execute!

The result

In the workflow, I’ve added three inputs:

  1. Select a resource element where the base64 image data is stored (using this overrides the field below).
  2. Paste in the base64 image data yourself.
  3. Set the MIME type of the image (i.e. PNG or JPG)

Running it against Ryan’s icon and the Union flag image from above, I got the change applied across all tenants in vRA as expected:

Want the workflow to try this yourself? I’ve popped it on to GitHub:

mpoore/o11n-vra-catalog-allservices

0

Some vRO IP address actions

I just wanted to take this opportunity to share a few vRO actions from my library that I’ve recently tidied up. Some started life as scriptable tasks in other workflows but it made sense to strip those bits out and put them in to discreet actions to enable better re-use.

Background

Several of these functions came from a single project. The IPAM system in use only returned an IP address for the vRA provisioned VM being worked on and a subnet mask. The gateway address had to be calculated. In another project, similar constraints existed but with the added complication of some networks having the gateway as the first address in the subnet and some having it as the last!

A couple of these functions also came in handy for some NSX automation, making sure that a new IP address was added to the correct interface of an NSX Edge Services Gateway (ESG) device.

The Actions

There are 7 actions in total. I added a couple to complete some extra functionality that was missing from the original use case requirements.

  1. areIPsInSameSubnet – Takes two IP addresses and one subnet mask as inputs and returns “true” if they’re in the same subnet or “false” if not.
  2. areIPsInSameSubnetUsingCIDR – Takes two IP addresses and one subnet CIDR value as inputs and returns “true” if they’re in the same subnet or “false” if not.
  3. convertCIDRToIPMask – Converts a subnet CIDR value (e.g. 24) to a subnet mask (e.g. 255.255.255.0).
  4. convertIPMaskToCIDR – Converts a subnet mask (e.g. 255.255.254.0) to a subnet CIDR value (e.g. 23).
  5. getIPBroadcastAddress – Takes an IP address and a subnet mask as inputs and returns the network broadcast address (e.g. 10.10.36.12 & 255.255.255.0 returns 10.10.36.255).
  6. getIPHostAddress – Takes a network / subnet address, a subnet mask and an address index as inputs and returns the specified IP address. The address index can either be a string or a number. If it’s a number, the corresponding address in the subnet is returned (e.g. 10.10.36.0 & 255.255.255.0 & 8 returns the address 10.10.36.8). If the address index is a string, it can be one the values “first”, “last”, “broadcast”. The correct address from the range is then returned (e.g. 10.10.36.0 & 255.255.255.0 & “last” returns the address 10.10.36.254).
  7. getIPNetworkAddress – Takes an IP address and a subnet mask as inputs and returns the network / subnet address (e.g. 10.10.36.12 & 255.255.255.0 returns 10.10.36.0).

Download

I’ve put all 7 actions individually in a GitHub repository along with a package that contains them all. (Note: some of the actions have dependencies on the others.) They’re free to use, so help yourself.

mpoore/o11n-utils-network-ipv4

0

Farewell FlowGrab

If you’ve been using vRO (formerly vCO) for any length of time, you might have encountered a third party source code repository service called “FlowGrab” in your travels. It was a great idea and had great potential but its owners weren’t able to maintain and develop it as they wanted to. Sadly, the service was shutdown at the end of October.

Dear FlowGrab user and supporter

We notified in spring 2016 that FlowGrab is looking for a new, good owner. Today we can state that after all the search and negotiations we have not found it.
Therefore we have to execute the scenario that we hoped to avoid – to shut down and unplug FlowGrab.com.

If you are an active user – please export your content from FlowGrab.com before 31st October as this is the last day for FlowGrab.com to be available.

If you consider to be a new and good owner for FlowGrab and continue what we started – continue development of a source code repository for VMware cloud automation portfolio, supporting VMware vRA and vRO customers with a solution that increases the productivity, visibility and compliance, please let us know. We will provide you all the information you need to make the decision to purchase FlowGrab in its entirety.

Regardless of further discussions 31st October 2016 FlowGrab.com will be unplugged.

Yours faithfully,
FlowGrab team
[email protected]

201611306_131182-capturfiles

0

vRA 7 / vRO 7 REST error (java.security.cert.CertificateException)

Whilst I was with a customer recently, I hit an SSL related issue whilst trying to put together a vRO workflow to orchestrate the creation of a load-balancer configuration on a Citrix Netscaler VPX.

Adding the REST host(s) to vRO was accomplished without any issue, but when I came to use them my workflow failed with the following error:

As this vRO instance was running on a vRA appliance, my first port of call was starting the vRO Control Center service and make sure that the REST host certificates had indeed been imported in to vRO and were trusted. They were.

Looking at the certificates themselves (as I had blindly accepted them up until that point) I noticed that they were self-signed and the cause of the error became clearer. Some software solutions generate fairly weak SSL certificates by default to maintain backwards compatibility with other legacy solutions, and in some instances because it’s easier and cheaper perhaps. Some of the algorithms used to generate and / or sign these certificates are weak or have know vulnerabilities and are increasingly untrusted by default. You only have to fire up an up-to-date version of the Chrome browser and point it at something using such a certificate to see that happen – Chrome says no!

The java implementation on a vRA 7 appliance is no different, there are certain untrusted algorithms. Should you need to, you can enable them again.

  1. SSH to the vRA appliance using either the root account or an account capable of using sudo
  2. Navigate to the directory /usr/java/jre-vmware/lib/security
  3. Copy java.security to java.security.old (always take a backup!)
  4. Use vim to edit the java.security file
  5. Comment out the line jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024
  6. Comment out the line jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768
  7. Save the file
  8. Restart the vRO service (service vco-server restart)

I should stress that this is very much a short-term solution. The preferred solution would be to replace the SSL certificates on the problematic solution and re-enable the settings adjusted above. If that isn’t practical, use trail and error to work out which specific algorithm is causing the issue and remove only that from the setting.

0

Changing vRA ExternalWfStubs Timeouts

I saw a question on Twitter this morning that I thought warranted a quick post:

201511324_111120-CapturFilesThe default timeout for the various workflow stubs is 30 minutes, but they can be changed. As always, take backup copies and be careful! The six stubs that can be changed are:

  • Expired
  • RegisterMachine
  • Disposing
  • UnprovisionMachine
  • BuildingMachine
  • MachineProvisioned

The timeout settings for External Workflow stubs are configured on the Windows server (vRA 6.x) that hosts the Manager role. The file and path required (assuming a default installation) is:

Using your favourite text editor, simply adjust the timeout value for each workflow. Most people tend to require a longer timeout for “MachineProvisioned”, often because they require longer running post-provisioning tasks to complete before handing a Virtual Machine over to the requester.

201511324_121183-CapturFiles

Save the file and make sure that the same changes are made on the standby Manager if you have a distributed installation. You’ll also need to restart the Manager service for the change to take.

One word of warning though, extending the timeout could mean that the DEM worker(s) get clogged up with running workflows. Remember that each worker can run 15 simultaneous workflows. So if you extend the timeouts, you may need more DEM Workers. Best to monitor the number of concurrent workflows for a few days if you’re unsure.

0

It’s time to retire “vCO”

Before anyone gets in a huff, I’m not suggesting that a key part of VMware’s automation, orchestration and cloud functionality be consigned to the bin. This is about the name.

It was Q3 / Q4 last year (2014) that VMware announced the changes to its product names. A whole year ago! Of course, it can take time for a change to take effect, particularly when any of the following are a factor:

  • Older projects and environments using older versions of the products that contain original branding
  • Document, diagram and presentation re-use where the older names are used
  • Investment in any other IP or collateral
  • Ingrained habits, especially if you’ve been using these products for years

But by now “vCO” should be a thing of the past. That’s not its name anymore. I won’t be removing the tag from my blog but I’ll be making an effort not to use it anymore (after this post) and I’m going to remove the search from my Twitter client too:

201511314_231164-CapturFiles