Trusted SSL certificates in OSX 10.15+ and iOS 13+

I deployed a new vSphere VCSA for my homelab in December 2019 (last month). By default these come with a self-signed SSL certificate that’s valid for 10 years. Of course I typically replace these with a signed certificate but it’s not always the first thing that I do.

What I found this time however is that on my Mac neither Chrome or Brave would allow me to reach the web UI. Only Firefox would. I expect security warnings for self-signed (and hence untrusted) certificates. On the former two browsers though the message suggests that the certificate is invalid in some other way:

What’s actually happening is that as of MacOS 10.15 and iOS 13 SSL certificates have to meet certain criteria to be deemed to be valid. These are documented here: https://support.apple.com/en-us/HT210176.

In the case of the vCenter VCSA, the duration (10 years) is over 825 days. Hence no dice. It would be better if Chrome was clearer about that.


vRA 7 / vRO 7 REST error (java.security.cert.CertificateException)

Whilst I was with a customer recently, I hit an SSL related issue whilst trying to put together a vRO workflow to orchestrate the creation of a load-balancer configuration on a Citrix Netscaler VPX.

Adding the REST host(s) to vRO was accomplished without any issue, but when I came to use them my workflow failed with the following error:

As this vRO instance was running on a vRA appliance, my first port of call was starting the vRO Control Center service and make sure that the REST host certificates had indeed been imported in to vRO and were trusted. They were.

Looking at the certificates themselves (as I had blindly accepted them up until that point) I noticed that they were self-signed and the cause of the error became clearer. Some software solutions generate fairly weak SSL certificates by default to maintain backwards compatibility with other legacy solutions, and in some instances because it’s easier and cheaper perhaps. Some of the algorithms used to generate and / or sign these certificates are weak or have know vulnerabilities and are increasingly untrusted by default. You only have to fire up an up-to-date version of the Chrome browser and point it at something using such a certificate to see that happen – Chrome says no!

The java implementation on a vRA 7 appliance is no different, there are certain untrusted algorithms. Should you need to, you can enable them again.

  1. SSH to the vRA appliance using either the root account or an account capable of using sudo
  2. Navigate to the directory /usr/java/jre-vmware/lib/security
  3. Copy java.security to java.security.old (always take a backup!)
  4. Use vim to edit the java.security file
  5. Comment out the line jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024
  6. Comment out the line jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768
  7. Save the file
  8. Restart the vRO service (service vco-server restart)

I should stress that this is very much a short-term solution. The preferred solution would be to replace the SSL certificates on the problematic solution and re-enable the settings adjusted above. If that isn’t practical, use trail and error to work out which specific algorithm is causing the issue and remove only that from the setting.


Howto: Creating a CA template for VMware services

Having setup my lab’s PKI infrastructure previously, one of the next steps I needed to complete was to create a template for certificates for VMware’s products to use as they require certain properties to be present in the certificates used.

There is a KB article that covers this but I wanted to run through it and use some of the specifics for my lab.

Template for VMware SSL Certificates

This template will provide certificates for ESXi hosts, vCenter, vRA, vRO etc. To create it, we first need the Certificate Templates Console. This can be opened by running certtmpl.msc.

Per the KB article, I duplicated the “Web Server” template as a starting point. My first task was to give the template a new name and set the validity to 4 years:


On the Extensions tab, although it’s possibly not required for vSphere 6 (it is for earlier versions of vSphere), I added “Client Authentication” under the Application Policies option.


Again, it may not be universally required but I’ve added the “Signature is proof of origin” option under Key Usage (also on the Extensions tab.


Depending on the use case required, it might be useful to be able to export a certificate’s private key. I haven’t worked on View for some years but this option came in handy then. It’s configured under the Request Handling tab.


On the Subject Name tab, ensure that “Supply in the request” is checked.


That’s it. Just hit OK to save it.

Template for VMware VMCA

If you want to set up the VMCA as a subordinate certificate authority on a vSphere 6 Platform Services Controller, a slightly different type of certificate is required. I don’t think that I deviated from the KB article here except with the validity period.



“Publishing” the certificate templates

This is a fairly straightforward process accomplished using the Certification Authority Manager. Templates are added one at a time by right clicking on “Certificate Templates” and selecting New > Certificate Template to Issue.


Once published, the templates are available via the CA’s web interface for new requests.



Howto: Configuring a homelab online subordinate CA

A quick recap of where I got to. I have an offline Root CA (well, it’s still online because I’ll need it in a minute) and I’ve created a website on my online subordinate CA server to host the Root CA certificate and CRL files.

The purpose of the subordinate CA is to handle certificate signing and repudiation for all services in my infrastructure that require them. It will be granted the authority to do so by the Root CA. So this post covers the remaining steps of the process, which are:

  • Installing and configuring the subordinate CA
  • Signing the subordinate CA’s certificate using the Root CA
  • Delegating control of the subordinate CA to someone other than Domain Admins

Some elements of this process are very similar to the process of setting up the Root CA in the first place but there are some differences.

Installing the subordinate CA

As with the Root CA, I’ve done a lot of this via the command line. Other methods are available.

Using a powershell prompt on the subordinate CA, create and edit a CAPolicy.inf file using this command:

Insert the following text, taking care to make sure that the URL on line 8 is specific to your environment. It shouldn’t break things if it’s wrong but it’s just the location of the Certificate Practice Statement file.

There are fewer options used than for the Root CA.

To install the CA using these settings (make sure that the file is named “CAPolicy.inf” exactly, no extra file extensions), the following commands were used:

If you’re following this, change the CA’s common name (O11N-IssuingCA-01 in my case) to suit your environment. What you’ll see at the end of it though is a big yellow warning that the installation is incomplete. This is because the CA’s certificate now needs signing and issuing by the Root CA.

Requesting and signing the subordinate CA certificate

The warning text mentioned above tells you where the certificate request has been saved to. In my case, it was saved as C:\ca-01.o11n.lab_O11N-IssuingCA-01.req. This file must be copied to the RootCA to sign and issue the relevant certificate for the subordinate CA.

On the Root CA, the certificate request can be submitted using the command:

If it works, you’ll see that the request is now pending. Take note of the request number. (It was “2” in my case.)


To sign and issue the certificate, you need the Certification Authority tool. Find the request and Issue it.


The issued certificate can easily be exported using the following command:

The certificate then needs copying back to a temporary location on the subordinate CA. Once there, it can be installed and the certificate authority service started:

As a final step, copy the contents of C:\Windows\System32\certsrv\CertEnroll to C:\pki. It now looks like this:


Configuring the subordinate CA CRLs

As with the Root CA, I wanted to set up the distribution points for the CRLs to suit my environment.

The first step is to remove the existing points:

Next, I created three new CRL distribution points.

  • The first point is to a local file on the CA server:

  • The second point is to a web URL that is hosted locally on the subordinate CA:

  • The third and final distribution point is to the “pki” virtual directory:


Configuring the subordinate CA AIA settings

As with the Root CA, configuring the AIA settings is necessary.

As before, we’re going to clear out the original settings first:

Next, the “certutil” command is used to config a few default settings:

Finally, we restart the CA service and re-publish the CRLs.

That’s the online subordinate CA configured.

Configuring CA permissions

In a lab environment, particularly mine, it’ll be used be very few people. All the same, it’s a good practise to configure proper delegated security controls. After all, I don’t want to give every account Domain Admin access just so that they can request and issue certificates. To that end, I created a couple of Active Directory security groups that I used to govern access to the CA.


To use these groups, all that is required is to open the Certification Authority manager and open the properties for the CA. On the Security tab, I just added the groups and assigned them relevant permissions.


Next steps

Now that my lab PKI infrastructure is setup, one of the key tasks is to start using it! This means generating signed and trusted certificates for vSphere and vRA for example. For that, I might need to setup some certificate templates. If the web enrollment is to be used to do this, the Subordinate CA’s IIS website needs configuring with an SSL certificate. The instructions in this Microsoft article describes how that is done.

It’s important not to forget though that there are some remedial tasks to complete. Both the Root CA and the Subordinate CA could do with being hardened. The Root CA in particular needs to be kept safe and taken offline.


Howto: Publishing offline Root CA certs and CRLs

Previously, I setup an offline Root CA in my homelab with the intention emulating a PKI setup that many enterprises seem to run.

The second stage of this process is publishing the Root CA certificate and CRL in a place that they can be accessed when the Root CA is offline. If you recall, I configured the Root CA to publish its CRL etc to a location on pki.o11n.lab. I now need to create that.

The Server

Rather than run my lab’s online CA on a domain controller, which might be tempting but causes other issues, I have a domain joined server setup that will eventually become my online subordinate CA.

It’s a vanilla Windows 2012 R2 server as before and a domain member.


The VM is called “ca-01”, but I need to have pki.o11n.lab pointed to it too. That is accomplished fairly easily with a quick DNS CNAME record:


Root CA Certificate and CRL

If this wasn’t a lab environment, I’d probably add a disk to my online CA VM to host its data. Instead, I’ve created a folder called C:\pki and copied over the Root CA certificate and CRL files.


I’ve also shared this folder and granted permissions as follows:

  • Domain Admins (Full Control)
  • SYSTEM (Full Control)
  • Cert Publishers (Change)


Publish to Active Directory

By publishing the RootCA certificate in to Active Directory, any domain joined Windows servers will automatically trust it. Standalone servers (and VMware appliances) will, of course, need to be instructed manually to trust the RootCA but I’ll take care of that as and when the need arises.

To publish the certificate to AD and trust it locally, the following commands were used:


CPS, CRL and Root CA Certificate Distribution

Before configuring HTTP distribution of the various files, I need to create the cps.txt file that was referenced in the previous article.

For me, that was simply accomplished by creating C:\pki\cps.txt and populating it with some sample text.

Next up, I installed IIS to serve up the files in C:\pki. That was completed by executing the following command on the online CA server:

Once that is completed, I created a new Virtual Directory for the PKI files:

It’s probably easier to do the next bit in the IIS Manager. The permissions for the Virtual Directory need configuring as follows:

  • Add “Cert Publishers” – allow Modify permission
  • Add “IIS AppPool\DefaultAppPool” – allow default permissions (read etc)

Note that with the second addition, it’s a server local account and can’t be found by browsing the local accounts. Sorry, you have to type it in verbatim.


The final step is to allow double escaping in the request filtering settings. With the “pki” virtual directory selected, double-click “Request Filtering” in the centre pane. Then select Edit Feature Settings. Tick the checkbox for “Allow double escaping”.


There is now a website available that hosts the cps.txt file (see below), the Root CA certificate and CRL.


The final stage is configuring the online subordinate CA. I’ll cover that in the next post.


Howto: Configuring a homelab offline Root CA

Self-signed SSL certificates are all well and good but they’re not meant to be for the real world. The trust issues they cause can be a headache on customer projects and anything that’s going in to production shouldn’t be using them.

For that reason, I thought it’d be better to change my homelab so that it uses a slightly more realistic PKI setup. The first phase of that is creating an offline Root CA as it’s something that a good number of customers use too.

Step 1: DNS

From a DNS perspective, my homelab is split up so that anything physical and fundamental to the lab (e.g. storage / NAS, physical hosts, switches etc) lives in its own DNS domain (home.lab). Everything else from vCenter and AD downwards is in one or more other DNS domains and on separate VLANs etc. Now even though I’m planning to setup my Root CA as a VM, I’m going to keep it in the same DNS domain and the physical stuff because that seems to be a better fit for it.

So before doing anything else, I needed to create DNS entries for my Root CA on my NAS (which doubles as DNS master for the home.lab domain).


Of course, the idea of an offline CA is that it’s not connected to a network. For the installation and setup of it though, I want to make sure that it’s patched etc.

Step 2: Deploy a VM

This sort of thing is bread and butter really but it’s just a vanilla Windows 2012 R2 VM that is NOT domain joined but I have set up its network identity (name, DNS name and IP details) and connected it to a portgroup that my other lab infrastructure exists on.

201511317_121103-CapturFilesStep 3: Add the CA Role

Rather than including blow-by-blow screenshots from server manager, my installation was performed through the command line as much as possible. The first step is creating a configuration file.

Using a powershell prompt, create and edit a CAPolicy.inf file using this command:

Insert the following text, taking care to make sure that the URL on line 8 is specific to your environment. It shouldn’t break things if it’s wrong but it’s just the location of the Certificate Practice Statement file.

When the CA is installed using the .inf file above, its certificate will be valid for 20 years and the validity of the CRL will be 26 weeks.

To install the CA using these settings (make sure that the file is named “CAPolicy.inf” exactly, no extra file extensions), the following commands were used:

If you’re following this, change the CA’s common name (O11NRootCA in my case) to suit your environment.

Step 4: Configure CRLs

By default, this CA will publish its Certificate Revocation List (CRL) in locations that aren’t useful as the server will be powered off for most of its life. Therefore the CRL distribution points needed reconfiguring.

The first step is to remove the existing points:

Next, I created two new CRL distribution points. For both, “%3%8” will be substituted by the CA with the CA’s name and the CRL Name Suffix values:

  • The first point is to a local file on the CA server:

  • The second point is to a web URL that I’ll host on another server later:

The second distribution point will be added to the CA’s certificate so that other servers know where to get the CRL from.

Step 5: Configure AIA

Authority Information Access locations are URLs that are added to a certificate in its authority information access extension. These URLs can be used by an application or service to retrieve the issuing CA certificate.

As before, we’re going to clear out the original settings first:

Next, the “certutil” command is used to config a few default settings:

Finally, we restart the CA service and re-publish the CRLs.

That’s the offline Root CA configured. If you look at the C:\Windows\System32\CertSrv\CertEnroll folder, you’ll see the CRL and CA certificate files. It’s a good idea to copy these off the server as they’ll be needed to configure an online subordinate CA. I’ll cover that off in the next post.