Automation

I’m not a developer. I’m an infrastructure guy. All I ever needed was to write some scripts. Therefore, I never needed more than DOS batches, BASH/ CSH/ KSH, Visual Basic Script and nowadays PowerShell. So why should I learn another programming language?

One to rule them all?

I don’t think that there is a single programming language that is perfect for all use cases. The spread and acceptance of a language shows a positive correlation with the number of available frameworks, tools and libraries. That’s why I love the Microsoft PowerShell. Nearly all vendors offer a PowerShell module for their products (think about VMware PowerCLI, Rubrik, Veeam, DataCore and much more). The downside: The PowerShell code has to run on a Windows box. I think the time of writing DOS batches is over. UNIX shell scripts are still awesome, but focused on UNIX.

Careful preparation is a key element to success. If you restart a storage controller, or even the whole storage, you should be very sure that all ESXi hosts have enough paths to every datstore. Sure, you can use the VMware vSphere C# client or the Web Client to check every host and every datastore. But if you have a large cluster with a dozen datastores and some Raw Device Mappings (RDMs), this can take a looooong time. Checking the path state of each LUN is a task, which can be perfectly automated. Get a list of all hosts, loop through every host and every LUN, output a list of all hosts with all LUNs and all paths for each LUN. Sounds easy, right?

Automation is essential to reduce friction and to streamline operational processes. It’s indispensable when it comes to the automation of manual, error-prone and frequently repeated tasks in a cloud or enterprise environment. Automation is the key to IT industrialization. Azure Automation is used to automate operational processes withing Microsoft Azure.

Automation account

The very first thing you have to create is an Automation account. You can have multiple Automation accounts per subscription. An Automation account allows you so separate automation resources from other Automation accounts. Automation resources are runbooks and assets (credentials, certificates, connection strings, variables, scheudles etc.). So each Automation account has its own set of runbooks and assets. This is perfect to separate production from development. An Automation account is associated with an Azure region, but the Automation account can manage Azure services in all regions.

Before you can manage Azure services with Azure Automation, you need to authenticate the Automation account against a subscription. This authentication process is part of each runbook. There are two different ways to authenticate against an Azure subscription:

• Active Directory user
• Certificate

If you want to use an Active Directory account, you have to create a credential asset in the Automation account and provide username and password for that Active Directory account. You can retrieve the credentials using the Get-AzureAutomationCredential cmdlet. This cmdlet returns a System.Management.Automation.PSCredential object, which can be used with Add-AzureAccount to connect to a subscription. If you want to use a certificate, you need four assets in the Automation account: A certificate and variables with the certificate name, the subscription ID and the subscription name. The values of these assets can be retrieved with Get-AutomationVariable and Get-AutomationCertificate.

To be honest: I’m lazy and I have a wife and two kids. Therefore I have to minimize the costs of my lab. I have a physical lab at the office and some VMs running on Microsoft Azure. Azure is nice, because I only have to pay what I really use. And because I’m only paying the actual use, I start the VMs only when I need them. Inspired by this very handy Azure VM wakeup & shutdown script, I decided to write my own script (yes, I invented a wheel again…). Very simple, nothing fancy. Feel free to use and modify the script according to your needs.

DataCore describes in their Host Configuration Guide for VMware ESXi some settings that must be adjusted before storage from DataCore SANsymphony-V storage servers will be assigned to the ESXi hosts. Today, for ESXi 5.x and 6.0, you have to add a custom rule and adjust the advanced setting DiskMaxIOSize. For ESX(i) 4 more parameters had to be adjusted. But I will focus on  ESXi 5.x and 6.0. You need to adjust these settings for each host that should get storage mapped from a DataCore storage server. If you have more then one host, you may have the wish to automate the necessary steps. The check the current value of DiskMaxIOSize, you can use this lines of PowerCLI code.

Disk space is rare. I only have about 1 TB of SSD storage in my lab and I don’t like to waste too much of it. My hosts use NFS to connect to my Synology NAS, and even if I use the VAAI-NAS plugin, I use thin-provisioned disks only. Thin-provisioned disks tend to grow over time. If you copy a 1 GB file into a VM and you delete this file immediately, you will find that the VMDK is increased by 1 GB. This is caused by the guest filesystem. It marks the blocks of deleted files as free, even if it only deletes metadata and not the data itself. Later, the data is overwritten with new data, since the blocks are marked as free and the new data is written in there. VMware ESXi doesn’t know that the guest has marked blocks as free. So ESXi can’t shrink the thin-provisioned VMDK.

It’s a common practice to replace self-signed certificates, that are used in several VMware products, with CA signed certificates. I did this in my lab for my vCenter Server Appliance and my VMware Update Manager. While I was working with vRealize Orchestrator I noticed, that it is also using self-signed certificates (what else?). For completeness, I decided to replace the self-signed certificates with CA signed.

My lab environment

1. VMware vSphere 5.5 environment running a vCenter Server appliance (already using CA signed certificates)
2. vRealize Orchestrator Appliance 5.5.2 (not version 5.5.2.1,  because I had problems with this release)
3. Microsoft Windows CA running on a Windows 2012 R2 Standard server

You don’t need a Microsoft Windows CA. You can use any other CA. There is no need to use a special vendor. I use a windows-based CA in my lab, so the screenshots reflect this fact. The way how certificates are replaced differs between vRealize Orchestrator Appliance and the windows-based standalone or vCenter Server embedded version. If you use the in the vCenter Server embedded or Standalone Orchestrator check Derek Seamans VMware vSphere 5.5 SSL Toolkit. I used the Orchestrator appliance.

Today I stumbled over a nice workaround. While installing a CentOS 6 VM, I needed to install the VMware Tools. I don’t know why, but I got an error message, regarding a non accessible VMware Tools ISO.

I remembered a blog post I read a few months ago, about a VMware online repository, from which VMware tools can be installed. You can download the repository information here. The RPM for RHEL can also be used for CentOS. Simply download and install the RPM:

I use Microsofts Deployment Toolkit (MDT) in my lab to deploy Windows VMs with Windows Server 2008 and Windows Server 2012. I described the installation and configuration of MDT in a small blog post series. Take a look into the intro post, if you’re a new to MDT. But the OS installation isn’t the time consuming part of a deployment: It’s the installation of patches. Because of this, I decided to automate the patch installation and make it part of the OS installation.