VM shows alarm - but no alarm triggered

Today I observed a strange behaviour of several VMs at a customer. Several VMs in a cluster showed an alarm, but neither on the alarm tab of the VM, nor the alarm section at the bottom of the C# client showed an error.

The customer still uses vSphere 5.0. An upgrade to 5.5 is on the roadmap. The symptoms:

• not all VMs in the cluster were affected
• all, except one VM, were running on one specific host
• the alarm on a VM disappeared after a vMotion
• no trigger for the alarm could be found
• vSphere HA status “protected”

The similar behaviour could be observed, if a VM is moved to another cluster using VMware vMotion technology. In this case, a VM isn’t protected by vSphere HA after it’s moved to another HA protected cluster. This is described in KB2012682. I ran a small PowerCLI script, which I found in this VMTN thread.

$vm = Get-VM -Name NAMEOFTHEVM | Get-View
 
foreach($state in $vm.DeclaredAlarmState){
    $alarm = Get-View $state.Alarm
    Write-Host "name:"
    Write-Host $alarm.info.name
    Write-Host "overall status:"
    Write-Host $state.OverallStatus
}

But I couldn’t find any hint pointing me to the root cause. At this point I was able to rule out several things:

• Problem with the VM
• Issue with HA/ FT
• Problem is host related
• Resource usage of a VM

At the end the workaround was easy: Restart of the management agents. After the restart, the alarm icons disappeared. Please note, that the customer was using VMware vSphere vCenter Server 5.0 U1a (latest release is 5.0 U3) and ESXI 5.0 U3 (latest release is 5.0 Patch 7). Maybe it’s a bug and it’s fixed in a newer release.