Conflicting information: Setting iops option for VMW_PSP_RR for HP 3PAR StoreServ on ESXi

Yesterday I received the following tweet:

@vMario156 Yes, this was stated in an older version of the best practice guide. 1 IO is currently best practice, not only for 3PAR.
— Patrick Terlisten (@PTerlisten) June 5, 2014

Later Craig Kilborn joined the conversation and I decided to clarify this 100 or 1 IOPS myth the next morning.

@Craig_Kilborn @vMario156 I check this tomorrow at the office. I don't know why this was changed, but it has changed from 100 to 1 IO.
— Patrick Terlisten (@PTerlisten) June 5, 2014

In order to give you some context: I wrote a blog post about adding a custom SATP claimrule for HP 3PAR StoreServ storage on ESXi. In this blog post I pointed out, that the claim rule is usually used to change the default behaviour for switching the path for active IO. For the VMW_PSP_RR this is 1000 IOPS, which means, that after 1000 IOPS for a specific device, the path for the active IO to this device ist changed to the next active and optimized IO path. I recommend to read this blog post from Duncan Epping for more information.

I checked two documents which are important and recommened to read, if you want to implement a HP 3PAR StoreServ with VMware ESXi:

HP 3PAR VMware ESX Implementation Guide (published in March 2014)
HP 3PAR StoreServ Storage and VMware vSphere 5 best practices (Rev. 5 published in April 2014)

The implementation guide provides you the necessary information for the implementation. The best practice whitepaper will cater to its name: it is a best practice whitepaper. So the first one shows you how to do the right things, the second document shows you how to do things right.

Lets have a look into the implementation guide on page 46. There are several commands to add a custom claimrule, depending on the selected host persona.

On page 43 HP writes:

As part of PSP Round-Robin configuration, the value of IOPS can be specified. IOPS is the number of IO operations scheduled for each path during path changes within the Round-Robin path selection scheme. The default IOPS value is 1000. HP recommends IOPS=100 as an initial value and starting point for further optimization of IO throughput with PSP Round-Robin. With the exception of ESX/ESXi 4.0 versions, it is preferable to set the IOPS value within a SATP custom rule.

Okay. Lets take a look into the best practice guide. On page 8 HP describes how to add a custom claimrule. This time with IOPS=1.

On the bottom of page 7 HP writes:

Managing a Round Robin I/O path policy scheme through the vSphere Web Client on a per datastore basis does not allow setting important Round Robin policy details that can be specified when using the command line on the ESXi host. To achieve better load balancing across paths, the –iops option may be issued on the command line to specify that the path should be switched after performing the specified number of I/Os on the current path. By default, the –iops option is set to 1000. The recommended setting for HP 3PAR Storage is 1, and this setting may be changed as needed to suit the demands of various workloads.

In a version of March 2013 of the HP 3PAR StoreServ Storage and VMware vSphere 5 best practices guide, this statement isn’t included! I assume that this happened due to the release of 3PAR OS 3.1.2, which was announced in December 2012. With 3PAR OS 3.1.2 the host persona 11 was released and it was recommended to use this with VMware ESXi. With host persona 11 the hosts use VMW_SATP_ALUA instead of VMW_SATP_DEFAULT_AA.

Depending on the document and the age of the document, different IOPS values are stated. IOPS=100 is stated in the implementation guide and should be used as a start value for further optimizations. The best practice guide clearly recommended IOPS=1. I recommend to do some tests with your customers workload and then choose whether 100 or 1 IOPS.