Home > Disk, SSD, tuning > Understanding Storage Performance

Understanding Storage Performance

Comparing storage performance is a bit more difficult than meets the eye.  This comes up quite a bit as I frequently address the differences between RAM and Flash-based SSDs.  This particularly comes up when comparisons are made between TMS’s Flagship Flash system, the RamSan-630 and the Flagship RAM system, the RamSan-440.  The Flash system has higher IOPS but the RAM system has lower latency.

There are three independent metrics of storage performance: response time, IOPS, and bandwidth.  Understanding the relationships between these metrics is the key to understanding storage performance.

Bandwidth is really just a limitation of the design or standards that are used to connect storage.  It is the maximum number of bytes that can be moved in a specific time period; response time overhead or concurrency do not play a factor.  IOPS are nothing more than the number of I/O transactions that can be performed in a single second.  Determining the maximum theoretical IOPS for a given transfer size is as simple as dividing the maximum bandwidth by the transfer size.  For a storage system with a single one Gbps iSCSI connection (~100 MB/s bandwidth) and a workload of 64 KB transfers, then the maximum IOPS will be ~1,500.  If the transfer size is a single sector (512 bytes), then the maximum IOPS will be ~200,000 – a notable difference.  At this upper limit, bandwidth will more than likely not be the performance limiter.

There is another relationship that ties together response time and concurrency – Little’s law.  Little’s law governs the concurrency of a system needed to achieve a desired amount of throughput.  For storage, Little’s Law is: (Outstanding I/Os) ÷ (response time) = IOPS.  I consider this the most important formula in storage performance.  If you boil this down, ultimately the limitation of IOPS performance is the ability of a system to handle Outstanding I/Os concurrently.  Once that limit is reached, the I/Os get clogged up and the response time increases rapidly.  This is the reason a common tactic to increase storage performance has been to simply add disks – each additional disk increases the concurrent I/O capabilities.

Interestingly, the IOPS performance isn’t limited by the response time.   Lower response times merely allow a given level of IOPS to be achieved at lower levels of concurrency.  There are practical limits on the level of concurrency that can be achieved by the interfaces to the storage (e.g. the execution throttle setting in an HBA), and many applications have fairly low levels of concurrent I/O, but the response time by itself does not limit the IOPS.  This is why even though Flash media has a higher response time than RAM, Flash systems that handle a high level of concurrent I/O can achieve as good as or better IOPS performance.

Categories: Disk, SSD, tuning
  1. Bob Norman
    June 24, 2011 at 7:11 pm

    To me a drive IOPS is simply 1/Tlatency_ave + TSeeka_ave
    These times are an order of magnitude more than Command overhead and Data Transfer Speeds. Much of these second order delays can be hidden by overlapping the transfers into or out of DRAM while the seeks are in progress.
    The command parsing and scatter gather usage in the RAID controller has a big impact. A smart interface like SCSI or SAS can give lower performance, as the two smart ends fight, but a simple interface like SATA will deliver as asked, giving better performance
    For SSD the mapping algorithm is surprisingly a bigger part of the delays than one would expect.
    I enjoy your writing very much!

  2. Dhanshree
    October 5, 2012 at 8:23 pm

    This was really good info

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: