NVMe Express 1.3: Addressing the Storage Needs of the Data Revolution from Enterprise to Client

The amount of data we are generating and consuming has exploded in recent years.   Social media, applications, multimedia streaming, 24-hour connectivity has us talking about Zettabytes of data in the data center that folks want to not only store, but analyze and access rapidly.

SSDs were great to address mechanical inefficiencies of HDD.  However, the constraints of the SCSI protocol still meant that you can only address needs one at a time since there is only one queue in which to submit read/write requests.  Then came NVM Express in 2011, the industry standard for enterprise and client non-volatile memory based storage solutions over PCIe.    It eliminated that constraint with the introduction of 64K queues that can handle 64K commands in each queue to lower latency dramatically. 

The latest NVMe Express 1.3 continues to introduce new features to address the ferocious appetite for better latency and higher bandwidth in the data revolution from the enterprise to the client space. 

Some of the ones causing a healthy buzz on the enterprise level are as follows:

Directives & Streams

One of the potentially more useful features is support for streams directives.  Let’s take a library and you’ve been tasked with shelving a pile of books.   You could take that pile of books and just place it on the first free shelf you see.  However, when someone comes by and asks you for books on a topic, you’ll likely wish you had shelved them in a more systematic manner.  This would allow you to retrieve it faster and likely move less books when searching for the one that you want

Directives is a framework that allows the exchange of meta-data between the device and host.  Stream directives, like the library Dewey system, allows the host to tag operations as being related.  The controller can then use this to store related data physically close to each other; just like related books are stored in the same area of a library.  This can improve read speeds; retrieving books on a specific topic.  It can also potentially low write amplification; books being shuffled around as you retrieve data.

Virtualization

Virtualization is a key component to cloud computing.  It allows for the optimal utilization of limited physical resources.   New for version 1.3 is the notion of primary controller(s) and secondary controller(s) within a NVM subsystem.  Secondary controller(s) depend on the primary controller(s) to dynamically assign sources from the pool of flexible resources available in the SSD.  The hypervisor configures the SSD and enables each VM to use standard unmodified NVMe drivers to communicate with what they ‘feel’ is s separate and distinct entity in the SSD.

Sanitize

With the vast amount of data out in the cloud, data security is and will be a hot topic for years to come.  Motivated by new government standards, the sanitize command introduces a new method to guarantee data removal from the media, cache and controller memory buffer.  There are three modes for the Sanitize command 1) block erase: a low-level block erase which physically erases NAND blocks 2) Crypto erase that changes the media encryption key and 3) overwrite that overwrites the media with data patterns.

Next time we’ll cover new features for the client space.  Signing off for now ...

Lana