In my last post I shared the application axis concept. This graphic on the left highlights application distance; mobile devices driving I/O into an IT infrastructure will experience latencies as the data moves southward down the application axis.
In a recent discussion with a manufacturing customer, I asked them if this visualization resonated with their IT infrastructure, and the answer was "yes" (with a twist).
This particular customer was driving sensor data into his IT infrastructure via RFID tagging. Recently manufactured products would travel down a conveyor belt and pass by an RFID scanner. The scanner would send this data over a network and into a data center. Once the RFID value was safely stored and acknowledged by the disk subsystem, it triggered a series of downstream business workflow events that could happen.
Towards the end of the quarter the customer began to experience a lag in the downstream business workflow. The storage infrastructure was not experiencing any failures or faults of any kind. A further analysis of the problem, however, discovered that an increase of RFID events (e.g. the conveyor belt moves faster) filled the cache of the storage system.
Once the customer discovered this phenomena they added a tier of enterprise flash drives to their system, and effectively handled the spike in activity.
This use case is essentially an Internet of Things (IoT) reality. Machines will continue to send more and more sensor data (e.g. RFID tags) into the system. Eventually the flash tier will be overwhelmed as well, and this particular customer is left wondering "how much flash do I need", and/or "how can I do IoT capacity planning"?
It's worthwhile to explore options for IoT architectures in the context of the application stack pictured above. Below are a set of solutions that we have built (or are building) at EMC.
Instead of trying to capacity plan the mix of flash and spindles, some customers are simply moving to an all-flash solution. The up-front CAPEX can eventually be offset by factors such as floor tile usage and electricity cost. For IoT, as soon as you are waiting on a spindle, you're probably moving too slow for the workload.
Some storage products (e.g. the VMAX) are turning their vertically tiered FAST algorithms on their side and horizontally tiering data to a Flash array. This has the benefit of extending advanced data services (e.g. Snap, Remote Mirror) to a new technology (all-flash arrays) that may not necessarily have those services.
Server-based Flash Storage
Distributed Server Memory
Technologies such as Pivotal Gemfire can absorb sensor data into a massively connected, in-memory data grid that leverages server memory and then tiers to underlying storage systems. I mentioned the applicability of this technology to sensor data in a previous post.
New Storage Deployment Paradigms
At EMC World this past May, Rich Napolitano's team demonstrated the complete encapsulation of the VNX storage software into a virtual machine and then deployed that storage system within a cloud provider environment. In the context of the diagram above, they moved the storage system into the carrier/cloud layer (right next to the mobile application). This is another strategy for solving the performance problem.
Apps to Storage
One final way to look for performance increases is to move the applications closer to the storage. For example, at EMC World both the VMAX team and the VNX team demonstrated the prototype capability to download and run applications inside their respective systems. Brian Gallagher discussed the ability to run virtualized VPLEX inside VMAX, and Rich Napolitano mentioned applications like virus scan and replication for VNX. Both of these solutions could not only solve the performance issue mentioned in this article but also address management simplicity by potentially removing the need for separate appliances on which to run these applications.
All of these approaches are under investigation at EMC. In future posts I will look at this same diagram and discuss new approaches forming in other fields such as security and backup/recovery.