I write a lot about the internal architectures of storage products like CLARiiON and Centera, and how customers deploy and use them.
There's another EMC product getting a lot of coverage today and that's PowerPath. The PowerPath software architecture is another example of software done right.
Several years ago I was given a temporary assignment to commute into EMC's facility in Cambridge, MA. The assignment had me working with the PowerPath team. PowerPath started out as failover and multi-pathing software, and has grown to become much, much more.
Historically, when EMC acquired Data General in 1999, it also acquired DG software known as ATF (application transparent failover). ATF enabled host applications to failover against a CLARiiON configuration.
At some point in time EMC had to make a decision of how to reconcile these two different software products. PowerPath enabled Symmetrix failover, ATF enabled CLARiiON failover. In my opinion the decision to keep one or the other was a no-brainer: keep PowerPath.
Let me tell you why.
A Cool PowerPath Experiment
I first learned about the PowerPath internal software architecture while experimenting with Symmetrix SRDF. I hooked up two Symms in the lab and established a "synchronous" R1-R2 mirroring relationship. I loaded PowerPath onto a Windows server and verified that PowerPath recognized both the R1 and R2 on the different Symms.
Here was the trick I wanted to play: if a single Windows server was performing read/write operations to an R1 volume, could PowerPath "intercept" some of the read requests and shuttle them over to the R2 volume? If the R1-R2 link was synchronous, shouldn't I be able to read from either? PowerPath could already load balance multiple read requests across different paths to an R1. Could I extend that functionality to load balance read requests across Symm frames?
In order to find this out, I had to see the actual PowerPath software and understand how it worked. I was able to get my hands on the code fairly quickly. This surprised me, because the PowerPath developers didn't know me. But after analyzing their software I understood why they were so willing to let me build my experiment.
PowerPath is a plug-in software architecture. It's straightforward to insert new value into their software. Within a week or two I had easily built my experiment. I ran performance tests against my prototype and analyzed the performance gains from multi-pathing reads between Symmetrix frames.
This is why PowerPath was chosen: it has an extensible software architecture.
There's More: OS Independence
The PowerPath architecture is a "device stack" architecture. As a read or a write request enters the kernel via an O/S-specific kernel call, it is translated into an O/S-neutral request, and handed to a top-level plug-in. This plug-in adds some sort of value to the request, and then passes it to the next plug-in. Any number of plug-ins can be invoked, until the request leaves the "bottom" of the stack and gets turned back into an operating-system specific disk request.
Once the disk request is complete, the result travels back up the "plug-in" stack, and further value can be added at each layer.
The architecture also has a management interface that allowed for programming the individual plug-ins. This allowed a developer like myself to instruct my plug-in to create a mirroring relationship between the two devices (establish the relationship betewen the R1 and R2, for example).
Once written, the plug-in functionality is available on other platforms as well (e.g. Solaris, Linux, Windows).
The Commute Into Cambridge
The PowerPath project I kicked off in Cambridge was the development of a plug-in that sat above a source and target device and migrated the data live. We also worked with a team in Cork, Ireland to build a plugin that copied data from one Symmetrix frame to another, and then "cut the cord" to the old Symm, all while I/O was running. The architecture was flexible in that it considered future migration methods such as migration in the SAN (Invista) or in the storage (e.g. SRDF, MirrorView).
This initial development eventually turned into what's now known as PowerPath Migration Enabler.
Working with the team in Cork was great (especially the visit over there!). I also enjoyed the chance to work in Cambridge, and frequently took a 10 minute train ride into Boston for lunch. I actually caught a Patriot's Super Bowl victory parade (their first Super Bowl victory I believe).
The Plug-Ins Keep on Coming
The flexible PowerPath architecture keeps on producing cool products, with one of my most recent favorites being the integration with RSA for PowerPath encryption. Earlier this year I wrote about the culture and environment in which we come up with these new products. That post is here.
I haven't mentioned that in addition to EMC products, PowerPath also interoperates with HP, IBM, and Hitachi arrays. Any nuances that these arrays present can of course be isolated and developed in a specific plug-in.
And I encourage you again to read the PowerPath/VMWARE vStorage post from Chad. PowerPath really packs a punch when it comes to the variety of functionality it offers.
This year PowerPath crossed over the 600,000 mark in number of licenses sold, once again underscoring the value of a solid software architecture.
Steve
Comments