Information Playground: CSX Technology

Last year I wrote a post about the "best acquisition" that EMC made last decade. I chose RSA, and I stated the reasons why I felt that way. Although the RSA acquisition was public, and the revenue generated by RSA was publicly known, I was basing my selection mainly upon the internal impact that I have personally seen inside of EMC.

I recently started thinking about innovation inside of EMC. Which innovation from last decade has been most impactful internally? My view of the innovative output of thousands of global employees is restricted. My exposure to new product ideas is limited to the engineering teams I worked with last decade (e.g. CLARiiON, Symmetrix, Celerra, Centera, VPLEX, PowerPath, Atmos, etc).

Given my restricted view, my choice is a technology known as CSX.

Common Software Execution Environment

EMC has a huge amount of software technology that can be described as "data path" software. This software has been the lifeblood of EMC innovation for a long, long time. Data path software allocates the buffers that hold customer data and protects this information from cradle to grave.

All data path software is characterized by similar software interfaces to underlying kernel primitives: interrupt handling, device driver APIs, scheduling, and memory allocation. For example, if the CLARiiON data path wants to invoke a lower-level device driver, it calls a kernel-level API. In the case of CLARiiON, the FLARE software invokes an embedded Windows subroutine. Symmetrix, Celerra, and Centera, for example, all have similar (but different) API calls.

One of the advanced development teams within EMC surveyed this landscape and had an idea: what if these APIs could be changed to a vanilla, common software API set?

Portability environments are nothing new. The Adaptive Communication Environment (ACE) is a great example of a portability API. But it doesn't provide support for the most basic, common primitives required by data path software (there's really nothing out there that does).

So EMC invented one. It is an API that is available to many different internal groups within EMC. New data path software can be written to invoke operating-system-neutral kernel services.

This is really, really cool. But it's only half the story.

The operating-system-neutral primitives must call into an execution environment that implements the primitives on each specific platform.

And this is where the true innovation comes into play.

Kernel or User Space

The engineers that built this API had a vision that the "execution environment" would have two unique features:

It could run on any platform
It could run in kernel or user space

These are two incredibly powerful features. Let me explain how these features can be leveraged.

Consider the compression functionality introduced inside Celerra last year. The compression software itself was written as a component using the CSX API. This resulted in a re-usable component providing operating-system-neutral compression functionality. This component was tested thoroughly in its standalone form. In order to run inside a product, however, this compression component needed to be embedded alongside a CSX execution environment.

EMC engineers created an execution environment within Celerra's operating system (DART). The compression component was embedded inside of Celerra. The integration of DART/CSX/Compression was tested thoroughly. The feature was shipped to customers.

Moving forward, the component can also be embedded inside of CLARiiON, in the Windows kernel, or in Windows user space. All that is required is a CSX execution environment that runs in either of those two places. This enables CLARiiON data path developers the choice to implement compression in either (or both) of those locations.

EMC engineers have been building, testing, and qualifying production CSX execution environments on multiple platforms, in both the kernel and user space, for the better part of the last (and current) decade.

Of Components and Services

In May I wrote from EMC World that more and more products built by EMC engineers were being created via reusable components underneath customer-targeted service interfaces.

It is much simpler these days for an engineer to take any asset from EMC's storehouse of software acquisitions, isolate the critical intellectual property into a reusable component, and begin to share it and embed it across multiple product lines (no matter what operating system runs inside that product line).

Why is it so much fun to work here? Think about the permutations of products I can build from scratch using this approach. I select the reusable components that I need, create a service interface over those components, and I'm off and running.

It's all high-quality. CSX and its execution environment are themselves components which can be tested standalone on any platform (in both user space and kernel space).

This is why I believe the CSX invention was the most powerful creation of the last decade. It has had a tangible impact on EMC's internal innovative capacity.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur