Object Cornucopia

I was introduced to the world of object-based storage in 2003 when I joined the Centera team.  After spending a large part of my career building block-based technologies like CLARiiON, Navi, and PowerPath, the world of objects seemed like a wide-open, growing field and a great opportunity to learn new things for years to come.

I was right.

As I enter my sixth year of exploring new directions for this technology it seems I'm still in the "on-ramp" phase when it comes to the opportunity for building new object-based products and features. 

That translates into some more posts on the topic and (hopefully) collaboration on new directions with other interested parties in the blogosphere.

So I'll start with my own "object" retrospective from 2008.

Centera

Centera is EMC's object-based storage system. I wrote several posts last year describing this product. My post comparing Centera to my experiences in the block storage world can be found here.

XAM

In the same way that file system APIs provide industry standard primitives for addressing individual files (fopen, fread, fwrite), the XAM API provides industry standard primitives for addressing objects (e.g. XSET.open). In fact, I like this analogy so much that I wrote my Master's thesis about it.

I'm interested in ways that XAM can be used. I'm so interested, in fact, that I asked my peers at EMC to participate in a global XAM coding challenge.  Final submissions included integrating XAM with the back of Oracle (the winner), using XAM meta-data to create Social Network Analysis graphs, and using XAM technology to create media players.

Finally, I theorized that XAM could be used as a type of "paper clip" to bind together the fields of the industry-standard Archival Information Package. An AIP is defined in the OAIS specification (which brings me to the next topic).

Digital Archiving and Curation

It seemed to me that XAM and object-based storage are great technology choices for digital archivists and curators. I decided to write a paper suggesting this choice to the researchers attending the December 2008 Digital Curation Conference in Edinburgh. The paper was accepted and presented; time will tell if XAM research will result.

I also made several visits to the JFK Library and Museum in Boston. I've documented how the Library is undertaking a rather comprehensive digital archiving project using EMC gear. The full series of articles can be found here.

Other Research

I have an interesting anecdote that illustrates the potential for object-based research. Each year EMC hosts a world-wide idea contest. Each year 30 finalists are picked. In 2007 a co-worker and I came in 3rd place out of 410 entries, while in 2008 I came in second place out of 986 entries. Both ideas involved Centera and XAM.  In fact, out of the thirty finalists in 2008, at least 15% of them used Centera and/or XAM as part of their idea. This tells me that EMC's technical elite recognizes the potential of the technology. Indeed there are at least four proof-of-concept Centera ideas being researched/pursued by EMC's CTO office.

In time many of these will be made public.  I wrote a post last year highlighting that object-based capabilities can provide the ability to form data lineage graphs (also known as provenance). Several of my EMC co-workers in our China facility have built a demonstration that we put on display for customers in May of 2008.

Atmos

I wrote a few posts during the announcement of EMC's object-based storage system (Atmos) that is used for geographic, policy-based distribution of dynamic content (as opposed to Centera's fixed-content design point).

Early Phase

There are still many angles to take and many conversations to be had about all of the topics mentioned above, including:

• the list of vendors that play in this space
• who is supporting XAM (as a vendor) and how?
• who is supporting XAM by writing applications on top of it?
• the issues and challenges of digital archiving and curation
• file systems on top of object-based technologies
• related academic research

That's a lot to blog about and I welcome anyone's thoughts on the topic.

Steve

Comments

[Chuck Hollis]

Always a great post, Steve!

I think most people overlook that -- given the fact that most applications are becoming shrink-wrapped in virtual containers -- applications themselves can be thought of as discrete objects, making this discussion even more relevant, IMHO.

Thanks for sharing!