Information Playground

My Trip to Scotland, Dec 1-3

Sometime during this past spring I received an email from a colleague of mine suggesting that I write a paper for an upcoming conference on Digital Curation. I was fairly intrigued because as a software engineer building products in the trenches it sounded like it would be a refreshing change from the day to day. So I looked into it.

I found that the fourth annual Digital Curation Conference would be held in Edinburgh, Scotland on December 1-3, 2008.

Edinburgh is a really, really cool place to go. So I decided to give it a try.

What happened next was a good news, bad news situation.

Captivating

Today's Captiva "intelligent capture" announce allows customers to put the full-court press on any document that tries to make its way into the enterprise. The announce describes even more ways to classify/identify document types using new services (e.g. .NET scripting, SOA Capture as a Service), as well as the traditional goodies (OCR/ICR, bar codes, check boxes, etc.)

What types of customers benefit from these features? Captiva supports a wide variety of industries, including insurance (claims), financial services (loan documents), healthcare (patient records), and government (tax documents).

There are many reasons for a customer to choose Captiva, but I've got my own personal favorite.

Captiva integrates with Documentum.

And Documentum integrates with Centera.

Atmos Metadata

In my last post I wrote about the ingest of data into Atmos, how it maps to a policy to determine locations and number of copies, and how it places the data based on a filled-in policy tree (an LSO). My example was based on the fact that a specific user (e.g. "joe") was storing the data.

Placement and movement policies can also be based off of user-supplied metadata tags.

TSA-2 asked where this type of metadata comes from <that would be Chris The Storage Architect, not the Barry the Anarchist ;>)>

The metadata will come from applications that integrate with the Atmos API and have designed a metadata tagging strategy for global policies.

The Atmos SOAP API serves as a good starting point for discussion.

Atmos Policy: Under the Hood

In my last post I described how "global policies" are truly the Atmos special sauce.

These policies can control the initial placement of data, the migration of data, the compression and dedup of data, and so on.

How does Atmos implement these types of global policies? Answering this question requires a peek under the hood of how the Atmos software is architected. In this post I'll cover one of the more important data structures in Atmos, as well as a subset of the Atmos software components.

The data structure describing an Atmos Policy is called an "LSO".

Atmos: Cloud Optimized Storage

With the announcement of Atmos today, EMC has also introduced a new class of storage: Cloud Optimized Storage.

This means there's a new acronym in town: COS.

For those of you already overwhelmed by the storage industry's use of acronyms, sorry 'bout that!

For those of you trying to understand what COS is, allow me to offer a definition. Think of COS in terms of the evolution of three storage system acronyms over the last ten years.

SAN ->  NAS ->  CAS -> COS.