Information Playground

When Objects Collide

The conversation about collisions.

The object collision paper, published nearly four years ago, was referenced last year in a conversation between Alex McDonald and 'Zilla.

The paper was written during a swirl of FUD, press, research, and legitimate customer concern.

I remember participating in the conversations that were occuring right outside my cubicle (I sat next to the Centera CTO) regarding the probabilities of object collisions in Centera.  I remember the changes that the developers were making to the software (adding new naming schemes) as a result of the conversation.

The mathematical conclusions were quite compelling. Centera's engineering activity, however, was even more compelling.  The math argued that there was no cause for customers to worry. In spite of this, we enhanced the software.  Why?

Because the customer is always right.

XAM: From Bleeding to Cutting Edge

Centera and the Centera API have been in the middle of a storage blog smack-up.

I was surprised to read the claim that the Symantec/Centera integration had issues. First of all, they've been an integration partner for years. Secondly, their implementation is top-notch, given that their integration architect is one of the most well-respected Centera-API experts in the industry! Therefore it doesn't make sense that their developers would have a problem with the Centera API.

Symantec has confirmed that there are no data integrity issues with Centera. It was a false alarm.

At the heart of the supposed issue was the need for partners to integrate their applications with Centera's SDK. Hundreds of them have done so.

It's worth exploring the reasons why these partners decided to integrate with Centera in the first place. They chose to integrate with a technology (object-based storage) that was bleeding edge.

Issues arose during the integration. Bugs were found. Bugs were fixed. Workarounds were added. Requests for enhancements were made. Features were added. Best practices were advertised. Quality improved.

Customer adoption of the resulting applications legitimized object-based storage systems as a third category of system (alongside "block" and "file").

Customers. Partners. Storage system vendors. What would they come up with if they had a chance to build a new API from scratch? 

They already did. It's called XAM.

Object Cornucopia

I was introduced to the world of object-based storage in 2003 when I joined the Centera team.  After spending a large part of my career building block-based technologies like CLARiiON, Navi, and PowerPath, the world of objects seemed like a wide-open, growing field and a great opportunity to learn new things for years to come.

I was right.

As I enter my sixth year of exploring new directions for this technology it seems I'm still in the "on-ramp" phase when it comes to the opportunity for building new object-based products and features. 

That translates into some more posts on the topic and (hopefully) collaboration on new directions with other interested parties in the blogosphere.

So I'll start with my own "object" retrospective from 2008.

Digital Curation Conference

I mentioned in a previous post that a paper I had written for the 4th Annual Digital Curation Conference in Scotland was accepted and presented in Edinburgh on December 3rd. The paper was about XAM.

I wasn't able to travel to present the paper but one of my co-workers from Centera presented in my place. I asked him how it went and he said roughly 30-40 people attended and that there were lots of heads nodding.

I do believe that by "heads nodding" he meant that the XAM message resonated with the Digital Curation audience, as I expected that it would.

I should have pressed him further to clarify his choice of words because sometimes terms take on different meanings between the US and the UK. Hopefully head nodding does not imply "dozing".

This post provides a link to the paper in case you wanted to nod your own head (for whatever reason).

Mechelen Closing

As reported yesterday, EMC's Centera development will be moving away from Belgium and the Mechelen development center will be closed. 

I certainly found this news to be disappointing, as the software developers in Belgium have always presented me with some great opportunities to collaborate and build interesting products. In fact we were just on the phone last week brainstorming about new directions. I have always found these discussions quite exciting. Best of luck to them.

I could make a very strong argument that the work done in Mechelen was some of the most disruptive and innovative storage software development of this decade.

In that spirit I'd like to highlight what I think is their main accomplishment.

They took a desktop application (Filepool) and turned it into a 5 9s storage system.

NICE Phone Calls

When I think about the type of fixed content that is traditionally stored on a Centera there are several use cases that come to mind.

Medical images. Scanned check images. Mortgage applications. Anything that comes from a scanner. E-mail.  Electronic files.

All of these use cases represent content that is not likely to be "opened and edited".

Lately I've been reading a lot about another use case that makes perfect sense for Centera.

Phone calls.

Picture a large call center handling 1000s of calls each day. As soon as each call ends, the entire conversation becomes fixed content.  Months later a customer may file a complaint about a service that was promised during a particular phone call.  How can you find the call?   Was it deleted?  Is it authentic?  Is it on tape somewhere?

One of the worldwide leaders in multimedia recording solutions is NICE, and NICE has integrated Centera into their solution.  For example, NICE's Perform product archives phone calls to Centera and then provides a wide suite of analysis tools such as word spotting, emotion detection, and voice analysis. This type of capability is valuable in any market that utilizes a large call center, including public safety, finance, retail, etc.

Integration Details

NICE and Centera integrate via NICE's Storage Center Server (SCS), an architectural component which is responsible for storing and tracking all phone calls entering the system.  The SCS pulls previously stored calls from NiceLog Recorders (pictured below), stores each call to Centera, and receives a Centera content address (CA) in return. This CA is stored in an SCS database along with other information about the call.

The SCS contains software that has integrated with the Centera SDK. This SDK integration takes advantage of the following Centera features:

• Multi-pathing: The SCS administrator can enter the IP addresses of up to four Centera ports in order to realize load balancing and multi-pathing to the Centera system.
• Automated DR failover: The SCS administrator can also enter the IP address of a replica Centera system in order to continue in the face of a disconnect from the primary Centera.
• Metadata annotation: The SCS is able to use Centera's metadata feature to annotate the phone call being archived. For example, the name of the SCS application can be stored along with the phone call.
• Retention: The SCS always stores a retention field in the Centera metadata associated with each phone call. Centera enforces "disallow delete" based on this field. The time is based on the call arrival time (not call archive time).
• Multi-Threading: The SCS fires up multiple threads (default of 20) to allow parallelization in the ingest of phone calls. This is particularly useful if batches of phone calls are archived at night, for example.
• Shredding: Once phone calls do expire, there is often a need to shred the phone call electronically. Centera supports this.

The Centera integration has been successful to the point where NICE refers to a Centera deployment as a "best practice". Over 100 NICE/Centera deployments are in the field. The NICE/Centera integration has eliminated a lot of headaches that used to occur with phone call archiving, including finding/loading tapes, proof of originality/authenticity (provided by Centera's content address), and configuration-free capacity upgrade.

NICE Wholistic Approach

I don't want to shortchange NICE by highlighting just the audio aspects of the Perform product. NICE has a much larger portfolio that can also archive customer interactions such as email exchanges, web interactions, video, etc. NICE provides wholistic management of all of these use cases (which all represent fixed content, by the way).

What I did want to highlight was the unique value proposition gained by NICE several years ago due to the fact that the NICE developers integrated with Centera. 

One more interesting point of note: converting the NICE SCS to use the XAM API would not only maintain compatibility with already stored audio files, but it would also provide NICE customers with the flexibility of moving to future (XAM-based) archival systems.

Steve

Captivating

Today's Captiva "intelligent capture" announce allows customers to put the full-court press on any document that tries to make its way into the enterprise. The announce describes even more ways to classify/identify document types using new services (e.g. .NET scripting, SOA Capture as a Service), as well as the traditional goodies (OCR/ICR, bar codes, check boxes, etc.)

What types of customers benefit from these features? Captiva supports a wide variety of industries, including insurance (claims), financial services (loan documents), healthcare (patient records), and government (tax documents).

There are many reasons for a customer to choose Captiva, but I've got my own personal favorite.

Captiva integrates with Documentum.

And Documentum integrates with Centera.

The 100 Year Paper Clip

One of the requirements faced by digital archivists is the guarantee that archived content will still be accessible and available for years to come. SNIA has written a report on this topic. The crisis described in the paper needs a solution. Will the digital content of today still be viewable and useful in 100 years?

There are many challenges to be solved in building such an archive. One of the more daunting challenges is to manage the relationships between the content being archived (e.g. a photograph or scanned document) and the meta-data describing the content. Wouldn't it be useful to have some sort of giant paper clip to permanently join together all of the disparate artifacts that relate to the digitized content?

Instead of pounding away at this problem with a hammer let's use paper clips instead. And let's use the latest in paper-clip technology.

XAM.

Centera. It's Academic.

East Carolina University's (ECU) data center had been experiencing two common problems: (a) explosive data growth, and (b) loooooong backup windows.

They continue to experience explosive data growth.  Backups, however, have shrunk from 10 hours down to minutes.

Academically speaking, they realized that a huge percentage of their data was unchanging. Unchanging data is also known as fixed content.  In order to reduce backup windows, ECU turned to the one storage system synonymous with fixed content.

Centera.

Centera and SAP

It may not occur to the average student of the storage industry that SAP and Centera go well together.  After all, SAP systems are highly transactional (e.g. frequent database changes), while Centera is geared towards the storage of reference information (e.g. fixed content, or data that changes infrequently).

However, SAP and Centera have been bridged (a variety of vendors have written SAP connectors, see below) together in a very unique and complementary way. This is due to the fact that users of SAP generate two critical pieces of fixed content:  print lists and ADK files.

Once generated, these two pieces of fixed content present a new set of challenges for SAP customers, including capacity management, retention, backups, authenticity, and compatibility (just to name a few).

Whenever customers have these "fixed content challenges" (especially SAP customers), they should consider Centera. Here's why.