Information Playground

The Demo Cloud

This week I attended an internal meeting about EMC's "Demo Cloud".  The demo cloud hosts a wide variety of EMC product offerings that can be demonstrated to any number of customers. These products range from simulations that highlight user interfaces, to pre-recorded demos, to full-featured virtual storage appliances (like the Celerra VSA).

This cloud is accessible 24x7 from anywhere in the world by logging into one of a number of "demo accounts". Each account provides a catalog of available product revisions. It's possible to pull up an impromptu demo at any time. The software and hardware environments that actually implement the demo cloud can be found in multiple locations, including Ireland, Russia, and the US.

Customer demos that are critical and scheduled in advance can be "allocated" to run at a certain time (and for a certain duration). This prevents against a spike in spur-of-the-moment "demo storms" that might lock out or otherwise disrupt an important demo. Are demo storms likely?  In 2009 the demo cloud hosted  17,000+ separate sessions. In 2010 this will increase significantly.

Compare this process against the demo logistics of 2-3 years ago. The ease of giving a demo in this environment highlights why there is so much buzz about delivering IT services via the cloud.

During the discussion we touched on delivering training via this environment. We also discussed internal usage of the demo cloud for engineers like myself. And then the light bulb went on.

One of the more regular activities I participate in these days is the packaging and internal distribution of  futuristic virtual storage appliances. Other groups can modify the VSA to explore new product offerings. At the moment I have 2-3 different groups doing this very thing. Each of them currently performs a LONG ftp of the image to their own ESX server or VMPlayer, and together we go through the bring-up and education process. It may take 1-2 days of hand-holding. It could take a week if we are doing it as a skunkworks (e.g. in our spare time).

Whatever the case may be, the demo cloud makes things a lot easier internally. Once the VSA is uploaded to the demo cloud I can form an "internal" demo account, create an instance, and send a URL to my co-worker. The URL brings them immediately to a booted-and-ready-to-go image.

In theory, as we start expanding our global research efforts with universities, partners, and customers, we could use some form of this process.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Wonky Provisioning

I was just re-reading two posts from early March: Rick Cook's article about tips for LUN provisioning in SMB storage, and 'Zilla's post about the value of turning storage knowledge into "application-speak".

These posts present two different ends of the spectrum. Rich's post is essentially a primer on logical units (LUNs). He introduces his post in the following way:

Although extremely helpful in large data storage enterprises, LUNs are just as important for small- to medium-sized businesses (SMBs).  This tip explores the background of LUNs in an SMB data storage environment, the different LUN features vendors have to offer and how to determine what type of LUN is best for your environment.

His post goes on to explain such concepts as striping, RAID-levels, LUN masking, zoning, etc.

I agree with the first statement (LUNs are important for SMBs), but I'm not sure if I agree that SMB administrators will want to "determine what type of LUN is best" for their environment.

This brings me to the 'Zilla-angle: RAID, LUNs, and masking are "wonky" to IT generalists and application administrators. I translate "wonky" as:  "I'd prefer not to know about these terms if they are not necessary for me to get my job done". Indeed, he clarifies this later by using the following statement about provisioning for applications:

    whatever voodoo has to go on to make that happen should be something they [administrators] don’t have to think about

User interfaces in typical data centers provide a high degree of visibility to LUN concepts.  What 'Zilla is positing is perhaps the concept of LUNs shouldn't be exposed at all. Wizards and workflows that are framed in the context of the application are more appropriate.

I like this line of thinking:  introduce application-aware workflows into SMB environments and extend them up into the enterprise (as opposed to bringing enterprise-level screens down into the SMB market).

I'm interested in hearing people's thoughts. I'm also interested in hearing if I've translated the Irish meaning of "wonky" appropriately.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Cloud Query: Prove It

When it comes to securing content in the cloud, section 7.2 of the AWS Customer Agreement currently says the following (in part):

We strongly encourage you, where available and appropriate, to (a) use encryption technology to protect Your Content from unauthorized access, (b) routinely archive Your Content, and (c) keep your Applications or any software that you use or run with our Services current with the latest security patches or updates. We will have no liability to you for any unauthorized access or use, corruption, deletion, destruction or loss of any of Your Content or Applications.

The encouragement to use encryption technology is something I can understand if my application sits outside of the cloud and I am sending files back and forth. On top of encryption there are other techniques that can be used to detect "corruption".

But what about database queries? When a query is sent into a cloud and a result comes back, how can we know that it's correct?

Trustworthy Outsourced Computations

I listened in on a lecture today given by Nikos Triandopoulos of Boston University. He visited EMC Cambridge as part of the monthly lectures sponsored by the EMC Research Cambridge Lecture Series.

Nikos introduced his topic by highlighting that cloud has inserted itself in between users and their trusted databases. When a database query enters a cloud it can have any number of "proxy" hardware and software entities that pass data back and forth. His research is trying to solve two problems:

1. How can a user verify the "correctness" of a query result? How can the user be sure that malicious software did not intercept the query and return a bogus answer?
2. How can this be done efficiently (e.g. performance optimized)?

At a very high level this research is attempting to do the following:

When a user submits a database query into the cloud, a cloud server will typically intercept and pass it on to the "true" database server that stores the actual database. The diagram above highlights that when a query result is returned to a user, it is accompanied by additional metadata that is labeled "proof". This metadata allows the user to determine whether or not the query is correct or not.

That's heady stuff, and the implementation is more than I can communicate in one blog post. But here is the gist of how something like this can be implemented.

First of all, the entire database has a hash function run against it, which results in a top-level "digest".

The same hash function is applied against pairs of adjacent database entries, resulting in a "digest-per-pair".

This hash function is then recursively applied against sets of adjacent pairs until the complete representation forms a Merkle tree, with the top-level digest at the top.

When a query result is returned, nodes in the Merkle tree are also returned, and the user can perform a mathematical calculation using the query result, the nodes, and the overall top-level digest.

Sound like a lot of work? What about database updates? Isn't it slow? The rest of Nikos' lecture touched on many of these questions.

As I listened on the phone I could hear a lot of questions being asked, and I recognized the voices of my RSA co-workers.  Encryption and correctness of database operations is right up their alley, and it will be interesting to see if future product roll-outs will leverage these ideas.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Side Projects

I always have a list of side projects on the back burner that co-exist with my main product deliverables. I added a few more to the list last week.

xCP and the Monster Mash

I'm always up for a contest, and last week I wrote about the announcement of an EMC coding challenge sponsored by EDN. In particular coders are encouraged to use the Documentum technology known as xCP. This technology claims to enable easy workflow creation, and last year Craig Randall and I proposed the combination of xCP and XAM technologies to address the Digital Preservation market.

When I heard about the Monster Mash I figured: "Why not try and build something"?  It's really a win-win if I can find the time. So I signed up and my first few days have gone something like this:

Day 1: read the contest rules and register

Day 2: download xCP and install

Day 3: figure out why my config seems to be blocking some of the Documentum ports

So I am off and running, spending 10-15 minutes here and there. I look forward to blogging more about about xCP and the prototype I hope to build.

Prove It

@gminks contacted me via IM and by the time we were done chatting she had signed me up to study for EMC Certification as part of the EMC Proven program. She sent me the textbook and I've started to read a chapter here and there at lunch and take notes. Why not follow in 'Zilla's footsteps? Doesn't everyone want to follow the trendsetter?

Chasing Funding

Once per month I'm on a virtual research team in Cambridge, MA that connects with local universities. After some lecture-swapping with Harvard we are interested in funding some research on scientific archiving (astronomy and medical). Chasing funding within EMC is a new experience for me and the angle I've been using is to ask interested business units for dollars that they would otherwise spend on co-ops this summer. Stay tuned.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur