Information Playground

Twitter: @SteveTodd

May 27, 2011

Towards the vApp

Several years ago Chuck wrote a blog post about the SpringSource acquisition.Within the post he entitled one of his paragraphs: "Towards the vApp":

"The idea is simple and compelling. Application developer is presented with a "service catalog" of pre-existing corporate application modules. Application developer selects the ones that make sense, and may add a few modules of custom logic that interacts with the pre-existing ones. Each application module presumably runs in its own virtual container.

The developer may need to add in other services as well: perhaps a virtual load balancer, or a virtual firewall, or perhaps a DLP compliance scanner. These are then bound into a "vApp" -- a composite of virtual machines that represents an enterprise application -- all self-contained, all dynamically relocate-able."

Just over a year after Chuck's post, EMC's IT department rolled out a vApp infrastructure that gives employees a "vApp Sandbox" for the very first time. As a user, I am now able to select pre-packaged VMs from a service catalogue, group them together, and build my very own application framework. This is done in a self-service and on-demand fashion, with no need to contact IT directly.

This brief video highlights the simplicity of this sandbox, known internally as "Cloud9":

There is a lot more to the Cloud9 Sandbox that meets the eye. KK, the CTO of EMC IT, wrote a blog post containing a great summary about the capabilities of Cloud9. The infrastructure is built on VMware's vCloud Director. Compare some of Chuck's quotes with KK:

Chuck: Application developer selects the ones that make sense

KK: "ready-to-go Windows & Linux VMs that you can provision in a self-service manner"

Chuck: add a few modules of custom logic that interacts with the pre-existing ones

KK: "users can also add in the following published platform pieces – SpringSource, Ruby on Rails, Groovy, Weblogic, Java, MySQL, PostgreSQL, Oracle and MS SQL Server for now and growing"

There is an extended version of the Cloud9 YouTube video which gives more detail. It's a glimpse into the future as far as IT services are concerned, and addresses the ShadowIT issue, as KK points out:

"It is also on the path of remediating what we have referred to as ‘Shadow IT’ before – i.e. provide a secure, reliable, monitored and managed location for non-IT groups to build applications in an agile manner to meet certain short-term needs."

Steve

Twitter: @SteveTodd

May 25, 2011

The Effects of Sociometric Feedback

I have written several posts about the research that is being done on group collaboration at MIT Media Lab. I attended a dissertation given by Taemie Kim: Enhancing Distributed Collaboration Using Sociometric Feedback.

My first post explored her thesis: measuring communication patterns can improve group performance when feedback on these patterns is provided.

My second post focused on the sociometric badges that were used to measure communication patterns.

My third post described the actual graphical feedback that group members would receive as a result of analyzing the metrics from the communication patterns.

This post dives into the results. The sociometric badges were given to teams. Some teams were co-located and some were geographically distributed. The teams were then given tasks.

One of the tasks focused on a social dilemma that forced each member of the team to choose between maximizing their own self-interest or that of the team. After eight minutes of discussion, they made private decisions. The "eight minutes" were measured using the sociometrics badges. The private decisions measured their cooperation level.

Another task involved information-sharing. Each member was given information that was hidden from the others. Four minutes of sharing would then be followed by the team trying to generate a "question" that was reflective of the totality of shared information (the more information shared, the more accurate the generation of the question). The "four minutes" were measured using the badges, and the proximity of the question to the ideal would measure their ability to share information.

During the information-sharing task, the distributed (non-co-located) groups received biometric feedback in different ways:

The "no feedback" groups did not perform as well as the others. The other three groups increased their speaking times, speech energy, and turn transitions. In fact, the diagram below shows that it is not the feedback itself which improved performance, but the improved communication patterns. Taemie used a mediation model that showed the following:

For those of you that are interested, the theory behind this graph is explained by the Baron & Kenny Mediation model.

In a nutshell, the sociometric feedback improves the communication patterns, which improves the group's ability to share information. Sociometric feedback on its own can actually hurt information sharing (e.g. the teams pay more attention to the feedback than to the sharing of information!).

Hearing the dissertation made me interested in using the devices. I believe that Taemie and team are forming a startup to launch this invention as a way for geographically distributed teams to improve performance. It would be interesting to use the devices in this way.

Regardless, the research is a very cool way of re-stating that age-old adage: share and take turns! It's a critical element

Steve

Twitter: @SteveTodd

May 18, 2011

Replicating Your Friends

Several weeks ago I read two interesting blog posts about the EBS failure. Jeff Darcy gave the developer's view of failures in distributed systems, while Robin Harris provided a great overall summary of what happened.

Any time I hear about random failures leading to timeouts (leading to retries), I think about staged event-drive architectures (SEDA). I heard about SEDA when Matt Welsh (currently of Harvard) came into EMC to talk to the Centera team. His thoughts about queues and feedback mechanisms were influential to Centera (which in itself is a distributed system).

No matter how you slice it, replication is a hard problem. Replication of online social networks (OSNs) are presenting new challenges that researchers are trying to tackle.

OSNs, like Facebook and Twitter, often use Cassandra for the ingest and organization of content. Cassandra uses a distributed hash table (DHT) as part of its operation. DHT's do not preserve social locality of reference (keep my data close to my friend's data). Social locality of reference is a key performance enabler during OSN query operations (e.g. refresh my home page feed). Research is already underway on the development of new algorithms that leverage social graphs. These algorithms attempt to co-locate data in an OSN (e.g. see the SPAR presentation based on the SIGCOMM 2010 paper).

This brings up an interesting question: can social locality be preserved at a disaster recovery site? Are there new replication algorithms that can leverage social graphs during replication?

This topic has been researched as part of a collaboration between EMC Research Cambridge and UMASS Boston. The resulting paper, Preserving Social Locality in Data Replication for Online Social Networks, will be presented next month at SIMPLEX 2011. I won't steal too much of their thunder ahead of the conference, but there are some new algorithms coming. I will say that the algorithms are replication-friendly.

Why is EMC expanding its knowledge of distributed algorithms in Cambridge? The research is essentially an outgrowth of the first place award winner in EMC's 2008 Innovation Showcase. Winning ideas go through an incubation process within EMC. After the incubation period, a course of action is recommended. Given EMC's emphasis on research partnerships with area universities, UMASS Boston was chosen based on the quality of their PhD candidates.

If you have any questions about the paper or upcoming conference, please contact me by leaving a comment below.

Steve

Twitter: @SteveTodd

May 13, 2011

Global Collaboration and Visual Feedback

In a previous post I highlighted how sociometric badges can be worn by group collaborators. These badges were tested by Taemie Kim of MIT as part of her PhD research. Her thesis was that feedback from sociometric badges could be used to improve group performance on a given task.

The badges measure a variety of individual metrics, including:

Body movements
Speech features
Whether or not wearers are facing each other
How close wearers are to each other
Interaction patterns
Etc.

The goal of the research is to take this data and somehow reflect it back to the team in a way that influences the overall group effectiveness. Two different approaches were tried:

Individual Feedback

This approach visually displays a collaborator's participation in comparison to the sociometric feedback of everybody else in the group.

These diagrams are visual representations of the sociometric badges. They communicate (to an individual) the speaking time (size of the circle), interactivity (color of the circle), and how well everybody is taking turns (location of the circle). This visual feedback is immediate and lets the individual know detailed information about their level of participation.

Group Feedback

This approach visually displays feedback at the group level:

In the example above, the location of the circle shows how balanced the participation is between groups, while the color of the circle measures group interactivity.

What are the results of the study? The feedback was shown to increase group performance of remotely-distributed teams. I'll describe the specifics of this phenomena in a future post.

Steve

Twitter: @SteveTodd

May 11, 2011

Robots Invade Vegas

For those of you attending EMC World, it's time to vote for "most innovative product". Out of all the announcements this week, which do you consider to be the most innovative?

Is it the cool new Root Cause Analysis software?

The cloud elasticity capabilities of VPLEX Geo?

The VNX connector to the Google Search Appliance?

The biggest file system in the world?

Hadoop Big Data analytics?

All of these new features (and many that I haven't mentioned) represent innovation and delivery by some of the most talented nerds in the industry. But....

They were all upstaged by some high school kids from California.

Because nothing is cooler than building a robot.

And the winner is.......

Out of all the products announced during EMC World this week, the "most innovative in show" belongs to three student teams from California:

Team 687D from California Academy of Math & Science in Carson, CA
Team 687F, also from California Academy
Team 1138, from Chaminade College Preparatory, West Hills, CA

All teams, as part of the VEX Robotics program, received a grant to fly to Vegas as part of EMC's Community Involvement program. EMC sponsored teams from all over the country, including Nevada, Washington, and Maryland. The California sweep was the result of some clever alliances formed during a multi-stage competitition against opposing robots. Judges award points during the competition as well.

The VEX Robotics program is innovative in its own right. The program increases student excellence in science, technology, engineering, and math (STEM), and also teaches leadership, teamwork, and collaboration among groups.

As you can imagine, the students had a blast. They travelled to Vegas, smashed robots into each other, and three skilled teams received their awards from EMC this morning.

To top it all off, every student was able to experience the thrill of a lifetime: a photo op with Bubba!

Steve

Information Playground

Twitter: @SteveTodd

May 09, 2011

The VNXe Fastball

After having worked on the VNXe for several years, it will be a pleasure to watch the product hit the floor of EMC World for the first time.

In order to emphasize the small form factor of the VNXe, the product has been riding around in the back of a Mini Cooper for several months. In spite of its size, customers can pack a serious amount of information inside the VNX's younger sibling.

When the Mini Cooper arrives in Vegas, however, the VNXe will be kicked out of the backseat for two reasons: (1) It will give EMC Marketing more space to cram as many baseballs into the car as possible, and (2) the VNXe will given to a lucky (or accurate) EMC World attendee.

But these aren't just any baseballs.

These are bona-fide EMC and Cooperstown baseballs.

Normally I would try and get my hands on one of these baseballs and have it signed by somebody famous (like StorageZilla, for example).

But in this case the baseballs are being given to charity. EMC and the National Baseballl Hall of Fame will be distributing the baseballs to organizations that support youth baseball (e.g. Boys and Girls Club of America).

So if you want one of these cool baseballs, I'm sorry, I can't help.

But if you want a VNXe, I might be able to help.

First, I suggest you show up at the Brocade booth (BOOTH 116).

Second, I suggest you measure the width of the baseballs that will be stuffed into the car. I've attached a picture of the baseball below. Hold up a ruler to the screen to get an approximate width.

Third, go online and look up the specs of the Mini Cooper in terms of cubic feet. I'm not going to tell you what it is because I've already helped enough with the baseball.

All that's left is to DO THE MATH. Now you have an edge over everybody else.

Why Cooperstown and EMC? The two parties are in the midst of an exciting collaboration to digitize many of the assets that can be found at the Coopertown National Baseball Hall of Fame and Museum. The Museum's collections contain more than 38,000 three-dimensional artifacts representing all facets of the game, from its inception in the mid-19th century to present. In addition, the Institution’s archives contain in excess of 135,000 Baseball cards and three million Library items, including photographs, books, magazines, newspaper clippings, films, video and audio tapes.

The Hall of Fame is dedicated to preserving the game of baseball for generations to come, and the charitable donation of the baseballs is one way to keep those kids playing and bats swinging!

Steve

Information Playground

Twitter: @SteveTodd

May 05, 2011

Framing the Cloud Portfolio

In a previous post I discussed a strategy for establishing research directions between EMC’s Cairo Center of Excellence (COE) and Egyptian Universities and national technologists. I used the diagram below to describe the initial information exchange that will serve as the foundation for establishing research directions.

A critical part of the innovation exchange process is framing the conversation. Any discussion about a cloud product portfolio should use the language of the customer use cases. They do not want to hear about “storage”, or “information security”, or “management frameworks”. They would prefer to receive a technology overview that maps to a specific subset of cloud characteristics (e.g. as defined by NIST).

What I’ve listed below are three sample characteristics, followed by product offerings that map to the category. When it comes to cloud, EMC’s majority ownership of VMware adds a large amount of technology to the discussion. The list is by no means complete, and several product offerings span multiple categories.

Once the product capabilities have been described, the teams will then use the innovation-by-adjacency approach to brainstorm research directions for cloud use cases.

Resource Pooling (multi-tenancy)

VMAX/VNX: scalable cloud storage systems that support multi-tenancy.

Vblock: pre-integrated, tested, and validated server, network, storage, and security for cloud deployments from VCE.

Isilon: cloud file system technology scalable up to 10PB.

GreenPlum: massively parallel, cloud-scale data storage and analytics.

Atmos: geographically distributed, multi-tenant cloud storage for unstructured content.

Documentum: a centralized document repository that supports multi-tenant workflows via xCP technology (see this use case).

vCloud Director: catalog-based tool enabling resource pooling of the underlying infrastructure.

Elasticity

VPLEX: federated data mobility and connectivity within and between cloud data centers.

CloudFoundry: normalized, open platform for application deployment to multiple cloud service providers.

RabbitMQ: industry standard cloud messaging queuing within and between clouds.

GemFire: scalable, high-performance memory-based caching across distributed servers.

Measured service

UIM: unified cloud infrastructure manager for vBlock, gathering statistics across server, network, storage, and application layers.

Hyperic: high-volume virtualized application and infrastructure metrics and eventing.

In addition to the subset of NIST characteristics, there are several other cloud topics that round out a more complete portfolio:

Cloud Application Platform Technologies

Spring: Enterprise Java development platform

ERS (Apache): Enterprise-ready Apache with cloud load balancing

tcServer: Tomcat runtime server for cloud applications.

Cloud Security Tools and Services

eFraudNetwork: RSA’s shared CyberCrime repository.

Consumer Strong Authentication: RSA Machine Learning/Modeling of cloud consumer transactions.

Archer GRC: industry standard governance profiles from the Cloud Security Alliance.

Tricipher: Federated Identify for Cloud Services.

vShield: security for virtual datacenters

A thorough discussion of this portfolio, when framed against the cloud use cases, should lead to the creation of research proposals. The portfolio of products listed above is by no means a complete list, but it is more than enough as a starting point.

Future posts will focus on EMC’s internal cloud roadmap and how it relates to potential topics for cloud research.

Steve

Twitter: @SteveTodd

May 04, 2011

Sociometric Badges

In a previous post I introduced a PhD dissertation by MIT’s Taemie Kim: Enhancing Distributed Collaboration Using Sociometric Feedback. The announcement for the dissertation summarizes the topic quite well:

Results show that sociometric feedback influences the communication patterns of distributed groups to be more like that of co-located groups, which results in an increase in performance.

The MIT Media Lab is known for creating gadgets and sensors that measure all sorts of biometric data. For this research the team created sociometric badges, a “wearable sensor that detects communication patterns of multiple wearers”:

These badges are able to take many different kinds of measurements, including:

Body movements
Speech features
Whether or not wearers are facing each other
How close wearers are to each other
Interaction patterns
Etc.

The researchers used sociometric badges for capturing this type of data during team working sessions. In some cases the teams were co-located, and in others they were distributed. The badges emit sensor data that can be graphed and analyzed. Shown below are two such graphs, one that measures the speech energy (think enthusiasm, for example), and one that tracks personal movement.

Speech Energy

Movement Energy

As each team worked on their different tasks, the sociometric badges churned away and created data.

What would happen if distributed team members received immediate feedback on the data? Would it effect team performance?

The answer to this question is the heart of the research. I’ll step through the results in a future post.

Steve

Twitter: @SteveTodd

May 01, 2011

Cairo: A Cloud Technology Collaboration

In a previous post I discussed a strategy for establishing research directions between EMC’s Cairo Center of Excellence (COE) and Egyptian Universities and national technologists. The exchange will focus on cloud technology, which is an area of considerable interest and growth as Egypt begins deploying more agile IT infrastructures. Different organizations in Egypt are interested in partnering with EMC given the large number of cloud assets the corporation brings to the table.

I used the following diagram to explain the knowledge exchange that will occur first. This exchange will be the foundational starting point for exploring research and innovation partnerships.

I’d like to use the rest of this post to describe the left-hand side of the diagram. There are two areas that are particularly relevant for research discussions:

The internal cloud being built by EMC’s IT organization. EMC desires to build a cloud that satisfies the use cases of EMC’s 50,000+ person organization. This cloud will span multiple geographies, support Infrastructure-as-a-Service, Platform-as-a-Service, Hybrid cloud services, application mobility, etc., etc,. etc. In order to pull this off the EMC IT has an architectural roadmap that extends years into the future. Some of these architectural tasks are so far out that EMC IT is not even sure how (or if) the features will (or can) be implemented. These tasks make excellent candidates for research partnerships.
EMC’s Product Portfolio contains a myriad of products that satisfy many of the five characteristics of cloud computing (as defined by NIST):
1. On-demand self service
2. Broad network access
3. Resource pooling (multi-tenancy)
4. Rapid Elasticity
5. Measured Service

As these products are described to our Egyptian collaborators, “gaps” will appear as they describe their own cloud requirements and use cases. These gaps also serve as excellent candidates for research partnerships.

The suite of cloud products is substantial. In a future post I will present an overview of the cloud technology portfolio and explain the process of “gap mapping” to research projects in more detail.

Steve

Twitter: @SteveTodd