Information Playground

Brazilian Big Data Research Round-up

One of the more interesting ways to discuss Big Data is from the research angle. Chuck pointed out several weeks ago that analytics is not the only theme. As he said: "There's another side to big data that is more about dealing with big files vs. big databases".

I have been paying close attention to Big Data research knowledge expansion across EMC. The two uber-themes (analytics and massive files/dbs) can be broken down into specific research topics. Some topics are being investigated across our global locations. Other topics are on the to-do list. The following list represents different research areas for Big Data (note the strong emphasis on performance):

• Ingest and Export Speeds. Getting the fish from the ocean to the plate as fast as possible is a critical research area. University researchers are still working on this problem nearly a quarter century after RAID.
• Information Storage. Should it be encrypted, compressed, deduplicated, or some combination thereof?
• Application-aware Data Layout. While distributed scale-out architectures and recovery mechanisms are fairly mature, the performance implications of data layout can vary from application to application. This is closely related to workflow (see below).
• Tiering and Mobility. The movement of big data between tiers for performance improvement extends up to the server flash level. The volume of data causes this research topic to overlap with the ingest/export theme.
• Use cases/Workflows. Understanding the applications that need access to Big Data becomes critical for a number of reasons: performance and configuration management are just two of them. Common workflows include:
  • Medical
  • Energy
  • Satellite
  • Video
  • etc.
• Collaboration. All of the scientists working on a data set are not necessarily in the same room or facility. The ability to annotate Big Data is an extremely helpful feature, and can result in loosely coupled Big Data / Small Objects. I'm a big fan of the GreenPlum Chorus approach, but it needs to extend  beyond the enterprise model. Which leads to....
• Geographic Mobility. Big Data captured into one data center may not necessarily stay there. Distributed scientists may want to bring the algorithms to the data and/or vice-versa. 
• Visualization. This may be one of the more fun areas of research to watch. Creative visualization may yield more influence than any other research topic.
• Analytics. As a computer scientist I always loved saying "BubbleSort". Analysis of algorithms lives on.
• IT infrastructures. The intersection of Big Data and Cloud. Storing data of this size and magnitude will impact all five cloud characteristics.
• Preservation.  If Big Data must age like fine wine, then the methods and processes used by digital curators can be applied to the field of Big Data.
• Provenance.  Where did all this Big Data come from?

Often times EMC acquires a company to enter an emerging market.

In Brazil, EMC has invested in a new research facility to expand research coverage of Big Data. This initiative will clearly result in knowledge expansion in the area of Oil and Gas workflows, but the research will also certainly spill over (no pun intended) into some of these other areas as well.

I've been involved with provenance research in the past and will continue to post updates as we move the research forward in many of these areas.

Steve

Information Playground

Twitter: @SteveTodd

EMC Intrapreneur

 

Innovation Longevity

This past week I visited Cambridge, MA and met with MIT Sloan Research Scientist George Westerman. The topic was EMC's Innovation Network and how to effectively run a global collection of research nodes that are responsible for advancing EMC's technology strategy.

One of the reasons for meeting with George on this topic is that it directly relates to his area of expertise. He studied 19 different innovation committees or teams and looked at their long-term ability to survive at their respective corporations. As the EMC Innovation Network enters its 6th year, I was interested to hear his findings.

He told me that 6 out of the 19 innovation teams survived. Out of those 6, he analyzed why they were successful. He broke it down into two phases: year one and beyond.

1. The first year of these successful teams were characterized by being separate, sponsored, and small. We did not dive into each term in detail, but I'm guessing he meant the following:
   • Separate: the group needed a certain level of autonomy.
   • Sponsored: the group was in some way "blessed" by the corporation, and in particular a high-ranking executive.
   • Small: the group was made up of roughly half a dozen people, give or take.
2. Beyond the first year, these teams stayed successful by becoming systematic, shared, and seen. My interpretation:
   • Systematic means that the team developed an effective, productive cadence
   • Shared means that the group expanded beyond their small team and began effectively partnering with others.
   • Seen means that their work became visible and recognized

Given that EMC's Innovation Network (EIN) falls into category two, I feel that George's theory applies. The EIN has certainly become systematic, shared, and seen. For example, Pat Gelsinger sent out an email to every EMC employee this week encouraging participation in the yearly Innovation Showcase.  That is an example of a high visibility team.

I'm going to apply his advice to the newly formed global team I'm involved with. We are in month one. According to George, we should work on being separate, sponsored, and small.

In a year or so, with any success, I'll write another blog post about our transition to being systematic, shared, and seen.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

The Flip Side of the COIN

One of the more interesting challenges working at EMC is the amount of global university research and innovation that occurs at facilities around the world: Russia, Ireland/Europe, Israel, Egypt, China, India, the United States, and most recently Brazil. It's challenging to track it all and collaborate efficiently.

I've been thinking about different ways to solve this problem. It occured to me that perhaps the solution should be crafted by the global community instead of at corporate headquarters in the United States. I began to look for a way to quickly form a team that works towards a vision of increased global collaboration for research and innovation.

I was also looking for a way to do it without asking permission. That sort of stuff takes too long, especially when it comes to global sign-off.

My research led me towards COINs: Collaborative Innovation Networks. If you would like a primer on COINs, I would start with the Wikipedia article. The article mentioned the following reference:

Peter Gloor (2005) Swarm Creativity: Competitive Advantage Through Collaborative Innovation Networks

I ended up purchasing the book and it was quite helpful. In addition, I found out that Peter works locally at MIT, so I contacted him and we got together. In fact, he invited me to a PhD thesis on global collaboration that I recently wrote about. 

I explained to Peter my thoughts about finding a better way to collaborate on global research and innovation. He told me that a COIN would be a wise idea. (He also expressed a significant amount of skepticism on whether a COIN could be implemented at a large corporation like EMC ;>).

He then laid out a set of logical steps for COIN formation. I took the following notes as he spoke:

1. Introduce the vision.
2. Create a "one-liner", an easy-to-memorize vision
3. Familiarize the people with the vision
4. Cede power
5. Set short-term and long-term goals
6. Keep it small, and try for regular face-to-face meetings
7. Blunt language is good
8. Build momentum via webcasts
9. Host brown-bag sessions
10. Spawn COIN prototypes

Peter reiterated that step #4 is often the hardest, especially at a large corporation. I've tried to follow these steps and we had our first meeting on Tuesday. The meeting literally spanned the globe from as far east as Beijing to as far west as California.

I recommend the Swarm Creativity book. The best part about a COIN (if we can pull it off) is that it leads to Collaborative Learning Networks (CLNs) and Collaborative Interest Networks (CINs).

I hope to write more about this journey as it unfolds.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

 

 

Cooperstown Symposium

Yesterday was Day 1 of the 23rd Annual Cooperstown Symposium on Baseball and American Culture. I'm speaking Friday about "The Digital Preservation of Baseball".  I discovered the conference during a visit to the Cooperstown Hall of Fame last November. The Hall of Fame and State University of New York Oneonta put out a call for papers and the topic of digital preservation is very relevant given the HOF-EMC partnership.

Here is a list of the lectures that I attended during the first day:

• Roberto Clemente as a model for Continuous Quality Improvement, by Oren Renick of Texas State University.
• Does George Steinbrenner Belong in the Hall of Fame?, by Joseph Dorinson of Long Island University.
• Pete Rose's Renewed Image Repair Discourse on the 25th Anniversary of Breaking Ty Cobb's Hit Record, by Todd McDorman, Wabash College.
• Competitors and Colleagues, The World of Baseball Scouting, by Lee Lowenfish, Columbia University.
• Unbroken Circle (about deaf baseball player Luther Taylor and some of his contemporaries) by Rebecca Edwards, Rochester Institute of Technology.
• The US Women's National Baseball Team of 2010, by Jennifer Ring, University of Nevada.

Red Sox fans would have appreciated Joseph Dorinson's answer to the George Steinbrenner question. He spent over 20 minutes justifying his adamant "no" in a very humorous way.

I was interested in the discussion about Pete Rose's image, given that I was a fan of the Big Red Machine in the 1970s. It was fascinating to watch a professor of rhetoric analyze Rose's image-repair strategies. He claimed that Rose has been using a wide variety of strategies: denial, bolstering, evasion, reducing offensiveness, transcendence, mortification, and corrective action. 

The keynote speaker to open the conference was Jack O'Connell, secretary/treasurer of the Baseball Writer's Association of America. The secretary/treasurer of the BBWAA is the one that "makes the call" to new Halll of Fame inductees. (for a great story about some of Jack's calls, read this article).

Jack painted a great picture of what it means to be a baseball writer. He related the formation of the BBWAA in 1908 as a statement from writers to owners that "writing is the primary medium between the game and the fans". Work conditions were abysmal, and the writers essentially said "just give us a place to work".  To this day, writers view their press box as a business-only location; e.g. no cheering and no autographs. In the 1930s the respect for the BBWAA reached the point that they were given responsibility to select players for awards (such as the MVP).

Perhaps the most interesting insight from Jack is that "he had it easier" compared to today's writers, who face a deadline of "now" for every pitch (never mind just delivering a game summary!). In addition, they often have to be extremely video savvy and quick on their feet in the social-media dominated landscape of blogs, Facebook, Twitter, and YouTube.

I'm looking forward to my presentation on Friday; I will be following two archaeologists from St. Louis and a history professor. I hope to post more from the Symposium soon.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur