Information Playground

Contests and R&D

My group is trying something different as part of our development cycle. Currently we're in the testing phase with our software while also beginning to think and plan about software that we'll write next year.

Typically during the planning phase the task of brainstorming about future efforts is given to a privileged few (e.g. senior architects).

This year we decided to follow the corporate lead and hold our own internal innovation contest within our business unit. While any type of idea could be submitted, we were really looking for very targeted and specific ideas for next year.

Our deadline for ideas was mid-June and voting for ideas was closed out yesterday. From what I've seen so far, a well-run idea contest can be an integral part of the development cycle.

Contest Logistics

The contest software we used was based on the "open innovation software" known as Idea Torrent. Earlier this year my co-workers in China customized Idea Torrent to make it more consumable inside of EMC (e.g. integrated it with internal single-sign-on servers).

The Idea Torrent framework does all of the things that social-media savvy users expect (tagging, commenting, and searching) as well as a great Digg-like voting mechanism. This allowed our developers to search for tags and search strings that were relevant to their interest, find overlapping ideas, and add comments to the proposals.

Participation spanned three different locations: two in the US (Mass and NC) and St. Petersburg, Russia.

Contest Experience

For the two weeks that idea submission was enabled, there was a steady stream of submissions, including the "hockey stick" of ideas that arrived on the last day of the contest. The voting mechanism resulted in several humorous exchanges between developers, especially when it came to the politicking and promoting of ideas as part of the everyday banter of work.

Not suprisingly (to me), participation was highest at our St. Petersburg site. I believe the contest was viewed as a way to propose (and perhaps potentially own or drive) new, exciting work for the future.

Contest Results

Now that the voting has closed there are a variety of conclusions that can be drawn:

• The most popular ideas were the ones that addressed common pain points experienced during development.
• The most common pain points were often "process-based", addressing inefficiencies in the software development experience that were taking too long to complete
• Many of the ideas were development tools that would save time
• Many ideas were "ease-of-use" improvements for the product
• Many ideas re-introduced the projects that were put on the "back burner" during development.

A number of the ideas were so straightforward that we've decided to hire a few interns to try and implement some of them this summer.

Contest Next Steps

Ever create a project plan? Well, the best and/or most promising ideas will be tracked as part of the project plan for our release next year.

What do I like best about this?  It forces everyone to "pause" and re-evaluate their way of doing things, and collaborate together on common pain points. This informs the managers that a certain percentage of the work for next year falls into the category of infrastructure and process improvement. This aspect of project planning often takes a back seat to feature and function in the software itself.

The timing of our contest also dovetails quite nicely with EMC's global innovation contest (which is running in parallel but lasts a much longer time).  Ideas generated by our developers can be fed into the global contest as well.

The experience has been positive to the point that I would definitely recommend "innovation-by-contest" as an integral part of the software R&D cycle.

Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd

A Tale of Two Customers

I received an email this week about two customers who are faced with the exact same problem: improving IT efficiency.

The common threads in their quest for IT efficiency are consolidation and virtualization. At a high level it stands to reason that by consolidating systems and deploying virtualization their IT infrastructure will become more efficient.

One of the customers (Highmark) is in the healthcare industry.  The other customer (JCPenney) is in the retail industry.

Highmark's Requirements and Challenges

As Highmark goes about solving the problem of IT efficiency they have some complex regulatory requirements that are common to healthcare infrastructures:

• security
• records retention
• standardization of healthcare records
• navigation of financial transactions (payer, provider, patient)

In addition, Highmark is facing a challenge in the area of service level agreements (SLAs) given their mixture of mainframe and open systems.

JCPenney's Requirements and Challenges

As JCPenney goes about solving its problem of IT efficiency they have a different set of requirements that fall into the category of uptime in their back office infrastructure.

• building HA and redundancy into the back office
• re-defining data protection
• scoping business continuity in terms of back office applications

The main challenge?  Historically back office applications aren't geared towards high availability. In JCPenney's case, this has to change.

A Tale Of Two Approaches

Both business will consolidate and virtualize in order to solve their problems, but they will do so in different ways.

Highmark will focus on a service-oriented approach, reducing complexity in their own computing infrastructure via stack integration.

JCPenney will build redundancy directly into applications with a specific emphasis on databases.

What's interesting about these two particular customers?

They have both volunteered to sit down with Dave Vellante of Wikibon and publicly discuss the strategies that I have mentioned above (thanks to Dave for providing the above information).

Mark O'Gara of Highmark and Daryl Molitor of JCPenney will join Dave this Monday June 22 at 3:00-4:00 PM EST.

I definitely plan on listening in, and you can too. Register here.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

Considering MIT Dataspace

The EMC Innovation Lecture that I wrote about several weeks ago went well. A webcast is available here. A few dozen people dialed in to listen to myself, Burt Kaliski from EMC, and Stuart Madnick from MIT talk about Digital Curation and a variety of related topics. Wayne Adams (EMC) from SNIA was also present during the call to talk about what's been going on lately with the XAM standard within SNIA.

Several attendees called in to ask questions and/or submitted them via chat.

When the lecture was over I found that I had a question of my own:

Can scientific research repositories benefit from archival standards?

MIT DataSpace

Distributed silos of research information don't necessarily lend themselves to external re-use. For example, if large amounts of anonymous medical X-RAYS are stored at a hospital for research and analysis, can the X-RAY data (and the results of the research) be accessed by another research team?  If environmental data is captured off the coast of Massachusetts and used to measure ocean temperature trends, can that data be re-used by other environmentalists around the world?

The answer can often be no, and the MIT DataSpace project is a proposal focused on turning that answer around.

The proposal has been submitted to the National Science Foundation (NSF). The NSF is accepting proposals as part of its DataNet initiative. If funded, MIT and a host of other academic and industry teams will work on a solution.

What is the Format of Research Data?

Back to my question. Research data in repositories around the world have a variety of different formats. Some of it exists in databases, some of it exists in file systems, and some exist in both. Let's assume that DataSpace would propose that a standard method be used to take these existing systems and provide some standardized form of access to the data.

And let's assume that DataSpace would like this research data to be available for a long time.

Keeping digital data for a long time is the job of a digital curator, and most digital curators have adopted a standard for their digital archives known as the Open Archival Information System (OAIS).

How does scientific research data map to OAIS?  Consider the oceanographic data example. The temperature measurements are the key piece of data, known in OAIS terms as a Data Object. Any Data Object, when it is contained within an OAIS archive, must be accompanied by the following mandatory metadata:

• Representation Information: how to interpret the oceanographic data. What format is it in?
• Reference Information: a persistent identifier that uniquely identifies the data.
• Fixity Information: authenticates the oceanographic data
• Provenance Information: documents the history of the data
• Context Information: metadata or description describing the data

It seems to me that the above five fields, if consistently made available for scientific research data, would be highly valuable. This leads me to conclude that the techniques used in long term, OAIS-compliant digital curation should be studied in the context of standardized access to digital research archives.

The next step that I'm interested in pursuing is finding some use cases (e.g. real-life research repositories) and beginning the process of mapping them on to OAIS.

I'd love to hear from anyone that has already thought along these lines (or is interested in doing so).

Steve

http://stevetodd.typepad.com  

Twitter: @SteveTodd

What a Difference a Decade Makes

All of this talk about open letters brings back memories.

This coming August will mark ten years to the day that EMC announced its plan to acquire Data General (DG). EMC recognized that the market for mid-range storage was going to be huge and the fastest way to go after that market would be to acquire the CLARiiON technology. In hindsight it was a great move that has probably raked in greater than $10 billion in the past decade.

As an "acquired employee" there were two aspects to the acquisition that I thought were important:

1. The logistics of integrating me as an employee
2. The logistics of integrating the CLARiiON product

Integrating Employees

The  EMC executive responsible for integrating Data General was Frank Hauck. His message was clear: the integration of DG employees will happen like clockwork. It will occur quickly and non-disruptively, and EMC will immediately qualify CLARiiON DGers for bonuses and stock to motivate the execution of the CLARiiON roadmap.

And it did happen like clockwork; it exceeded my expectations. As a 14 year DG employee it was refreshing to have new management that communicated the plan clearly and then executed the plan.

This seamless acquisition of employees, accompanied by perks that were INFINITELY more generous than us DGers were used to, played a large role in the explosive growth CLARiiON would achieve.

Integrating the Product

Once the acquisition was finalized, there was a 6-month "moratorium" on transfers to or from Data General engineering groups. EMC did not want any disruption of the momentum that CLARiiON was building.

I understand the rationale but in hindsight it was like building a wall around CLARiiON.  Because once the six-month moratorium was over, the acquisition was no longer a front-burner topic, and the discussions of high-level "architectural integration" of the product line was pursued with less aggression than the "employee integration" process.

The Flood of Acquisitions

It's my impression that subsequent acquisitions experienced a similar pattern. Acquired technologies such as Centera, Documentum, RSA, etc., underwent fairly seamless employee transitions, which enabled successful growth rates as part of EMC.  After forty or so acquisitions, EMC has the employee reward and transition thing down pat.

Discussions about overarching product integration lagged behind. Certainly there were obvious touch points that came together (integrating PowerPath with RSA encryption keys, for example). However, newly acquired engineers (especially top architects or technology experts) did not have a forum for discussing product and architecture integration. It usually happened at a grass-roots level instead.

This has changed. EMC recognized that there was an issue of technical unity within the company. For the past few years I have noticed some fairly impactful initiatives being rolled out:

• Global Innovation Network: EMC assigned an "acquired employee" (Dr. Burt Kaliski from EMC) to "gather" the pockets of innovation occuring across all of the acquired companies and create a loosely-coupled forum where technical advances are monitored and cross-pollinated. This effort also involved listing all of the outstanding research relationships with academic institutions.
• EMC Innovation Lecture Series: a live, monthly discussion of technology topics that also draw from employees that have been acquired into the system.
• Monthly CTO Council: Once a month EMC's Office of the CTO (Jeff Nick) runs a 2-day summit which unites the employees from acquired companies and provides a forum for architectural discussion and product innovation. Acquired companies usually show up immediately and are asked to "open the kimono".
• Innovation Showcase: a yearly event exclusively focused on technology unity and innovation across business units. Level playing field for any employee in any business unit to submit a killer idea and potentially be trotted out in front of EMC executives.
• EMC|ONE: EMC's internal social media site is pushed heavily as a method of uniting all employees (not just engineers). It can actually be used as a "research portal" to learn about new products (e.g. Symmetrix V-Max), new initiatives (e.g. VMWARE/Cisco/EMC), and most importantly it "puts a name to the product" in order to reach across business units and learn how something really works.
• EMC Ed Services: when an acquired company becomes part of EMC, it is AMAZING how quickly computer-based training for that product becomes available (followed closely by live training courses). Whenever I want to learn about an acquired company I'm about 5 mouse clicks away from watching a training course on the topic.
• EMC's External Blogging Community: this community is perhaps one of the fastest methods of finding something out within EMC. There's not one external blogger who doesn't instantly respond to a call for help. Those who have chosen to blog externally (like myself) have naturally formed huge amounts of internal connections.

None of this was in place when I joined EMC. What a difference a decade makes. Nowadays, acquired employees have that "cool new toy" that everyone wants to hear about.

Does it sound too good to be true? The simple fact is that I use these tools all of the time. They are there for everybody.

And this is the point of my post. Sure, when EMC acquires a company's technology the new product typically experiences explosive growth. But would a given employee from a company acquired by EMC fit in? Well, that would depend on whether or not that given employee possesses one critical characteristic.

Initiative.

Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd

Lunch With Darcy

Just had lunch with Jeff Darcy who is back on the street looking for work. If you're looking for a guy who can come up with a solid software architecture (especially when it comes to storage-related architectures) he's a great choice. He made the interesting comment that when it comes to cloud-based efforts, there seem to be a lot-more cloud-startup companies on the West Coast USA, while here on the East Coast cloud innovation tends to be happening at more established companies.

He also gave me a zinger during lunch when he said "Steve, you need to post more on your blog". He didn't want to hear any of my whining about "crunch time" at work. Note to Jeff's potential employers: "He tells the truth no matter who is in the room". LOL!

I thought I would hustle back to work and try and crank out a post before Jeff got back to his RSS reader.

Blogging/Innovating During Deadlines

My argument to Jeff about letting blog posts slide as a result of engineering deadlines is a bit hypocritical. One of the efforts I've been pushing within my own team is the theory that all of us should be carving out times during our day to see the forest through the trees. EMC invites all employees to do this as part of its global Innovation Contest each year (the 2009 contest started last week by the way).

We've had some pretty big software deadlines recently. Simultaneously,  we've had some requests to begin planning next-generation products. The next-generation planning process usually involves carving out some time from the key architects within the group (maybe 2-4 people), while the rest of the masses stay in the trenches debugging and adding quality to the product.

The architects are traditionally given permission to innovate during the deadlines.

This can be a bummer on several levels. Firstly, a large percent of the team don't get to do "the fun stuff". Secondly, the people in the trenches during a deadline are often at alpha and beta sites gathering key input firsthand. This type of first-hand exposure is critical in the planning of future products.  And finally, the people in the trenches feel the most pain when it comes to development processes. Trench players are the perfect people to answer the question: "Is there a way to increase development speed and quality?"  Process ideas are often the most powerful.

Enter Innovation By Contest

The innovation-by-contest paradigm that EMC has been espousing for the last several years is starting to occur at the individual business unit level as well. My group is currently right in the middle of its own idea contest, with a focus on ideas for the next generation product. If anyone enters an idea it becomes part of a "level playing field" of ideas.

The mechanics of this type of approach are straightforward: every idea has the same visibility, co-workers can rank and comment on the ideas (stimulating collaboration), and managers can engage their employees by officially assigning their ideas as MBOs or goals for the rest of the year.

Perhaps the most important employee opportunity is the chance for increased recognition. This morning, three of my co-workers from Russia presented further detail on their ideas to the VP of my business unit and his staff. 

Innovation contests are low-touch and high-value. They are low-touch because it only takes 5 minutes to enter the idea into the system, and maybe 15 minutes to browse the ideas during the day. In the grand scheme of things this is negligible and does not affect deadlines. They are high-value because of the morale impact: management is presenting an opportunity for anyone to participate in the beginning stages of development (where innovation can often occur).  And most importantly, the ideas that are coming in are really, really good. I've been seeing about a dozen new ideas per day.

This is why I believe that innovation-by-contest is a most effective organizational innovation tool. Contests, and the ideas that come with them, are refreshing.

And there's nothing wrong with a little refresher during a deadline.

Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd

Cuban Missile Crisis: EMC Innovation Lecture Series 6/3

Tomorrow, June 3rd, at 2:00 PM EST, I'll be giving a talk about digital curation and archiving as part of EMC's Innovation Lecture Series. You're invited.

The goal of my talk is to cover research directions within EMC when it comes to XAM technology and digital curation. I briefly covered the agenda in my last post, and gave an example of a cool historical document that has already been scanned into the archive: President Kennedy's 1962 speech at Rice University regarding the United States space program.

I've received another document from the archivists at the museum that also has great historical significance. Below is a scanned portion of the speech given by President Kennedy just one month after the speech given at Rice:

This speech was given to the American public from the White House as part of what is now known as the Cuban missile crisis.

What's interesting about this speech is that the original document from the White House does NOT use the term "Cuban Missile Crisis". This label was applied after the speech. Indeed, the three countries involved (the US, Russia, and Cuba) all gave the crisis a different name. If this document is simply scanned into an archive it will not be found by a search unless a human somehow attaches metadata to the image.

The "somehow" of attaching metadata to an image is a big part of my talk tomorrow.

In addition we're hoping that Stuart Madnick from MIT will be available to participate as part of the session. Professor Madnick and MIT are proposing an industry-wide collaboration to solve the problem of uniting distributed, heterogeneous, scientific digital archives. This effort is known as the DataSpace initiative.

Please join us, registration is open to all and can be found here.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd