Information Playground

Phase 2 Innovation Analytics: Data Quality

I am blogging my way through the Data Analytics Lifecycle as taught in EMC's Data Science and Big Data Analytics course.  I am running a Data Analytics project that employs a team of volunteer data scientists from around the world, and I have communicated an analytic plan to them (along with a set of hypotheses). The entry into Phase 2 of the process (typically the longest phase) has resulted in preparing the data, loading it (without transform), and exploring it.

At this point it is worth mentioning a quote that I heard during the course:

If you do not have data of sufficient quality or cannot get good data, you will not be able to perform the subsequent steps in the lifecycle process.

How Clean is the Data?

One of the data scientists on my team was evaluating a tool called Tableau, which can be used for data exploration (among other things). They began to use the tool and explore the data that had been previously loaded into the analytics sandbox. They sent me the following screenshot (I zoomed in and circled my name):

I am showing up twice in the database because some entries have a space before my first name. This is a classic problem (and not always easy to fix).  Addressing this problem within the sandbox is clearly a much easier proposition than doing so in the production database. However, it could take a long time to get it right (another reason why phase 2 takes so long). 

Who typically does this work?  Is it a database admin (DBA)? A data engineer? Both typically play a role in Phase 2. The "Database Administrator" provisions and configures the database environment to support the analytical needs of the working team. The “Data Engineer” tends to have deep technical skills to assist with tuning SQL queries for data management and extraction. They also support data ingest to the analytic sandbox.  These people can be one in the same, but many times the data engineer is an expert onqueries and data manipulation (and not necessarily analytics as such).  The DBA may be good at this too, but many times they may simply be someone who is primarily skilled at setting up and deploying a large database schema, or product, or stack.

In addition to mis-spelled names, the data scientists exploring the data are starting to uncover missing data that will help them prove the hypotheses. For example, consider one of the hypotheses generated in phase 1:

H5: Knowledge transfer activity can identify research-specific boundary spanners in disparate regions.

The association of boundary spanners to the geographic location where they work requires that any names found in the sandbox (e.g. "Todd,Steve") have an associated location (e.g. Hopkinton, MA).

Our data scientists found that this data was nowhere to be found within the sandbox.

In addition to DBAs and Data Engineers, IT often plays a large role in Phase 2.  For our project, once the names were "cleansed", we had to bring in IT resources to help generate geographic associations via our employee database. In our particular case we were fortunate: not only did IT grant access to our request, but the IT resource had Data Engineering skills and cleansed the data for us! In general, bringing in additional data from the IT realm is no easy task. Access to these types of assets is typically a very tough, time-consuming part of Phase 2.

I could write paragraph upon paragraph describing issues that we've come across (and solved) for Phase 2. It may be more useful, however, to summarize some of the lecture material that describes common problems:

• Consistency of data types (e.g. confirm that all numeric types contain numeric fields)
• Data feeds can often change over time (e.g. someone removes a column without telling anyone)
• Fields that contain calculations (e.g. interest charges) may change over time (if interest rates change over time)
• What are the legal ranges of data and are there any values that are out of bounds?
• Is the data standardized/normalized? If so, what is the scale?
• Are geospatial data sets consistent (e.g. metric versus english units, two-letter state abbreviations versus full-names)?

During this phase the data scientist may discern what to keep and what to discard. They had probably formed an opinion of what model they will use. Data exploration and cleansing has either validated their assumptions or caused them to select a different model. Data cleansing is a big job, so the objective should be to determine "what is enough?". What is clean enough data? What is sufficient quality for the operating context? What will properly enable the analysis?  These questions give people boundaries for the data cleaning, which is quite intensive.

Phase 3 is about model planning. How does one know when they are ready to leave phase 2 and move on to phase 3 (keep in mind that a return to Phase 2 is highly likely!)?

In general, Phase 3 begins when the data quality is "good enough" to start building the model. In my case, once we had cleaned up erroneous names and associated the names with geographies, the team had sufficient reason to enter Phase 3.

I will relate our team experience with Phase 3 in future posts.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

Director, EMC Innovation Network

Phase 2 Innovation Analytics: Exploring the Data

At this point in the series of articles on the Data Analytic Lifecycle, raw data has been identified and imported into a Data Analytics sandbox. The data (a mix of structured and unstructured data) is depicted below. Contained within the sandbox now lies a large amount of data representing research and innovation activities occurring globally throughout my corporation (EMC).

At this point in the lifecycle it is recommended that the data scientists and engineers begin to “get used to the data”. This means that they can use any number of tools to inspect the format, structure, and quality of the data. They have already immersed themselves into the analytic plan generated in Phase 1, and they are likely looking for data sources to validate the hypotheses in the analytic plan.

This process will almost certainly identify “gaps” in the data that will prevent the data scientists from being able to prove or disprove the hypotheses.

At my corporation we have created a volunteer team of data scientists that ensure that our approach scales globally.  One of these data scientists is Vladimir Suvorov of Saint Petersburg, Russia. Vladimir described the tools and approach he used to explore the data:

R-studio provides the easiest way for initially examining the data placed within a sandbox. R-studio provides a simple connection string and SQL query on one side and a powerful statistics and graphing package on the other. Besides that, R is open-source software, so it can present additional value for the community.  I recommend R for fast prototyping and quick summaries of the data.

Vladimir created a chart that focuses solely on university activities. This chart uses color coding techniques to categorize the types of university activities that have occurred in previous months. What is actually happening here is broadly considered "data exploration".  In this phase, people like Vladimir explore the data, assess data quality, examine basic information about the data itself (relationships in the data, trends, etc.), and they try to understand what kind of data they actually have.  This lets them form ideas of how they can test in future phases, and what kinds of insights they may be able to drive towards.

Vladimir’s simple visual chart shows the pace of visits to universities (yellow), as well the emergence of a dark blue color, which represents meetings of the Research Advisory Team (RAT) for the purpose of discussing university research funding for 2012. These meetings continue throughout the first quarter of the current year (2012-03).

The relatively small amount of employee lectures at universities (red) and professor visits to EMC (aqua) is a potential indicator that programs could be put in place to accelerate these types of exchanges.

This chart falls under the category of “descriptive statistics”. 

It’s an important activity in Phase 2 which, as just explained, can already lead to actionable conclusions.

During the Data Science and Big Data course, I learned that bar charts like this, while helpful for the data exploration phase, may not be the best choice in future phases. They can actually be improved via a few simple tips. Stacked bar charts are good for showing a few aggregated data points and exploring the data points.  When looking at data over time (as above), it is generally better to display line charts, to make it easy to see trends (people generally have a more difficult time examining trends in stacked bar charts).  Likewise, best practice is to use very soft colors and strong, emphasis colors as a way of highlighting key points; red is also a signal for things like “danger” or problem areas, especially in certain world countries.  In summary, a graphic like this gives us good initial information at this stage in the process. For more polished visualizations (used later in the lifecycle), Data Scientists need to address these kinds of aesthetic considerations.

During the data exploration exercise the data scientists and engineers begin to notice that (a) certain data needs conditioning and/or normalization, and (b) there are data sets that cannot be found anywhere which are critical to proving analytic hypotheses.

These activities will be further profiled in the next post. Once again, thanks to David Dietrich, who not only taught the course I attended, but continues to oversee this series of posts.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

Director, EMC Innovation Network

 

Phase 2 Innovation Analytics: ELT

This series of articles describes an analytic lifecycle being used to gain insight into the innovation and research practices of a multi-national corporation (EMC). After creating an analytic plan in Phase 1, a previous post described the Data Preparation phase of the lifecycle. This phase involves the creation of an enormous sandbox (e.g. ten times the size of a data warehouse configuration).  Data scientists and engineers are encouraged to extract data from many sources and load it into the sandbox unchanged. This approach may seem a bit revolutionary (most processes transform the data first). This lifecycle, however, is geared towards the data scientist. The possession of the raw data allows for more robust analysis. The diagram below provides an overview of this approach.

As I mentioned in the last post, there are two types of data that will allow a data scientist to analyze innovation. The first type, depicted on the left, is a structured SQL database containing thousands of innovation ideas submitted by employees over a five year period. The second type of data consists of minutes and notes from global innovation and research activities. This content is highly unstructured.

It’s worth taking a moment to discuss how the global team of users and data scientists came up with relevant innovation activities (depicted below). EMC’s product line consists of high-tech products and services that have been introduced into the marketplace. Tracing the lineage of these products and services usually results from an idea that happened long ago during a specific activity. The team came up with a candidate list of activities that is often associated with innovation and research:

Visiting universities, creating publications, attending conferences, visiting customers and partners, holding internal knowledge sessions, holding idea contests, and creating intellectual property are all activities commonly associated with innovation, and therefore an effort was made to gather six months worth of these activities from data sources worldwide.

After resisting the urge to transform these documents before loading them, a next logical step in Phase 2 is to explore the data. I will step through an example of this process in my next post.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

Director, EMC Innovation Network