Information Playground

Vatican Preservation

One of my favorite technology initiatives at EMC is the digital preservation project at the John F. Kennedy Library and Museum. The project is cool on a number of levels. It has a state-of-the-art digital archive  assembled with a wide variety of EMC products. It is locally relevant for me, given that I grew up in Massachusetts. And of course it has national (and international) significance for those researchers that want online access to JFK material.

In November 2012 the EMC TV team headed to Rome to document the newest Information Heritage preservation project. 

The Bibliotheca Apostolica (as the Vatican Library is known) is one of the oldest libraries in the world. It contains more than 80,000 historic books, documents, papyrus texts and other ancient manuscripts.  In order to further preserve the manuscripts and make them available to scholars and historians around the world, the Vatican is launching a multi-year project to digitize, store, archive and make the entire collection available on line.

As with the JFK Library, the archive will be assembled with a variety of EMC products, including VNX, Isilon, Atmos, Networker, and Data Domain.   EMC consultants, in partnership with systems integrator Dedagroup, are working with the curators at the Vatican on designing a 2.8 petabyte system. It is estimated that a system of this size should be capable of preserving approximately 40 million documents. 

The video below was the result of EMC TV's unprecedented access to the Vatican library, reading rooms, archives, and preservation rooms. The footage includes some rare books and manuscripts that are not typically accessible to the public. During the video you will notice that a small camera was mounted to the scanning machine. The digitization process used by the technicians was recorded as well. Appearing on screen is Timothy Janz,  Scriptor Graecus of the Vatican, and Luciano Ammenti, Coordinator of the data center for the Vatican Library.

 

As with the JFK Library, I am hoping to track the progress of this initiative over the next few years and document the best practices that continue to emerge as digital preservation technology advances.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Fellow

 

 

Boolean Preservation at UCC

Campus at University College Cork

I've written before about the EMC Heritage Trust project, which provides grants to various institutions that wish to engage in digital preservation of locally valuable artifacts (see the list of 2012 grant recipients here).

In 2011 the Boole Library at University College Cork received a grant. This past week I just so happened to stop by the UCC campus for a research visit, and I was given an update on the state of the project (the scanned library is in the process of being shared via the web).

The best part of the update, however, included the invitation to walk down into the heart of the library and view the Boole papers themselves. The mathematical notes, textbooks, and quizzes that I saw were quite spectacular, but more impressive still were the handwritten notes to family members that described life in Cork at the tail end of the Great Irish Famine.

Out of all the Boole material that we were allowed to see, my favorite was the Laws of Thought, and I was allowed to photograph the title page of the book, which contained many of his handwritten notes:

As a software engineer that has spent a lot of his career leveraging the Boolean datatype, the visit to the library was certainly a great highlight of my research trip to Ireland (more on the research aspects of this trip in future posts).

George Boole had a rather binary view of the world: it all boils down to true or false. His work greatly influenced the digital explosion we're going through today.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

Director, EMC Innovation Network

University College Cork

Captiva Speed at JFK

I was perusing the Captiva announcement today and a quote from one of the leading archivists at JFK caught my eye:

Erica C. Boudreau, Archivist for the John F. Kennedy Presidential Library in Boston, said, “This new version of Captiva enables us to compress our scanned images using a lossless method, saving storage space and costs while maintaining high quality images for the historical documents that the JFK Library is capturing. The ability to quickly manipulate images and to process them using Captiva’s latest optical character recognition (OCR) engine produces more accurate results and saves our staff significant time.”

Several weeks ago some of my co-workers and I visited Erica's team at the JFK Library and Museum to ask some questions about their digital archive, find out what was working well, and ask what we could do better. The archive holds a massive amount of scanned documents from JFK's presidency. A key part of their ingest strategy relies on Captiva's automated ability to quickly scan documents and (a) immediately convert them to multiple image formats, and (b) perform automated optical character recognition (OCR).

The picture below highlights the steps that are taken before the multiple images are handed to the back-end content management system.

During our visit the JFK archivists were asked when they would be "finished" scanning documents. The answer, based on their current process, was measured in decades (well over a hundred years)! With only two scanners, there are only so many documents that can be pushed through the system.

Their process was greatly accelerated, however, when they upgraded their Captiva software and leveraged the ability to generate multiple image sizes on the fly. These images are then sent to the back-end system (Documentum) along with the OCR metadata. From that point on the images and meta-data can be accessed using Documentum's Digital Asset Manager. The image below shows has DAM can be leveraged to manage and display these images. These images are photographs (not scanned documents), but they highlight the benefit of a Captiva integration:

In order to learn more about how Captiva transforms scanned documents so quickly I read this benchmarking report by Wipro (registration required). The report is a fair comparison of Captiva's capabilities versus the competition. For some use cases Captiva comes out on top, and in other cases they don't. If you look at the performance comparisons, however, there is no contest. Captiva is built for speed.

When considering that over 200,000 documents have been scanned into the archive to date, performance truly counts.

Steve

Information Playground

Twitter: @SteveTodd

EMC Intrapreneur

JFK Project Going Live

It's been nearly four and a half years since the joint announcement between EMC and the John F. Kennedy Library and Museum.

There has been a flurry of activity at the Library recently.

I attended a celebration of the 50th anniversary of John Kennedy's election victory. The evening was a simulation and re-creation of the events from 1960, complete with Walter Cronkite broadcasts and predictions from IBM computers that Richard Nixon would win.

The Library also announced that on January 13, 2011, a redesigned website will be unveiled (thanks to Gil Press for pointing out the Boston Globe article). The website (www.jfklibrary.org) will in part be the result of the multi-year digital archiving project that I've written about over the past few years.

Many of the original articles I wrote are slightly out-of-date because of a technology refresh. The refresh falls into these categories:

• The CX300 has been replaced with an NS-120.
• The NS-120 will hold another five years worth of digital artifacts, and it also satisfies the video/audio streaming capabilities required to access certain content.
• The Library upgraded their Documentum configuration to Digital Asset Manager. This platform will allow for the addition of historical metadata to each artifact. This metadata allows JFK Archivists to add historical relevance and interpretation to the collection.
• A second Centera has been installed at a disaster recovery site at Iron Mountain in Pennsylvania (the previous configuration used Networker to back up to tape).

The disaster recovery strategy is a reflection of the enormous amount of preservation that has been going on inside the museum over the last few years. For example, this quote was taken from the  announcement of the January 13 website launch:

This unprecedented initiative, which has begun with the challenge of scanning, digitizing, and describing more than 250,000 documents, 35,000 photos, 250 audio reels and 72 reels of film footage has been underway for more than four years by the staff of the Kennedy Library’s Archives.

All of this footage will be online and available for JFK scholars and researchers around the world.

In addition to the digital archiving website, a separate website will be launched a week later: www.JFK50.org. This teaser on the JFK50 website gives a great look at the type of footage and history that will be available via the digital archive.

Steve

Information Playground

Twitter: @SteveTodd

EMC Intrapreneur

 

 

Put Me In Coach

I've seen a lot of EMC press releases in my career, but this announcement from September 21, 2010 is at the top of my list. EMC is sponsoring the digitization of assets from the National Baseball Hall of Fame and Museum (and helping to sponsor the museum's education initiatives as well).

I'm a huge baseball fan. I called the museum (located in Cooperstown, New York) and let them know that I've been assigned to publicly document the relationship, technology, and archival strategy that will develop between the museum and EMC. Who gave me the assignment? 

I did!

When I spoke with Ken Meifert at the museum, he told me that it would be best if I visited in person, but not right away, because he had to attend Game 1 of the World Series in San Francisco (I would have been perfectly willing to go with him to get a deeper understanding of the museum's mission and purpose).

So I waited a few weeks before taking my wife and son with me for a visit. Here's what I learned.

The Big Picture

Most people recognize Cooperstown for the Hall of Fame Induction Ceremony that occurs each year. Indeed, the public area inside the museum dedicated to the Hall of Fame is measured at over 4,000 square feet.

The total square footage of the public museum exceeds 55,000 square feet, so it is clear that the recognition of baseball greats represents only a portion of the overall mission, which is described as follows:

Preserving History. Honoring Excellence. Connecting Generations.

The game of baseball and the history of America are deeply intertwined. This helps to explain the selection criteria for accepting new artifacts into the museum. They are looking specifically for contributions that offer unique and valuable insights into American history.  For example, chief librarian Jim Gates explained to me that they recently received a slide rule that had at one time been used to help calculate baseball statistics. This type of donation has more value than autographed pictures of famous baseball players, because it sheds greater insight into the American past (so send all of your autographed pictures of Ty Cobb to me!).

The library is another important facet of the museum. It contains millions of documents, over half a million photographs, and thousands of hours of recorded film, video, and sound. According to their website they receive over 60,000 research inquiries per year! Jim was kind enough to display some of the more interesting artifacts for us. The picture below shows some of them.

 The table contains the following:

• The large book represents hand-entered statistics for every player in the National League during 1947. If you want a closer look at the page from this book you can click here. Starting on April 15, 1947, this page represents the statistics of a rookie baseball player named Jack Robinson.
• The promissory note for the sale of Babe Ruth from the Red Sox to the Yankees.
• A base ball rule book from 1856. Each "club" had their own rule book, and when they met to play they had to decide on the rules for that particular game. In reality this practice in still in effect today, given that each league has different rules (e.g. designated hitter), and each team has different rules for their home park (the ground rules).
• An 1867 book called "Base Ball as Viewed by a Muffin". A muffin was essentially a third-stringer who was on the team because each club needed members (and the dues that each member paid). Their skill level was lower than the first-stringers, and the baseball phrase "he muffed it" evolved from these individuals.

Between the contents in the library and the museum, there are an overwhelming number of options when it comes to the strategy for digital preservation of the assets. So what's the plan for the use of the EMC equipment and relationship?

A Two-Phased Approach

The National Baseball Hall of Fame and Museum already has their own IT department, and just like any IT department, they have to keep up with the pace of information growth. Phase 1 of the engagement will be targeted at improving the efficiency of the existing IT operations. I hope to document the steps that they are taking along the way.

The second phase will be the ongoing (and never-ending) digitization of museum and library assets. This of course is similar to the effort sponsored by EMC at the JFK Library. The first step in this phase is the creation of the business plan for exactly what gets archived, why it gets archived, and the ongoing cost associated with doing so. This business plan will drive to an implementation over the course of time.

I'm Ready to Play

When the Red Sox run out onto the field before each home game, they play the John Fogerty song "Centerfield". The song includes the familiar phrase: "Put me in coach, I'm ready to play". That's certainly how I feel about this project. The intersection between baseball and American history is fascinating and represents an extremely unique and fun way to learn. 

I think it would be helpful for me to move out there during the summer to closely track how things are moving along.

On a personal note, I decided to post some of the pictures that I took when my family and I received a personal tour of the museum archives. Enjoy!

Steve

Information Playground

Twitter: @SteveTodd

EMC Intrapreneur

 

Digital Difficulties

One of my EMC co-workers in China, Hang Guo, told me about a library in Beijing that is so amazing that he and his wife go there just to hang out on the weekends. I decided to look into this library myself and uncovered some pretty cool statistics.

The Peking University Library, founded in 1902, is the largest university library in all of Asia. I assume that the word "largest" (as taken from this web page) refers to the number of holdings contained within.  According to the university there are 6,500,000 million different artifacts, including rare books, periodicals, rubbings from inscriptions on bronze and stone tablets, and various historical journals. The library has also made great strides with digital assets as well. All in all it sounds like a great place to visit and a well-run, famous institution.

If a physical building can house millions of physical objects as part of a library collection, then housing hundreds of millions of digital objects in a digitized library shouldn't be a problem, right?  After all, the artifacts all have the same characteristics; they are a sequence of bits.

There are many articles like this that lament the fact that digital preservation is "too hard". One of the reasons seems to be the very fact that the artifacts all have the same characteristics! The main difference between the actual artifacts themselves is bit patterns and bit length!

Perhaps one of most concise problem summaries that I've come across is from the CASPAR project (Cultural, Artistic, and Scientific knowledge for Preservation, Access, and Retrieval).  A training lecture on the topic can be found here. I pulled a quote from the website which describes the problem well:

The peculiar characteristics of digital objects imply several difficulties in their preservation.

Even if it is obviously necessary to preserve the bits, this is not enough since more information, at many different levels (e.g. the format in which they are encoded, the semantics necessary for their intelligibility, the rights which are on them, etc.) is required in order to reach the ability to maintain and reuse these objects.

Moreover, digital preservation undergoes several threats which are related to financial and legal aspects, technological obsolescence, changes in environment and in the people knowledge base, trustworthiness of repositories.

Object-based storage can solve some (but not all) of these problems. The first decade of the new millennium saw the rise of object-based storage (there are now thousands of customer-purchased, object-based systems containing multiple petabytes of fixed content). These types of systems excel at the grouping of archived content and the metadata that helps preserve it.

I've long felt that XAM technology can lend a helping hand when it comes to digital preservation, and I've laid my reasoning out in a research paper. Indeed, research at Penn State is testing out XAM in the context of digital archiving. I also found it interesting that Dell threw their hat into the ring and announced XAM support for their new object-based system.

In 2000 there were zero commercially-available and well-known object-based storage systems. I'm certain that by 2020 we'll see an incredibly wide and diverse range of object-based storage, and it will be interesting to see if XAM has taken hold as the industry standard way to interoperate and preserve the world's digital information.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

NSF Meeting

Yesterday was a culmination of years of preparation when the National Science Foundation visited MIT. The NSF is a branch of the US CyberInfrastructure Department, and their team is in the final phases of evaluating grant requests as part of their DataNet initiative.

The NSF will grant five awards of $20 million apiece (spread over five years) for proposals that target the following vision:

Chapter 3 (Data, Data Analysis, and Visualization) of NSF’s Cyberinfrastructure Vision for 21^st Century Discovery presents a vision in which “science and engineering digital data are routinely deposited in well-documented form, are regularly and easily consulted and analyzed by specialists and non-specialists alike, are openly accessible while suitably protected, and are reliably preserved.” The goal of this solicitation is to catalyze the development of a system of science and engineering data collections that is open, extensible and evolvable.

Stuart Madnick and MacKenzie Smith of MIT have pulled together a proposal called DataSpace.  They've also gathered together a team of of participants (should the proposal be funded). More information about the motivation behind DataSpace can be found by reading Dr. Madnick's interview with On Magazine.

The full-day session was put together by Stuart and MacKenzie. Having never participated in a grant process before I found it quite interesting to experience the flow of the meeting. In general, most of the DataSpace presenters fell into the following categories:

• Scientists: the DataSpace project would initially target neuroscience and oceanography.
• Digital Curators: Curators from Oregon State, Georgia Tech, Rice, and MIT discussed issues and solutions for the management of digital scientific data.
• Industry: Google, HP, Microsoft, and EMC described technologies and research directions that are applicable to DataSpace.
• Software: several team members presented software technologies and open source tools that were applicable to DataSpace.
• Program management: MIT staff discussed how the project (and therefore the grant money) would be managed.

The meeting ran for about ten hours, and I spoke for 5 minutes. In general my presentation stated that EMC has a set of technologies that are applicable to DataSpace and could contribute people, equipment, and lab space to the effort. If you pick apart the NSF quote there are a set of technologies that map quite nicely:

• "science and engineering digital data are routinely deposited in well-documented form". The term "well-documented" implies metadata, and object-based storage systems would provide strong binding between scientific content and metadata.
• "are openly accessible". If object-based access methods are used, the XAM standard is an open standard for access.
• "while suitably protected". In addition to protection capabilities like parity, mirroring, backup, remote copy, etc., time-stamping and content addressing can provide extra levels of protection and authentication.
• "and are reliably preserved". Data preservation is a key part of XAM, given that it has built-in support for migration over multiple generations of technology. EMC is also doing research on data provenance and long-term preservation; these research directions have a strong overlap with DataSpace.

The presentation to the NSF had a great mix of vision, people and technologies. I'm pleased to have my name on the proposal. These technologies are right up my alley, and it would be a privilege to work with such a diverse and talented team.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Research Papers Moving to the Cloud

The ACM has been publishing scholarly works since 1954. They have been diligently maintaining an online library of research papers that has continually benefited the technical community. For years much of their budget has been spent on print-based services as well.

I read an announcement by the ACM last week that they have decided to scale back their investment in print-based services and focus instead on the long-term digital preservation of ACM content as part of a public cloud.

Public clouds that specifically target digital preservation have a different set of requirements than a public cloud like Amazon EC2, for example. The focus in a "preservation cloud" is longevity, and the administrators of said cloud must think like digital curators.

It's an interesting exercise to study the (a) system and (b) curator choices that ACM has chosen. The ACM has chosen to go with CLOCKSS and Portico, respectively. Both CLOCKSS and Portico are not-for-profit organizations.

CLOCKSS

LOCKSS stands for "Lots of Copies Keep Stuff Safe", and the 'C' stands for "controlled". The LOCKSS software framework was developed at Stanford University and is a peer-to-peer, de-centralized model.

The CLOCKSS initiative is run by "the world’s leading scholarly publishers and research libraries" with a goal of ensuring "the long-term survival of Web-based scholarly publications for the benefit of the greater global research community". CLOCKSS ingest boxes are located at Rice, Indiana, and Stanford university. As libraries and researchers submit content to these "ingest boxes", they are stored in normalized, maintainable format in triplicate across the sites. Once all of the ingest boxes have cross-audited the content, the artifacts are moved to "archive nodes" spread throughout the globe. These boxes continually audit themselves and verify the authenticity of the content, and create new versions when hardware fails (very similar to a RAIN architecture).

Interestingly enough, the solution is a "dark archive". The initial content is not accessible to the general public. The ingest and preservation within CLOCKSS is initially focussed on maintaining content for the long-term. When a "trigger event" occurs, however, content is made public by migrating it to the newest format and storing it on publicly available nodes at Stanford and the University of Edinburgh.

A description of how CLOCKSS works can be found here.

Portico

Portico can be thought of as a third-party partner that supplies the people and processes behind the CLOCKSS solution. The best way to explain the process they bring to the table can be found in one of their brochures:

Diagram from the Portico brochure

The CLOCKSS and Portico solution relies heavily on migration of data formats as the years roll by. I've always wondered about the feasibility of such an approach. Can it scale to billions of documents? Does the system have to continually upgrade all documents to newer file formats as these new formats become available? One thing I picked up as I learned about their solution is that the format conversion occurs when the format goes from "dark" to "publicly available". In other words, there's no need to continually upgrade file formats.

The alternate approach to format conversion is something I've been researching here at EMC. As an entire class of documents is imported into an archive, can a Virtual Machine also be imported that is able to "read" all of these documents, both now and in the future? This would require some time of virtual machine "player" that has infinite playback capability (no easy task).

I hope to learn more about the server/storage implementation of CLOCKSS at locations around the world.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Preserving Profiles in Courage

The digital preservation effort at the JFK Presidential Library and Museum keeps rolling on, one scanned document at a time.

All of President Kennedy's office files (POFs) have been scanned into the archive, and the digital curators at the museum are now focusing on digitally preserving President Kennedy's personal papers.

I have had the good fortune to personally see some of these documents, and for me the most interesting documents are those that contain the President's handwritten notes. In a previous post I wrote about his 1962 speech regarding the US Space Program, and how JFK scribbled notes on the first draft to highlight emphasis and relevance to the audience at Rice University.

Attached below is another example of a heavily edited document from President Kennedy's personal papers. This particular document is a draft of the book Profiles in Courage. The document, along with the handwritten notes, was probably typed and edited around 1953-54 (the book was published in 1955).

NOTE: The image above is PNG format; the curators at the JFK Library use TIFF. I converted to PNG because it's 30x smaller. The curators use TIFF format because they want the highest quality scan; every pencil scratch must be legible.

This is cool stuff. It highlights the importance of digital preservation efforts. As a software engineer working for a company (EMC) that builds information storage systems and workflow management software, it's also a great opportunity to try and build complete solutions that more fully address the needs of digital curators.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Intrapreneur

Passing Through the Dust of Swift-Tuttle

Stargazers will be looking heavenward tonight, but where's the data going?

 Tonight is the peak night for getting a great look at the Perseid meteor showers (if you happen to live in the Northern Hemisphere). The Earth is currently hurtling through the debris from the comet known as Swift-Tuttle. On the morning of August 12th we'll pass through the densest amount of this debris. These meteors are called "Perseids" because they radiate outward from the constellation known as Perseus. Here are some tips for viewing the Perseids. Lie down in any large, flat area, preferably one that is not paved, and/or without orange lines running down the center.

When it comes to the study of space phenomena, the amount of data generated and stored is, well, astronomical. I've often wondered how scientists configure their storage systems to handle this amount of data.

One group of scientists that is on the job capturing data about meteors are these guys from NASA's Meteoroid Environment Office. The MEO is the "NASA organization responsible for meteoroid environments pertaining to spacecraft engineering and operations". I grabbed this picture of them from their website and I'm glad that they are working hard and wearing ties.

I started to dig around for the purpose of understanding where and how the world is storing the scientific data about outer space. Part of my motivation is to understand more about the re-use of scientific data as part of the MIT DataSpace proposal. I happened upon an effort called the Virtual Meteor Observatory (VMO), which has a goal of "improved access to astronomical data and computing resources".

In December of last year there was a meeting of the International Space Science Institute (ISSI) to discuss the architecture of a VMO. The meeting was attended by a variety of meteor observer teams, such as the Polish Fireball Network, the University of Western Ontario in Canada, and the IAU Meteor Center. The full report from the meeting can be found here.

The VMO has agreed upon a standard XML format for the description of meteoric data, which I've cut directly from the report and pasted here (for what it's worth I was unaware that Heidi Klum was involved in this effort):

<?xml version="1.0" encoding="utf-8"?>
    <vmo xmlns="http://www.imo.net">
        <fireball>
            <time>2008-11-23T15:24:13</time>
            <brightness>as the full moon</brightness>
            <observer>Heidi Klum</observer>
            <location_latitude>
                35.24351
            </location latitude>
            <location_longitude>
                -89.62907
            </location longitude>
            <country code>US</country code>
            ...
        </fireball>
    </vmo>

With the agreed-upon metadata format, the team then determined five different types of "data formats" to be stored in the VMO repository:

• Visual meteor observations
• Video and still camera data
• Forward and backward scatter radio observations
• Fireball observations
• Orbit data

Existing scientific software that generates data and metadata must be converted into the VMO format, and ultimately everything is stored in a PostgreSQL database. Discussions are ongoing on whether  all scientific data is stored in one large database or if a distributed approach will be used.

Two thoughts that I had as I read through the information about this effort:

1. The researchers are involved with the submission and ingest of scientific data, as well as the dissemination of the data for scientific analysis. This workflow is very similar to the standard workflow described by the Open Archival Information System specification (depicted in Figure 4-1). The researchers could benefit from their awareness of this standard.
2. I was unable to find mention of the scalability limits of the system and requirements on the storage in general. Given that nearly all of the data being generated is fixed content (otherwise called reference information), then the XAM specification is a good candidate to abstract away the dependency on PostgreSQL.

Get out there tonight, find a safe piece of grass, and watch the show.

Steve
http://stevetodd.typepad.com
Twitter: @SteveTodd
EMC Intrapreneur