Len's News, Notes, and Stuff

The Digital Civilization is Built on Sand – and It Won’t Take A Catastrophe to Realize That

We are building a digital civilization on sand.  In fact, we don’t need anything as drastic as a worldwide catastrophe to realize this.  On an individual level, simply consider the data you may have in old file formats.  On an organizational level, consider how you might restore data off your backup tapes when you need that data most.  On a societal level, consider how much data you could pull off the Internet without a solid telecom and electrical infrastructure.

Digital content is easily transported, free to replicate, and can be replayed back in a myriad of forms.  Unfortunately, both the media upon which the data is stored and the file formats used are short-lived beasts that will prove to be insanely difficult obstacles to surmount in the future.  “The future” doesn’t even have to be very far away.  5 years or 10 years down the road and you’ll have serious problems.

Anyone over the age of 20 likely remembers magnetic media in the form of floppy disks.  Maybe you remember the not-so-floppy 3.5″ floppies (remember those puppies?  1.44MB on a high-den, 720kb on a double-den).  Maybe you even remember the really-floppy 5.25″ disks (1.2MB on a high-den, 360kb on a double-den, if I remember correctly!).  If you have data stored on those things right now, how much of a hassle would it be to try to get that data out of there?  And how much luck would you have in opening an old Wordperfect document, Harvard Graphics presentation, Quattro Pro spreadsheet, or running some of those classic DOS programs that weren’t built for systems with more than 1024kb of RAM?

Even if you have data stored on media that you can still read, there’s the problem of actually accessing it.  When I was working for a different provincial ministry, I asked our key IT contact about the process involved with restoring critical information back to our file servers if they were ever damaged or lost.  He told me, no problems, we have the data on backup tapes, cycled at varied intervals, all protected off-site, in a vault, or wherever.  Unfortunately, in order to restore the image backups, you had to have a new server that was the exact same configuration as the one that you’ve lost.  That is the big challenge – how do you get the exact same server as the one you lost if the one you lost is 1, 2, or 3 years old?  Technology moves on.  Vendors generally don’t offer systems and configurations that they did 1, 2, or 3 years ago.

His answer was that unless we got the exact same configuration, we would not be able to restore data off our tapes.  Simple as that.  How long would that take?  Anyone’s guess.  If you had the system on-hand, it would take a few hours to restore the data.  But if you didn’t, it could take a week, two weeks, or even a month or more to get the right machine delivered.  That’s unacceptable for critical data that you must have on-hand for business continuity purposes.

How much worse would it be for an archivist or any sort of researcher trying to get the data off that tape 10, 15, 25, 50, or 100 years from now?   They probably would never be able to get the records and information off there.  It would, for all intents and purposes, be absolutely lost to us, even if the physical media itself survived.

The move from physical records to digital records is one of the big challenges and headaches that archivists around the world are wrestling with.  There’s no going back to a world that relies on paper and physical documentation.  But key, critical data and information needs to be stored and protected in physical form.  There simply is no choice in the matter.

There are business continuity reasons for this in the short-term.  There are historical archival reasons for this in the long-term.

Etchings on stone slabs will survive for thousands of years.  Printed material will survive for centuries.  Digital data may only survive for a handful of years.

If everything goes right, digital data is magnificent.  But things will go wrong, somewhere down the road.  You might need to access information stored in a digital file that you can’t open anymore.  Or maybe you don’t even have the hardware to read the media.  Or, perhaps someone millennia later is trying to learn about our age or about something that happens in our age.

The article that I linked to above talks about how our knowledge legacy will really only be what we have on printed media.  Digital content will all be lost in due time.  I agree with that view, and I just wanted to point out that there are also near-term, dollars-and-cents reasons for worrying about putting all your eggs in the digital basket.