Going to depart from my original plan for entry three this week to talk about disaster recovery, since I experienced a hard drive failure on my work laptop on Wednesday and DR is an important aspect of data architecture, so it still fits thematically with the other entries.  First though, some context.  I would say that my organization has an unstructured data problem.  We have tons and tons of unclassified data sitting on end user hard drives, network file shares, and cloud storage environments.  Tons of it.  Duplicate data.  Incorrect data.  Corrupt data.  End users who bother to backup their personal data do it poorly: It’s not common for someone’s personal file share to be filled with manual redundant copies, e.g. Jan, Jan-Feb, Jan-Mar, Jan-Apr, etc etc  Some of them even encrypted their data, which sounds good on the face of it, but they didn’t use an appropriate managed encryption system, so not if but WHEN they lose/forget their password, nobody is able to unlock it for them.  For the rest of the user base, they don’t backup.

Nobody cares about backups, until they need them…then it’s always IT’s fault that they don’t exist and everyone is scrambling to and paying a lot of money to recover the data.  I think my favorite story was a sales guy who spilled wine on his laptop.  Hard drive was toast, but we have an agreement with data recovery vendors who can actually recover data from fairly destroyed drives, as long as we pay through the nose.  The sales guy insisted he had important data that needed recovery, so away the drive was sent.  When the ~$8,000 bill arrived, it prompted some questions:  he had to justify the expense.  Turns out, the data he was after was for his fantasy football team.  Whoops.

For the last several years, end user PCs have been backed up to the cloud (much like our servers).  It happens automatically, many times a day, and incrementally, only files that have been changed are backed up.   And that’s great for the end user, but all of these backups of (questionably useful) data take up bandwidth and bandwidth isn’t free.  In fact, due to the expenses of bandwidth plus the costs of the backup service itself increasing from $4 per user per month to $9.50 per user per month, the company made the decision to end the cloud backup offering.  At the same time, for data loss prevention issues, all writing to externally mounted volumes is now blocked as well.  The only means for end users to backup their data is by manually copying to internal cloud storage services designed for collaboration, not archival purposes.   This has generated a great deal of animosity towards IT, however it has saved quite literally millions of dollars, since we have ~300K employees globally.