According to SiliconValley.com, a technician reformatting a disk drive at the Alaska Department of Revenue accidentally deleted information about a $38-billion oil-funded state account that distributes payments to residents. The technician then mistakenly reformatted the backup hard drive, too. Then, the Department discovered that its backup tapes were unreadable. Alaska officials pressed consultants from Dell Inc. and Microsoft Corp. into service, but none were able to restore either drive or to read the backup tapes. So the department had to bring in the 300 boxes of paperwork that had been captured and stored electronically, and press a half-dozen seasonal workers and 70 regular employees into overtime and weekend service to get the data back into a computer-usable form. At a cost of more than $200,000.
Now, I'm not in or near Alaska, but my tiny mind says that if Dell and Microsoft were involved, the system being worked on – or worked over, as it turned out – was probably based on standard PC-type architecture and components. So one of the critical BPM-related lessons to be learned from this debacle is to test backups regularly, and to revise the processes by which they are made and maintained whenever any fail their tests.
Another BPM-related lesson: implement protections equal to the criticality of the resource being protected. This means, for example, that it should be nearly impossible for any one technician to have simultaneous access to a primary hard drive and its only backup. It also means that the Alaska Department of Revenue, along with any other organization, might consider equipping highly critical systems with outboard backup hard drives such as those from Iomega Corp., Imation Corp., LaCie, and other manufacturers, or to connect them to automated online backup services such as Backup.com (and NOT AT&T Online Vault, as discussed here previously). Either or both has got to cost less than $200,000 – or whatever reputational damage is caused by such a disaster if it hits your enterprise…
Michael Dortch has been an analyst, 'information entrepreneur,' speaker, and writer about IT and 'the real world" for the past 30 years. After almost 10 years at the Robert Frances Group (RFG), Michael begins a new position with Aberdeen Group in September.
Dennis Byron is an analyst with ebizQ, focusing on Open Source Software as well as Business Process Management technologies.
