Critical Computer Systems Held Hostage
A recent incident that crippled the City of San Francisco teaches a lesson for small businesses
Recently, a network administrator employed by the City of San Francisco locked down the City’s computer network. By keeping a single password secret, Terry Childs (no relation to me!) denied access to IT administrators, thereby crippling important municipal functions, such as the City’s payroll and law enforcement records. After spending several days in jail and meeting in secret with San Francisco Mayor Gavin Newsom, he gave up the password and power has been restored. Subsequent investigation revealed that Mr. Childs was a disgruntled employee with a criminal record; he had been arrested for aggravated robbery 25 years ago in Kansas.
As we had advised in the first edition of our book, “A good network administrator builds his or her reputation on trust that has been earned throughout their careers. But even with the best service administrator in your service, you must still protect against the risk of internal sabotage. These measures are not difficult to implement and should be welcomed by your network administrator as being in the best interest of the organization.” How can your small business avoid San Francisco’s experience of being held hostage by a disgruntled network administrator?
1. Apply basic auditing methods. There are simple auditing methods that you can apply and review periodically, such as identifying who accessed which files, who generated which external network traffic and who sent a large number of e-mails or attachments to which addressee. You should, of course, inform your staff that activities on the IT network are monitored and the results of these activities are not matched with personal information unless there is a compelling reason to do so. Ask staff to refrain from storing personal information on company computers. These guidelines should be formalized in company policy.
2. Automate independent backups. It is critical to back up your business data and certainly your network administrator needs to have access to the backups in the event it becomes necessary to retrieve data in the course of ordinary business or emergency. But always have one backup mirrored on a site to which the administrator does not have access. There are tools that can do this automatically at designated times during the day. This mitigates your risk of sabotage. If the City of San Francisco had such a system in place, the Mayor would not have been compelled to visit a saboteur in his jail cell.
3. Outsource your e-mail service to a third-party provider. I always advise small businesses to outsource their e-mail service to a third-party provider, as it is generally not cost-effective for them to manage these services in-house. This approach offers an additional benefit: it makes the e-mail system independent of internal systems staff, both reducing their work burdens and the opportunities for internal sabotage.
4. Do not use any built-in “Administrator” accounts, but instead give two users administrative rights on the system. This way, each week those two people can independently monitor and audit suspicious activities on your network and system administrator tasks can be traced to their user identifications.
One of the key messages of Prepare for the Worst, Plan for the Best: Disaster Preparedness and Recovery for Small Businesses (Wiley, second edition, 2008) is that by preparing for the everyday disaster, you automatically build resilience for the more serious threat. No one wants to think about internal sabotage; it is deeply upsetting to imagine that your trust could be betrayed in such a manner. Thankfully, few of us will have to deal with this possibility. But what if San Francisco’s IT administrator had suffered an accident or a medical emergency (a statistically more likely outcome than the perpetration of sabotage)? The City’s IT systems would still be brought to a stand-still, without the solution of a jailhouse visit by the Mayor to retrieve the password. Restricting access to a single individual, no matter how apparently trustworthy, is not a good policy. Let your small business learn from the experience of San Francisco.