A 2012 white paper by ABB Data Centers reported that 95% of data center operators surveyed experienced an unplanned power outage in the previous 24 months, and that around 80% of them could have been prevented.
In other words, power outages happen to just about everyone, they cause major problems, and you need to be prepared for them.
Total outages averaged 107 minutes in duration, while partial outages lasted an average of 152 minutes. Power outages result in:
- Lost revenue
- Disruption of business
- Decreased productivity
- Possible legal ramifications
Studies show that running around in circles waving your arms in the air actually does very little to address problems related to power outages.
When you create your organization’s IT disaster recovery plan, know your power generation assets, like commercial power and backup power systems. Understand what power protection you have, including grounding, lightning arrestors, line conditioners and surge suppressors.
Warning Before Unavoidable Power Outages
With some power outages, you have no control and no warning. For example, power outages in 2011 covered an area from southern California to Arizona, affecting over seven million people. The 2012 Superstorm Sandy is still on many minds in the northeastern US, even though meteorologists gave plenty of warning. Weather-related power outages may be unavoidable, but they often give you time to prepare.
Bad weather usually gives you some warning so you can get ready. Most of the time you’re not that “lucky.”
However, even when you have no warning of power outage, should to understand how power outages affect your network, and what you can and cannot do about them.
Power Outages May Not Be Obvious at First
You may not be aware of a power outage on your IT network right away. Maybe a printer goes offline. Or maybe the central server farms stop receiving information from outlying nodes. An end user at one of these outlying nodes may be suddenly cut off from the data and software they were accessing via the server farm at the center of the network. In other words, a power outage may not be as obvious as when you’re home during an ice storm and every electrical appliance screeches to a halt all at once.
When the power goes out, some parts of a network conk out sooner than others. Chips that require a constant power supply fail immediately. Hard drives may last longer, but that’s not necessarily good, because files can still be corrupted. Additionally, the power surge resulting from an outage can cause older nodes in the network to fail altogether.
Uninterruptible Power Supplies
One defense is equipping nodes with uninterruptible power supplies (UPSs), so that devices can shut down the way they are programmed to. Some corruption should still be expected, but you may be able to limit it.
Select a UPS that has its own communications device so that it acts as a node in the network that will warn you if the power goes out. That way, you can make a quicker assessment of what the damage is, what still works properly, and what may need to be repaired or replaced. This helps you achieve your goal of minimizing data loss after a power outage.
You don’t have to connect everything to your UPS. For example, a monitor consumes power, but it doesn’t play a role in protecting your data. Equipment that may restart on battery power should be configured to remain shut down in order to minimize potential damage.
Also, when you select a UPS, know how much power the servers use. It may be less than you think. There are websites that will tell you how much power you need once you input the hardware specs of your servers. You could also use a power meter to determine this, and it’s a good idea to have a power meter to help you assess the damage in the aftermath of a power outage.
If you have solid documentation of your organization’s power system and have a disaster recovery plan for power outages, you’ll waste less time when a power outage hits. Furthermore, once the power is back on and you’ve recovered as much as possible, you’ll have an easier time figuring out what you did well, and what you could have done better. After an outage and recovery, you may want to forget it ever happened, but documenting lessons learned is the best way to improve your power outage response so that hopefully the next one will be less destructive.
Samanage is SaaS IT service management software, so as soon as your network is back online, your IT service desk and IT asset management programs are ready to go. It’s one less thing you’ll have to worry about when the lights go out and you’re frantically trying to contain the damage.