By Gil Santaliz. Originally published by Disaster Recovery Journal in May 2021.
Disasters cause thousands of deaths and billions of dollars in damage worldwide each year. In the aftermath of a disaster, in addition to immediate and direct assistance to victims, communication is crucial. In fact, communication and access to information have become as important as water and electricity as a basic necessity. This is even more important at a time of crisis. Events that have the potential to cause connectivity interruptions can be predictable or completely unexpected. In either case, continuity of communication must be a priority.
The Internet is Essential
The internet becomes our go to place for news and information. During the past year, it also enabled an entirely homebound society. From access to learning, work, entertainment, banking and more, organizations around the world rose to the challenge of increased traffic and demand. Our internet infrastructure must be able to withstand any eventuality. Consider the issues that have affected connectivity this year alone: below-freezing temperatures, ice storms, hurricanes, tornadoes, wildfires, floods, the bombing of a major metro area and a global pandemic. You may have not noticed any major or permanent disruption to your internet connection. You may not have realized that your bank was down momentarily. Maybe you were wondering why you couldn’t make a cell phone call for a few hours. While disruption was minimized, in today’s instant on-demand society, a few hours can seem like an eternity. What’s worse, even limited interruption can infuriate customers and put your brand at risk. During such an emergency, telecommunications professionals at every level are scrambling behind the scenes to keep those connections up and running.
The Dawn of Communications
North American telecommunications originated in New York, with the first transatlantic cable. The first of the legacy carriers, “Ma” Bell, AT&T and others being headquartered at the source of those very first international communications, led to a centralization of communications in the better part of the last century. Even as connectivity branched out from that origination point, those legacy routes and origination points remained the same. One U.S. route that has become chronically congested and dilapidated is the Northeast Coast Corridor. Major tax incentives have led to a clustering of massive data centers in Virginia and the Carolinas. Northern Virginia alone is flush with 4.7 million square feet of commissioned data center space, according to market research from datacenterHawk. Carrier consolidation led to many long-haul networks being used for local distribution, which made record keeping impossible during the many integrations. Institutions historically engineered multiple paths but were beholden to what was offered by carriers as to routes and diversity. Telecom is as collaborative as it is competitive. Fiber providers often lease access from one another. In addition, it’s very difficult to ascertain the underground routes that these fiber links take in the last mile. In the always on, always available climate, network design must get smarter for both the steady state and when systems are disrupted. Flexibility to control and redirect the network needs to be the new norm. The ability to do so comes back to having access to hubs that can maintain connectivity, as well as offer alternate pathways.
The Key Lies in Operational Independence
There are a few ways to ensure internet traffic does not come to a standstill. The first is network diversity. Sometimes, diversity and redundancy are used interchangeably. But they are completely different, and both must be in place. There may be a false sense of security, as organizations misinterpret diversity vs. redundancy. For example, a redundant system will help if one path goes down. But there may be several carriers in the same ductwork. Even if an organization buys capacity from three or four different carriers, the flow of data could still be impacted. It’s a false sense of security. There also needs to be a physically diverse path. And even if there are diverse physical routes, those routes might go through the same building at some point in the path. If the building has issues, you’re in the same situation. So the best way to handle network security is a mesh solution, where you have redundant routes, resilient routes AND physically diverse routes. In other words, a plan A, B, and C. This is considered operational independence.
One way to ensure operational independence is with an interconnectivity hub. A carrier-neutral location with access to multiple terrestrial and subsea routes is an ideal situation. With this model, organizations can control the network. The right way forward is to provide this operational independence to orchestrate how networks function. If there is a hard cut, (a fiber optic cable is damaged) traffic can be redirected. If there is a soft cut, such as a hacker, the organization can also redirect traffic. In the past, networks were hard coded, it was difficult to turn paths on and off. If a large circuit went down, it stayed down. With advances such as SD-WAN and network function virtualization (NFV), institutions can react quickly and redirect data traffic. Organizations should have the flexibility to control and redirect the network whenever needed. But in order to do this, a hub environment is required.
As more advances in telecommunications develop – such as 5G, Edge computing, blockchain and more – data needs to be as close to the end-user as possible. Getting close to the edge means bringing the interconnection point to the source. When it comes to a global business, that includes subsea systems. A new model of connectivity also includes bringing the data center to the landing point of subsea cables. This creates a low-latency connection. In addition, organizations should consider disaster recovery sites that are away from a major metro hub. At the time of the September 11th attacks, lower Manhattan was not only the epicenter of the terrorists’ targets, it was also where most of the major network hubs were housed. Since then, we have learned to have backup locations that are far enough away from strategic sites to be secure, but close enough to provide an alternate route, quickly.
Expect the Unexpected
As we have seen with many natural, manmade and even biological events, preparation is key to keeping connectivity up and information flowing. For the infrastructure sector, this means plan and prepare. Consider uncertainties before they become a reality. No one could have predicted a global pandemic that stretched on for more than a year would change the way most of us live and work. At the heart of this disruption was our connectivity. If the pandemic hit just a decade ago, it would not be possible to remotely do very much of anything for an extended period of time. Increased capacity this past year put a demand on global networks, and those networks and their operators rose to the challenge. While it was certainly stretched thin, the internet did not break. One reason for this is the behind-the-scenes efforts to provide increased capacity and connectivity to organizations. In order to do this, essential workers remained on site at data centers, cable landing stations and IT departments around the world. To have that team in place, operations-based preparedness measures are critical to avoiding any and all downtime for customers. For a predictable event like a storm, those measures should include:
- Daily operations team meetings. This could include things like reviewing an impending storm’s status, and the potential impact it could have.
- Inventory of the supplies on hand across key areas of the facility
- Supply enough food to last a minimum of 72 hours for two people
- Ensure there are supplies available in case of emergency, such as ropes, sandbags, tarps, flashlights, batteries, ratchets, etc.
- Assess diesel fuel levels to make sure there is sufficient reserve to ensure the facility doesn’t lose power.
- Determine what current building load is and how long the fuel will last.
- Closely coordinate with critical vendors such as those that maintain generators and switchgear in the building.
- Maintain clear lines of communication with vendors to ensure support.
- Arrange to have satellite phones for one person on-site and one person off-site in case phone lines are impacted.
- Walk the grounds, perimeter and rooftop to make sure everything is secure and bring items inside as needed.
This high level of preparedness has become imperative. Extreme weather is no longer limited to the coasts. This year we saw a deep freeze in Texas, wildfires, tornadoes, hurricanes, and flooding. Expect the unexpected. It’s a lesson learned from past natural and manmade disasters and one that we have all been forced to contend with during the COVID-19 pandemic.