By Garrett Datz, Principal | Technology Advisory
Record low temperatures pummel the US. Ice storms wreak havoc on the west coast, power outages riddle the south, flights to every corner are grounded, and holiday travelers are stranded. This isn’t a dystopian scenario; it’s rooted in fact. December 2022’s unexpected cold front, a spike in travel, and outdated tech resulted in Southwest Airlines canceling 65% of flights and impacting millions of passengers between December 26 and 31. The unrecoverable cost to the airline to account for refunds and compensations is estimated at close to a billion dollars in net outlay and lost revenue.
This could be perceived as one of the largest and most extensive business crises in recent history. And, with the proper proactive measures and technology upgrades, it could have been prevented.
How Could This Have Happened?
While there is not a simple answer, it’s safe to say it was the perfect storm (pun intended). Ultimately, it’s likely that uninformed or reactive decision-making, a lack of proactivity, and inadequate systems amalgamated into a dramatic cascade of misfiring.
Let’s explore the facts:
- While some flights were canceled due to weather, many flights were delayed due to ground crew shortages and temperature safety issues, leading to longer flight clearance times.
- Southwest uses a point-to-point flight system that requires substantially more careful tracking of resources at each airport and assignments based on schedules.
- Southwest’s IT system to track all their personnel assets to inform flight crew decisions is decades old with minimal automation. All reporting and change tracking are done through a call center that had recently been upgraded and was unable to meet the surge capacity.
- Once Southwest fell behind the only method to catch up was to cancel flights but, due to the previously mentioned manual efforts, the entire system failed.
While Southwest will spend years recovering from this event, for other organizations there are lessons to be learned and applied to prevent catastrophes on any scale.
How Can Our Organization Prevent a Technology Crisis?
The following four tactics will help reduce the risk of system failure irrespective of business size or industry.
1. Understand the limits of your systems and how they can scale
Gartner defines scalability as the measure of a system’s ability to increase or decrease performance and cost in response to changes in application and system processing demands. Our technology advisory team defines scalability as the upper limit of any system to increase output by simply throwing people at the problem before a new solution is required.
While the answer is sometimes to throw people at the problem, it’s not a safeguard. In this instance, Southwest likely didn’t understand the upper limit of this approach as a solution. To be truly successful, IT leaders and technology teams must fully grasp the capacity of their system(s) to accommodate a greater amount of usage at any given time and under varying circumstances. Major factors involve:
- Effectively measuring system scalability and breakpoints
- Budgeting for appropriate investments in technology and upgrades
- Correctly understanding your systems lifecycle age
- Evaluating system and operational failures to measure risk state
2. Appropriately Assess and Acknowledge Business Risk
Herein lies the real problem. In the case of Southwest, and too often many businesses, there is a disconnect between both understanding and anticipating just how bad the problem could ever be and the repercussions for the company. We often encounter extremely old, antiquated systems and, when discussing this with company leadership, usually start with a simple question:
If this system turned off tomorrow and it never came back on, how bad would that be?
The core of that question comes from the fact that new systems and technology are rapidly sprinting away from the skill set to keep these systems online and viable. This means businesses with aging infrastructure are fighting for an ever-shrinking pool of talent, tools, and platforms capable of maintaining that viability.
3. Proactively plan and budget for costs associated with digital transformation
Don’t fear or avoid the hard and soft costs of any IT systems change. Think of it as a meaningful and worthwhile investment in the future of your business. The hard truth is, investing in your systems is inevitable. Technology is being designed to evolve, mature, and scale—which means users must be prepared to do the same.
While Southwest knew about the need to modernize its systems and the associated costs, for whatever reasons, the airline deferred action and is unfortunately now facing the consequences of that decision. Many executives recoil from having to address bad news directly, instead preferring the benefits of positive spin (who doesn’t?). However, every increasing technical debt of short-term decision-making paired with the failure to acknowledge the actual state of the technology environment will lead to inevitable crisis situations.
Whether proactively planning for system upgrades, change management, or direct outlay to long-term process or business interruption—make it part of your annual strategic planning, have the appropriate conversations with the right experienced experts, and budget accordingly. In this day and age, transformation is unavoidable.
4. Know what’s going on in your organization
A common problem we encounter during our IT assessments is a consistent misalignment in the operational and IT state of the organization that stems between the c-suite and functional staff. Those working in the systems daily recognize when and what change is needed, and do their best to communicate this to leadership. Leaders and boardrooms, however, are often blinded by the investment to make such upgrades and, when paired with positive aspects of the business, struggle to understand why such changes are necessary—the old, if it isn’t broken, don’t fix it mindset.
Authentic leadership entails making a genuine effort to understand the whole the entire problem, not just the horizon for which executives strive. Effective leaders will continuously ask questions and assess the current state of their organization all while plotting and planning for an effective future state.
When it comes down to it, both parties (leadership and staff) want the same thing: business growth. And to realize that growth, it’s important to get on the same page, understand current or impending limitations, and develop a forward-looking plan to get ahead of them.
As Southwest responds to this system failure—already having budgeted $1B to upgrade its IT systems and infrastructure—use this scenario as a learning opportunity. Equipped with the sound strategies shared in this article, you’re at an advantage to prevent system failure and intentionally scale your technology.
In summary, here’s how to get started:
- Develop a true understanding of your IT, systems, and business risks
- Address antiquated and legacy systems and, at a minimum, understand how long they can realistically be maintained and serve the business
- Understand the limits of adding people to your business and processes to solve problems—and where growth and capacity stops, or worse, fails
- Establish actionable and effective plans for when things go wrong
- Build capability through the organization to address the fear and cost of change
- Take action before things go wrong
