(Reuters) - Services from airlines to healthcare, shipping and finance were coming back online on Friday after a mistake in a security software update sparked hours-long global computer systems outages, another incident highlighting the vulnerability of the world's interconnected technologies.
After the outage was resolved, companies were dealing with backlogs of delayed and canceled flights and medical appointments, missed orders and other issues that could take days to resolve. Businesses also face questions about how to avoid future blackouts triggered by technology meant to safeguard their systems.
A software update by global cybersecurity firm CrowdStrike (CRWD.O), one of the largest operators in the industry, triggered systems problems that grounded flights, forced broadcasters off air and left customers without access to services such as healthcare or banking. Global shipper FedEx (FDX.N) faced major disruptions and some moderators who police content on Meta's Facebook were hit.
CrowdStrike is not a household name but it is an $83 billion company with more than 20,000 subscribers around the world including Amazon.com (AMZN.O) and Microsoft (MSFT.O). CrowdStrike CEO George Kurtz said on social media platform X that a defect was found "in a single content update for Windows hosts" that affected Microsoft customers.
"We're deeply sorry for the impact that we've caused to customers, to travelers, to anyone affected by this, including our company," Kurtz told NBC News.
CrowdStrike has one of the largest shares of the highly competitive cybersecurity market, leading some industry analysts to question whether control over such operationally critical software should remain with just a handful of companies.
The outage also raised concerns that many organizations are not well prepared to implement contingency plans when a single point of failure such as an IT system, or a piece of software within it, goes down. But these outages will happen again, experts say, until more contingencies are built into networks and organizations introduce better back-ups.
CrowdStrike shares closed down 11%. Its rivals SentinelOne (S.N) shares closed up 8% and Palo Alto Networks (PANW.O) closed up 2%. Microsoft closed down 0.7%. The scale of the outage was massive, but not yet quantifiable because it involved only systems that were running
CrowdStrike software, said Ann Johnson, who heads Microsoft's security and compliance business.
"We have hundreds of engineers right now working directly with CrowdStrike to get customers back online," she said.
President Joe Bidenwas briefed on the outage, a White House official said. The U.S. Cybersecurity and Infrastructure Security Agency said it observed hackers using the outage for phishing and other malicious activities.
U.S. Customs and Border Protection said it was experiencing processing delays and working to mitigate issues related to international trade and travel. The Dutch and United Arab Emirates' foreign ministries also reported disruptions.
"This event is a reminder of how complex and intertwined our global computing systems are and how vulnerable they are," said Gil Luria, senior software analyst at D.A. Davidson.
"CrowdStrike and Microsoft will have a lot of work to do to make sure that it won't allow other systems and products to cause this kind of failure in the future," he said.
Wall Street's main indexes fell on Friday, deepening a sell-off driven by tech stocks and mixed earnings. The Cboe Volatility index (.VIX), known as Wall Street's "fear gauge," hit its highest level since early May, and the dollar climbed as the worldwide cyber outage unnerved investors.
THOUSANDS OF FLIGHTS CANCELED
Air travel was immediately hit, because carriers depend on smooth scheduling that, when interrupted, can ripple into lengthy delays. Out of more than 110,000 scheduled commercial flights on Friday, 5,000 were canceled globally with more expected, according to aviation analytics firm Cirium.
Delta Air Lines (DAL.N) was one of the hardest hit, with 20% of its flights canceled, according to flight tracking service FlightAware. The U.S. carrier said it expected additional delays and cancelations potentially through the weekend.
Airports from Los Angeles to Singapore, Amsterdam and Berlin said airlines were checking in passengers with handwritten boarding passes, causing delays.
Banks and financial services companies warned customers of disruptions and traders across markets spoke of problems executing transactions. Insurers could face a raft of business interruption claims.
U.S. healthcare providers reported that outages were affecting call centers, patient portals and other operations. Mass General Brigham in Boston said it was treating only urgent cases while Tufts Medical Center warned that patients might experience delays or need to be rescheduled.
In Britain, booking systems used by doctors were offline, posts on X by medical officials said, while Sky News, one of the country's major broadcasters, was taken off the air.
As the day progressed, more companies reported a return to normal service, including Spanish airport operator Aena (AENA.MC) U.S. carriers United Airlines (UAL.O) and American Airlines (AAL.O), and Australia's Commonwealth Bank (CBA.AX).
U.S. Transportation Secretary Pete Buttigieg said system issues appeared to be resolving and transportation would hopefully be back to normal by Saturday.