McAfee's botched virus definition file update last week crashed what some reports are indicating could have been thousands of PCs around the world, enraging customers and sending the security software vendor's customer support team into full scramble mode -- and had executives issuing a string of mea culpas along the way.
The faulty signature update file mistakenly flagged the critical SVCHOSE.EXE file for Windows XP SP3 as a threat and quarantined it, causing machines running XP to shut down every 60 seconds.
Further complicating matters was the fact that most of the affected users, particularly those folks just using their PC at home and not at work, weren't able to even get online to access the McAfee site to find any explanation or remedy for the issue.
Though widespread media reports suggested impacted users could number in the thousands, the company downplayed the number of systems affected. Barry McPherson, McAfee's (NYSE: MFE) executive vice president of support and customer service, wrote in a blog posting Thursday that his company believes that the problem "impacted a small percentage of our enterprise accounts globally and a fraction of our consumer base -- home users of products, such as McAfee VirusScan Plus, McAfee Internet Security Suite and McAfee Total Protection."
Officials also tried to do some damage control in the hours and days immediately following and eventually provided a fix for the crippling error and offered to reimburse customers for any "reasonable costs" for home or home-office users who weren't able to immediately fix their PCs themselves.
"I want to apologize on behalf of McAfee, and say that we're extremely sorry for any impact the faulty signature update file may have caused you and your organizations," McPherson said in his post.
The apology came almost 36 hours after the initial screw-up was first discovered, which disabled computer systems at hospitals, police departments, government agencies and companies large and small around the world.
More than a few customers took issue with what they described as McAfee's delay in apologizing immediately to the users impacted by the flawed update release, as well as what they criticized as lackluster quality control.
"What a spectacular failure of QC," wrote one response to McAfee's security blog. "Normally I dont wish for this kind of thing but I seriously hope jobs are lost over this seat-of-your-pants operation."
"Like so many others, as soon as this issue is resolved, I am switching to another virus program, probably Symantec," wrote someone calling themselves "Lane." "If this release of McAfee's passed their quality control, it seems to me they have no quality control."
Security update testing failure
McAfee officials said it recently made a change to its QA environment that resulted in "a faulty DAT making its way out of our test environment and onto customer systems."
"Mistakes happen," McPherson wrote in his blog posting. "No excuses. The nearly 7,000 employees of McAfee are focused right now on two things, in this order. First, help our customers who have been affected by this issue get back to business as usual. And second, once that is done, make sure we put the processes in place so this never happens again."
McPherson said the company is implementing additional QA protocols for any releases that directly impact critical system files in the future and plans to add capabilities to its cloud-based Artemis system that will provide an additional level of protection against false positives.