The Christmas Day Terrorism Plot: How Information Overload Prevailed and Counterterrorism Knowledge Sharing Failed
There is no question that analyzing mountains of information and determining what is important, urgent, and worthy of follow-up (three separate and distinct categories) is a daunting task in any organization.
Are we sharing all of our knowledge yet?
When the organization is the United States Federal Government and the amount of information that has to be addressed daily dwarfs what most people can conceptualize, lives may be at stake when an individual or system fails to connect the dots.
Such a failure occurred on December 25, 2009, but it need not have.
The tools to manage information on a massive scale do indeed exist and it is clear that the U.S. government is either not deploying the right ones or not using them correctly.
The National Counterterrorism Center, created in 2004 following recommendations of the 9/11 Commission, has a mission to break “the older mold of national government organizations” and serve as a center for joint operational planning and joint intelligence. In other words, various intelligence agencies were ordered to put aside decades-long rivalries and share what they know and whom they suspect. Unfortunately, while this sounds good in theory, in practice this mission may not yet be close to be being fully carried out.
In addition to the fact that old habits die hard (such as a disdain for inter-agency information sharing), it appears that the folks at the NCTC failed to grasp basic tenets of knowledge sharing, namely that search, in order to be effective, needs to be federated and contextual, that is to say it needs to simultaneously search multiple data stores and present results in a coherent manner.
Discrete searches in separate databases will yield far different results compared to a federated search that spans across multiple databases. All reports indicate that intelligence agencies were still looking at discrete pieces of information from separate and distinct databases plus the agencies themselves were not sharing all that they knew.
In this case, much was known about Umar Farouk Abdulmutallab, the Nigerian man accused of trying to blow up Northwest Flight 253. In May, Britain put him on a watch list and refused to renew his visa. In August, the National Security Agency overheard Al Qaeda leaders in Yemen discussing a plot involving a Nigerian man. In November, the accused’s father warned the American Embassy (and a CIA official) in Abuja that his son was a potential threat. As a result, the son was put on a watch list that flagged him for future investigation. He bought his plane ticket to Detroit with cash and boarded the flight with no luggage. Yet, almost unbelievably, no one saw a pattern emerge here.
Shouldn’t a system somewhere have put the pieces of this puzzle together and spit out “Nigerian, Abdulmutallab, Yemen, visa, plot, cash ticket purchase, no luggage = DANGER!”?
Information Overload is partially to blame as well. Given the vast amount of intelligence that the government receives every day on suspected terrorists and plots, it could very well be that analysts were simply overwhelmed and did not notice the pattern. Rather than being immune from the problem, given the sheer quantity of the information it deals with, the government is more of a poster child for it.
Regardless of what comes out of the numerous investigations of the Christmas Day terrorism plot and the information-sharing failures of the various intelligence agencies, one thing was abundantly clear by Boxing Day: the Federal Government needs to greatly improve its ability to leverage the intelligence it gathers and connect the dots.
Clearly, there are many changes that need to occur in order to improve security but one relatively simple way for the government to proceed is to take the first steps to lower the amount of Information Overload and raise the signal-to-noise ratio so that critical information can rise to the top.
Jonathan B. Spira is CEO and Chief Analyst at Basex.
January 5th, 2010 05:01
It’s easy to blame information overload here – not least because the National Counter-terrorism Center reportedly gets 8,000 messages a day. While you can’t draw firm conclusions from press coverage, my guess is that the problem here is lack of role clarity between the various agencies. It has been suggested that both the Counter-terrorism Center and the State Dept. thought the other was putting the Detroit bomber on a terrorism list. In the event, nobody did.
January 7th, 2010 09:26
My biggest concern is always that someone don’t connect the dots. They want to preserve the information in their back pockets for future or different use, which in this case a report indicates a plan to detain the person after the plane landing.
Another concern of mine is that sometimes mid-level personnel deem the information unimportant or typical run of the mil info.
Last not least, why some airlines see no-fly list, some don’t. Another example of not connecting the dots.
January 7th, 2010 16:06
Jonathan, thanks for your useful commentary on how information overload obstructed discovery of the Nigerian plotter. It is difficult for even very smart people to connect the dots when there are so many. They need to use strategies by which technology would connect the dots, or at least highlight the most interesting dots.
IF a “Nigerian” in “Yemen”, a man to watch for, was in the database
(from source #1, CIA)
AND a “Nigerian” “radical” named “Abdulmutallab” was in it (from source
#2, the Embassy)
OR a “Nigerian” on a British “no-fly” list was in it (from source #3,
the British),
OR a “Nigerian” flyer who “paid cash” was in it (from source #4, the
airline)
THEN a simple, recurring Boolean search should have discovered him. A recurring search for [+Nigerian +(radical OR Islamist OR plot OR suspect OR "no-fly" OR "paid cash")] would have put him in the spotlight.
The unknown suspect was on the first list. The search could have identified him from either source #2, #3, or #4.
January 12th, 2010 18:19
Jon, I agree with everything you wrote, but the problem is even more fundamental then separate databases kept by the govt. My initial reaction after reading the terrorist’s name was “look at the length of that last name, I bet someone misspelled it.” In my many years of database experience, I’ve seen many instances of names such as AT&T spelled 5 or more different ways in the same database (ATT, A.T.T., AT & T, etc.), so I figured that with this guys name, we had no chance. Sure enough, I later heard that my guess was correct – Google “Abdulmutallab misspelled” and you’ll get many news detailing that one of the many failures that led to the terrorist being allowed to fly was that the State Department misspelled his name.
The terrorists have an inherent advantage just due to the length and complexity of their names when spelled in English – it’s unlikely that 2 govt employees hunting and pecking at their keyboards would both ever spell a name like that correctly! Maybe we should pass a law, that anyone wanting to enter the US needs an English version of their name with no more than say 10 letters, of which no more that 4 may be vowels!