» Archive for the 'Business Intelligence' Category

The Christmas Day Terrorism Plot: How Information Overload Prevailed and Counterterrorism Knowledge Sharing Failed

Monday, January 4th, 2010 by Jonathan Spira

There is no question that analyzing mountains of information and determining what is important, urgent, and worthy of follow-up (three separate and distinct categories) is a daunting task in any organization.

Are we sharing all of our knowledge yet?

Are we sharing all of our knowledge yet?

When the organization is the United States Federal Government and the amount of information that has to be addressed daily dwarfs what most people can conceptualize, lives may be at stake when an individual or system fails to connect the dots.

Such a failure occurred on December 25, 2009, but it need not have.

The tools to manage information on a massive scale do indeed exist and it is clear that the U.S. government is either not deploying the right ones or not using them correctly.

The National Counterterrorism Center, created in 2004 following recommendations of the 9/11 Commission, has a mission to break “the older mold of national government organizations” and serve as a center for joint operational planning and joint intelligence.  In other words, various intelligence agencies were ordered to put aside decades-long rivalries and share what they know and whom they suspect.  Unfortunately, while this sounds good in theory, in practice this mission may not yet be close to be being fully carried out.

In addition to the fact that old habits die hard (such as a disdain for inter-agency information sharing), it appears that the folks at the NCTC failed to grasp basic tenets of knowledge sharing, namely that search, in order to be effective, needs to be federated and contextual, that is to say it needs to simultaneously search multiple data stores and present results in a coherent manner.

Discrete searches in separate databases will yield far different results compared to a federated search that spans across multiple databases.  All reports indicate that intelligence agencies were still looking at discrete pieces of information from separate and distinct databases plus the agencies themselves were not sharing all that they knew.

In this case, much was known about Umar Farouk Abdulmutallab, the Nigerian man accused of trying to blow up Northwest Flight 253.  In May, Britain put him on a watch list and refused to renew his visa.  In August, the National Security Agency overheard Al Qaeda leaders in Yemen discussing a plot involving a Nigerian man.  In November, the accused’s father warned the American Embassy (and a CIA official) in Abuja that his son was a potential threat.  As a result, the son was put on a watch list that flagged him for future investigation.  He bought his plane ticket to Detroit with cash and boarded the flight with no luggage.  Yet, almost unbelievably, no one saw a pattern emerge here.

Shouldn’t a system somewhere have put the pieces of this puzzle together and spit out “Nigerian, Abdulmutallab, Yemen, visa, plot, cash ticket purchase, no luggage = DANGER!”?

Information Overload is partially to blame as well.  Given the vast amount of intelligence that the government receives every day on suspected terrorists and plots, it could very well be that analysts were simply overwhelmed and did not notice the pattern.  Rather than being immune from the problem, given the sheer quantity of the information it deals with, the government is more of a poster child for it.

Regardless of what comes out of the numerous investigations of the Christmas Day terrorism plot and the information-sharing failures of the various intelligence agencies, one thing was abundantly clear by Boxing Day: the Federal Government needs to greatly improve its ability to leverage the intelligence it gathers and connect the dots.

Clearly, there are many changes that need to occur in order to improve security but one relatively simple way for the government to proceed is to take the first steps to lower the amount of Information Overload and raise the signal-to-noise ratio so that critical information can rise to the top.

Jonathan B. Spira is CEO and Chief Analyst at Basex.

In the briefing room: Bluenog ICE

Thursday, June 25th, 2009 by Jonathan Spira and Cody Burke

Ten years ago, Basex laid the groundwork for the Collaborative Business Environment (CBE), a conceptual framework for a workspace for the knowledge worker that is now starting to supersede the traditional desktop metaphor of separate and distinct tools.  A properly designed CBE facilitates knowledge sharing and collaboration and, especially in today’s economic environment, managers are looking to technology to give their organizations a competitive advantage.

Bluenog, an enterprise software company, this week released Bluenog ICE 4.5 (ICE stands for integrated collaborative environment), the latest version of the company’s enterprise software suite.  Bluenog integrates multiple open source software projects to form the basis of its platform.  The company, through its professional services division, will further integrate ICE into an organization’s existing systems.

Bluenog ICE originally included content management, portal, and business intelligence functionality.  ICE CMS is a content management system built on Apache Cocoon, Apache Lucene, OS Workflow, TinyMCE, and HippoCMS open source projects.  ICE Portal is a portal solution that leverages Apache Portals, Apache Jetspeed-2, Apache Wicket, Adobe Flex, and Spring Source.  ICE BI provides business intelligence and reporting and is based on Eclipse BIRT and Apache Jackrabbit.

These core components have all received enhancements for the new release.  The HTML editor in ICE CMS has been replaced by the TinyMCE HTML editor and ICE BI has improved report viewing and search integration.  Also new for this release is ICE Central, a simplified central management console for all ICE components, and a propagation tool to move content, portal artifacts and configurations across environments.

These improvements are all worthy of note but what may really help organizations realize significant enterprise productivity and efficiency gains is that Bluenog added significant collaborative technology to ICE, namely ICE Wiki and ICE Calendar.  The wiki component is based on the JSPWiki, Apache Jackrabbit, Apache Lucene, and Apache FileUpload open source projects.  The wiki is accessed through an ICE portlet and features rich HTML editing page level permissions, version control, reporting on page and link usage through ICE BI, the ability to manage attachments, support for wiki markup language, and support for multiple wikis running on a single server.

Wikis are an increasingly popular tool for content management within organizations of all sizes and ICE Wiki allows non-technical knowledge workers to create, edit, and maintain content using a fairly easy-to-understand interface.

ICE Calendar is a group calendaring application based on the open source Bedework project.  Just as in ICE Wiki, the calendar is available as an ICE portlet, and enables publishing of events, workflowing of events for approval, and importing and exporting events to other iCalendar-based calendars.

Bluenog ICE falls into the category of commercial open source software.  It’s built using open source projects but sold as a commercial package.  Virtually unknown several years ago, commercial open source is becoming a popular alternative for organizations of all sizes that want the openness of open source but don’t necessarily have the skills to do the heavy lifting to deploy and integrate multiple open source projects.

We’ll be taking a look at the changes that are taking place in the content management space, including where commercial open source fits in, in a report slated for next month.

Jonathan B. Spira is CEO and Chief Analyst at Basex.
Cody Burke is a senior analyst at Basex.

Finding the Needle in the Corporate Haystack: Dow Jones and Generate join forces

Friday, April 25th, 2008 by Cody Burke

Searching for information is a time consuming task, one that often results in disappointment and overwhelming quantities of information.  A search may result in the correct answer, however, wading through and separating the relevant from the irrelevant is no small task for the knowledge worker.  Ultimately this is the consequence of searches returning correct results, but not necessarily correct answers.

With an eye towards resolving this problem, Dow Jones announced last week that it had acquired Generate, a business intelligence company, and would be forming a new Business and Relationship Intelligence unit within the Enterprise Media Group.  The Generate platform works by crawling millions of Web sites and extracting data on four million companies and over six million executives.  This, when combined with so-called trigger events, such as mergers, executive changes, venture funding, and partnerships, provides precise reports that allow companies to detect changes in the competitive landscape, identify prospects, and nourish their own networks.  Extraction is complemented by relationship mapping technology, showing the best possible path to approach an executive, anticipate shifts in the market, and make the most out of personal and corporate connections.  While busy executives may not think to update their Xing or LinkedIn profile for weeks after a change, if at all, a system utilizing Generate technology would pick up on a change almost instantaneously and ensure that those who need to be aware of the change are.

The savings in time and headaches for the knowledge worker – and by extension in productivity for the enterprise – from such a system should not be underestimated.  The pairing of Dow Jones and Generate is indicative of the massive importance of better searching technology to the knowledge worker.  The potential gains in the fight against Information Overload are no less exciting – as searching technology improves, those overwhelming piles of results will shrink, and the information we were looking for all along will float to the top.

Cody Burke is a senior analyst at Basex.


google