» Archive for January, 2011

Search and the Quest for the Perfect Dishwasher

Thursday, January 27th, 2011 by Jonathan Spira

Search. Sit. Stay. I get my commands mixed up occasionally.

Even if a knowledge worker were to enter a perfectly-formed query into Google’s search box, there is still a high degree of probability that he would not find what he was looking for.  While, in the past, providing the best, most accurate search results and winnowing the wheat from the chaff had more to do with not providing out-of-date and/or non-relevant information than anything else, today that is no longer the case.

In the Web’s earliest days, once marketers figured out how to game the search engines of the time with keywords that had nothing to do with the content on the associated Web page, search had become fairly useless.  Indeed, it was the arrival of Google on the scene in 2000 <check date> that made Web search useful again. Search engines including Google continue to be overwhelmed by various forms of Webspam; as soon as Google and others vanquish one form of spam, however, another rears its ugly head.

Paul Kedrosky, founder of GrokSoup, one of the first hosted blog platforms, an analyst on CNBC and currently publisher of a financial blog , tried to use Google to help him select a new dishwasher in December of 2009.   His conclusion (which was dispatched as a tweet): “To a first approximation, the entire web is spam when it comes to appliance reviews.”

In “Dishwashers, and How Google Eats Its Own Tail,” an article that was published in InfectiousGreed, on December 13, 2009, Kedrosky explains how Google “has become a snake that too readily consumes its own keyword tail” as marketers exploit Google’s technology to rise to the top of the search results page, regardless of whether they deserve to be there or not.

As a result of this attack of sorts, Web search’s usefulness is once again in question as low-quality, unreliable, and frequently plagiarized content continue to supplant legitimate entries in search results.

It appears that getting the right answer or response from Google has never been harder, and even Google is taking notice of the problem.  On January 21, 2011, writing in the Official Google Blog, Matt Cutts, the head of Google’s Webspam team, wrote that there had been “a spate of stories” [examples include "On the Increasing Uselessness of Google," "Why We Desperately Need a New (and Better) Google,"  and  "Content Farms: Why Media, Blogs & Google Should Be Worried"] concerning Google’s less than stellar search quality.

Google is fighting the war on Webspam on three fronts and this is no mean feat.  Today, Webspam comes from three main sources: 1.) individual Web pages with repetitive spammy words and phrases, 2.) hacked Web sites, and 3.) content farms (sites that either produce shallow content of their own or copy other sites’ content).  The topic of Webspam could fill a book unto itself but it is worthy of this brief mention because it brings Information Overload to a whole new level in terms of making it harder for knowledge workers to find what they are looking for.

If we were to repeat our study of the failure rate for searches, Webspam would most likely result in an increase in the number of searches that actually failed but were not recognized as a failure by the searcher.  This is a dangerous trend if it continues unabated.

Search. Sit. Stay. I get my commands mixed up occasionally.

The problem has even given rise to a new type of search engine.  Blekko, “a better way to search the web,” uses slashtags to cut out spam sites and my tests of the search engine show that it provides significantly less spam in its results.

Google has heard “the feedback from the web loud and clear” and believes that the company “can and should do better.”  It announced plans (via Cutts’ post) to combat all three types of Webspam, including making changes to its algorithm and giving users a means of providing “explicit feedback” on spammy sites.

Jonathan B. Spira is CEO and Chief Analyst at Basex.

Man v. Machine: Who Analyzes Information Faster?

Thursday, January 20th, 2011 by Cody Burke

Who do YOU think will win?

That information functions as currency is no surprise to anyone in this day and age.  What may be surprising are the lengths that people go to collect the raw data, turn it into actionable nuggets of information, and disseminate it.

Two recent articles in the New York Times call attention to the way that information has become meaningful and critical in both the business world and the political world.

Both articles, “Computers That Trade on the News” on December 22, 2010 and “Where News Is Power, a Fight to Be Well-Armed,” on January 16, 2011, detail the extent to which information is valued in politics and trading.

The subjects of these articles are also a study in contrasts.  As junior political aides parse the news and pull out the most useful and relevant insights and ammunition for the day’s politicking, the computers of Wall Street are humming away, using advanced algorithms and linguistic-based software to extract meaning from the wealth of unstructured data available online.

Politics has always been a bit old fashioned, but the divide between how Wall Street and Washington are utilizing information has never been starker.  In Washington, the day for junior aides and staffers begins early in the morning as they read the news, monitor blogs and social networks, and condense their findings into memos for their superiors.  These information gatherers put in a few hours of data collection and selection, then report to an office for the rest of their day that can stretch well into the evening.

The staffers are valued for their ability to ferret out tidbits of information online that have value.  Describing a media monitor under his supervision, Dan Pfeiffer, White House communications director, stated that, “For such a young guy, Andrew has a great ability to sniff out stories that need to be handled with dispatch. During our biggest fights, from health care to the Supreme Court confirmations, Andrew repeatedly spotted potential problems in the farthest reaches of the Internet before anyone else. That information was essential to our success.”

On Wall Street, a very different picture is playing out.  Traders are using software tools that analyze the massive amount of online unstructured data to extract sentiment around companies, analyze it, and then trade on it, often without human assistance.  The software analyzes words, sentence structure, and emoticons from blogs, company Web sites, editorials, news articles, and social software tools such as Twitter.  If the software detects anything that reflects positively or negatively on a company or a section of the market, then it can trigger automated trading.
According to Aite Groupa, a financial services consulting company, around 35% of quantitative trading firms are currently exploring the use of unstructured data in automated trading.  The trend is not likely to stop there, given the competitive advantage that even a few seconds can give a firm in high frequency trading.

Could Andrew Bates, our talented political aide, sniff out information faster and more accurately than the algorithms of Wall Street?  Short of a face-to-face showdown, it is hard to say.  One thing is for sure though; he would probably appreciate the extra hour of sleep.

Cody Burke is a senior analyst at Basex.

The iPhone Cometh

Thursday, January 13th, 2011 by Cody Burke

It would be understandable if you were under the impression that nothing at all happened this week except for the unveiling of the long-rumored Verizon iPhone.  The launch has been hyped and anticipated for several years, in part because it would be the moment when AT&T would lose its exclusivity death grip on what has become perhaps the most iconic mobile phone ever.

So what did actually happen?  Well, for those hoping for a slew of new features, or an LTE-powered world phone, not much.  The iPhone 4 that Verizon Wireless will be offering is similar to the iPhone 4 that AT&T offers with a few exceptions.  The phone and its antenna have been redesigned to work with Verizon’s CDMA network, and there is hope that this will result in a solution to the “antenna-gate” problem, whereby users of the AT&T iPhone 4 lost calls when holding the phone in a certain way.  In addition, the Verizon version of the iPhone will be able to serve as a Wi-Fi hotspot for up to five Wi-Fi enabled devices.

So now that iPhone customers have a choice in terms of a mobile operator, which network should they choose?  While the accepted wisdom is that AT&T’s 3G network is actually faster than Verizon’s, its coverage is not nearly as broad.  If you spend most of your time in a major city and currently do not have many problems with your connection, then leaving AT&T for Verizon might be a bit hasty.  If you travel around the country, and find yourself on rural back roads, away from major metropolitan areas, Verizon will be the more attractive operator.

Verizon released the iPhone on its 3G CDMA network instead of on its new 4G LTE network.  According to Apple COO Tim Cook the official reason for this is that Verizon customers “wanted the iPhone now” and that the LTE technology would have forced design compromises that Apple was not willing to make.

While the Verizon iPhone gains a feature (Wi-Fi hotspot capabilities) compared to AT&T’s it also loses some capabilities, namely support for GSM and the ability to work in most countries around the world.  While other Verizon smartphones including almost all those from Research in Motion support CDMA for Verizon Wireless’s network and GSM for roaming, the iPhone does not.  This may be a deal killer for users who travel frequently.

Because of limitations on CDMA networks the iPhone loses one additional feature, the ability of to be on a phone call and maintain a data connection at the same time.  Verizon may address this in the future, but for now, users will have to choose if they want to look up locations on Google maps or talk on the phone.

The new enhancements and limitations on the Verizon iPhone may make choosing a mobile operator for your iPhone easier than originally thought.  For current AT&T iPhone customers, moving to Verizon will require the purchase of a new iPhone (one that runs on Verizon’s CDMA network) and a possible cancellation fee (AT&T raised its cancellation fees in 2010 in anticipation of Verizon’s announcement).  But iPhone fans will have to choose between simultaneous voice and data and Wi-Fi hotspot capabilities not to mention the ability to roam internationally.

Cody Burke is a senior analyst at Basex.

Some New Year’s Resolutions for the Information Overload-Minded

Thursday, January 6th, 2011 by Jonathan Spira

To lowering Information Overload!

The past year has been marked by increases in the cost of Information Overload, which for the U.S. economy now stands at $997 billion, as well as hopeful signs that awareness of the problem is rising as well.  The second annual Information Overload Awareness Day was a smashing success, and drew several hundred attendees from across the globe.  CNN ran a feature entitled Happy Information Overload Day and the Belgian government, taking note of the problem, tried for a day without e-mail.

With the New Year already underway, we can start with a few simple New Year’s Resolutions that are time tested in lowering the amount of Information Overload we all face.

1.)    Learn better search techniques.  Control search results with Boolean logic by using AND or OR and use advanced options to narrow the field.

2.)    Use restraint in communications.  Don’t cc the world, don’t include more people than necessary in any communication, avoid gratuitous “thanks” and “great” replies, and avoid reply-to-all at all costs.

3.)    Write clearly.  Better yet, refrain from combining multiple themes and requests in one single e-mail.  And make sure the subject is specific as opposed to general (writing “Help needed” without further details helps no one, especially the recipient).  These simple steps will add instant clarity with little effort.

4.)    Read what you write – before you click send.  Unclear communications result in excessive and unnecessary back-and-forth communications that would have been unnecessary were the first missive unambiguous and to-the-point.

5.)    Read what others write – before replying.  While it would be nice to believe that people will place the most important information at the very beginning, often times the key facts are buried in the closing paragraphs.  What you are about to ask may already have been covered.

6.)    Value your colleagues’ time as if it were your own.  If a response to an e-mail is not immediately forthcoming, don’t pick up the phone or send an IM saying “did you get my e-mail?”.

Happy New Year!  Prosit Neujahr!

Jonathan B. Spira is CEO and Chief Analyst at Basex.