Economically, this market is finally starting to take shape — the ideas and attempts have been out there for a few years, but consumer companies have been on the fence about whether the blogosphere is worth listening in on. Until recently, that is. Umbria claims they’ll have $2M revenue this year and will be profitable next year, but the overall market for this kind of service is still only $20M according to the article (Intelliseek has about 1/3rd of that market).

Technologically, Umbria also sounds pretty interesting. They claim to have a competitive edge in automating most of the process:

Umbria’s solution is entirely software-based. [Umbria's] competitors also meet with clients to interpret the data and suggest strategic responses. “Ultimately we rely on both technology and humans for analysis,” says Max Kalehoff, marketing director for BuzzMetrics [another Umbria competitor]. “Umbria takes an extremely automated approach.”

Umbria’s technology sounds like a pipeline of parsers that generates features that in turn drive product and sentiment classifiers (and those drive reporting):

Every few hours Umbria sends an application called a spider out over the web to scour the blogosphere for postings about the firm’s clients, most of which are big consumer companies, such as Electronic Arts, SAP, and Sprint. By analyzing keywords in blogs, Umbria can classify each citation thematically. In the case of Sprint, for example, Umbria’s software can tell whether a blogger is talking about customer service, the company’s advertisements, or a particular calling plan.

Another big challenge is to decipher what’s on a blogger’s mind. To figure out whether an opinion is strong or tepid, for example, it helps to know that “awesome” is a stronger endorsement than “pretty cool,” and that “shoddy” is less damning than “abominable.” Umbria has several employees with Ph.D.s in linguistics and artificial intelligence who are forever tweaking the software to make it better at categorizing opinions.

I can’t help thinking that more manual tweaking goes into each client’s setup than this description lets on, but still, I’m glad they’re seeing success, and I bet those linguists are having fun with the blogosphere, even if they have to do a bit of slumming to come up with their rules:

The software can also estimate the author’s age and gender. Elongated spellings (”soooooooo”), multiple exclamation marks (!!!), and acronyms such as POS (”parent over shoulder”) suggest a teenage female member of Generation Y (born after 1979). The blogger is probably a teenage boy if a posting is rife with hip-hop terminology such as “aight” (translation: “all right”) and “true dat” (”I agree!”).

There you have it, you don’t even have to know the language to have your voice heard by the people who want to sell you more stuff. Now that’s power. On one side of that function, at least.

So far, I’ve tried a few different texts, and about half the time the returned tags are great. This is impressive, because this is not an easy problem to solve, but 50% precision is not quite enough for prime time. If someone (sploggers?) unleashes Tagyu to auto-tag a large volume of posts that feed back into the del.icio.us and Tagyu system, that would be detrimental to improving precision of the system, unless you could assign some kind of a score to the quality of tags (yes, that’s a chicken/egg thing).

Maybe we need some kind of a large-scale tag-quality feedback system. Some clever piece of javascript that lets you click “this tag is right on” or “this tag is a cruel joke” when reading someone’s blog or feed. Of course, if you’re an idiot at tagging, you’re not going to install that piece of javascript. An aggregator might be the best place to do that, where attention.xml lives (eventually).

This is the first service of this kind that I’m aware of, and there are lots of applications of this kind of thing in blog search. There could be an ad-matching app in there, too. And, an intermediate step in Tagyu is matching content to other content (and then to tags). I hope Adam Kalsey keeps up the R&D effort on this. Tagyu has a super-clean looking site. Very nice.

btw, for this post’s text, Tagyu returned the following tags: tagging del.icio.us tools. Looks good to me.

—–
(Via BuzzMachine)

Greg Linden wonders whether the small blog/feed search engines will survive the entry of the giants into the field:

… it is good for a startup to see the entry of a big company into its area since it attracts attention and legitimizes the field … but competing directly against these giants is scary if you have no differentiator.

While the small players have driven innovation and broad acceptance of concepts like link popularity and tagging, they continue to struggle with scalability. Also, the most compelling products to come out of the blog search startups, while they’ve been exciting and even revolutionary from a user’s point of view, have not been technologically deep in the sense of difficult to duplicate by the search giants. There have been exceptions, of course, but no really deep technology is in evidence among those services that have made the biggest splashes (technorati, bloglines, flickr, del.icio.us).

So, when a search giant comes in with equal-or-better features, scalability, and a huge engineering team that can relatively quickly merge ideas emerging from the programming part of the blogosphere into the vast search toolkit that the giants already have, that might just cast a bit of a cloud over the little guys.

Having said that, I believe there will continue to be a place for the little guys in the blog search ecosystem. They’re the real innovators and they have their ears to the ground. And even at the break-neck speed at which Yahoo and Google have been rolling out features lately, an army of little guys can still cover a lot more ground than the two giants in the search for the next cool thing that will make users’ lives (even) better.

…
1. How well will tagging work as an organizing and information retrieval method when there are millions of tags?–That’s where having additional filters, such as identity, trust or cohort group becomes relevant–becomes needed.

2. How can developers move tagging into a wider market? I describe tagging to non-geek friends and they are interested, but these folks aren’t blogging, don’t use tag-friendly photo services and are a world away, still–how can the tools bring them closer?
…

I don’t think that tagging will turn out to be the emperor’s new clothes (which isn’t at all what Susan is suggesting, either). But there’s a sense here that the honeymoon is over and it’s time for tagging to get serious about earning its keep for readers and searchers and to make stuff not just more broadcastable in flickr and Technorati, but also to make the good stuff more findable.

Do you find blogs via links or through search? We wanted to know, so we lined up the top 20 blogs in the blogosphere according to Technorati … we did peek at the number of searches each received over the last week.
…
Turns out the lead blog on Technorati runs in the middle of the search pack. Fark ranked #5 on Technorati, but in terms of searches — it’s the top dog, er, blog.

Note that Yahoo! restricted its sample set to the top 20 blogs according to Technorati, so if all blogs were lined up according to search popularity on Yahoo, the top 20 might or might not include any of the Technorati top 20.

So, what’s this mean? It means that searches map to a different popularity ranking than links do. They’re two different measures. Technorati takes a sort of intra-blogosphere measurement of popularity, while Yahoo! is taking an external measurement of blog popularity, looking into the blogosphere from the web at large (or at least from search.yahoo.com). If you’re a marketer, you probably like Yahoo’s measurement better.

(via Steve Rubel)

(That’s Greg’s image. Don’t have Google Earth running here, waiting for the OS X version. Impatiently.) Blogdigger seems to have found its niche with Blogdigger Local, and it’s a good one.

“I don’t think of us as being in the newspaper business,” said Mr. Simons, the editor and publisher of The Journal-World and the chairman of the World Company, the newspaper’s parent. “Information is our business and we’re trying to provide information, in one form or another, however the consumer wants it and wherever the consumer wants it, in the most complete and useful way possible.”
…
“We believe that journalism has been a monologue for so long and now is the perfect time for it to become a dialogue with our readers,” said Rob Curley, 34, the World Company’s director of new media. “We want readers to think of this as their paper, not our paper.”

Danny, from his exchange with Tristan:

Also, Google did say “of about” with the numbers it reports. That’s not an accident. They’re saying that this is an estimate. But no disagreement with me. If you put up a count, it would be nice if the count was as accurate as possible. Google’s have come under question.

Hmm. From what I’ve seen in Tristan’s data and my own testing, it’s Yahoo’s counts that ought to come under question, specifically for link: queries.

Danny to Tristan again:

The link: command is completely different than the site: command. The link command tells you nothing about the size of the index. As for a confirmation that all links aren’t reported, this past blog post from SEW gives you confirmation and this page on Google mentions links are only a sampling of what Google knows although this other Google page fails to make this clear.

link: and site: are very different, that’s true enough. And maybe the link command doesn’t tell you much about the size of an index, but if link collection methods are similar between Yahoo and Google (and why wouldn’t they be, it’s a relatively easy part of the whole game), then the counts ought to be similar. But they’re not, not by a long shot.

By the way, a big thanks to Tristan for posting his studies and kicking off this discussion. Most of us don’t take the time to do analysis of that depth to support our opinions, and to post the entire method and dataset so others can reproduce it, shoot holes in it, go off on tangents from it.

(I stumbled onto Danny’s post via John Battelle)

• For the query link:boingboing.net, Google reports “about 40,700 results”, i.e. pages in its index that link to BoingBoing.net
• Technorati has indexed 23,358 links to BoingBoing.
• Yahoo’s results claim that it has indexed about 1,320,000 pages linking to BoingBoing.

Interestingly, Technorati is the only one of the three that gives the same count whether the link query includes www before the domain name or not (I happen to think that’s the correct behavior).

So much for Tristan’s method, which is transparent and easily reproducible. The picture that emerges is that Technorati’s coverage of the blogosphere is worse than Google’s, which in turn is [much] worse than Yahoo’s. By the way, Tristan’s post has more depth than is relevant to this post, and it has some interesting statistics that pull apart this data. Read it.

Anyway, Dave smelled something fishy in Tristan’s data (he’s onto the right question, but he goes after a red herring and misses a different, interesting feature in the data):

… I believe that Tristan’s analysis begs a question that hasn’t been asked yet: How accurate are the numbers that search engines report about the size of their result sets? … For example, when you search for all the results for “Tristan Louis” on Google, it reports “about 575,000″.

Whoa, hold it. That’s a keyword query, which means Dave’s now running a different experiment from Tristan’s, which uses link queries. I recommend you read Dave’s entire post, but from this point forward, he’s on a different track, using keyword queries instead of link queries throughout.

Dave’s objection is to the limit on “viewable results” that Yahoo and Google implement (Technorati doesn’t). Both Yahoo and Google only serve up to about 1000 pages of a results set. Crunching through 1-N results not only gets more expensive for higher N, but the value for the user falls off pretty rapidly after a while. And as a bonus, this limit keeps pranksters with robots from chewing up bandwidth by paging through millions of results, wreaking havoc on caching. Not to mention that nobody wants to wade through more than a few pages of results anyway, instead of just rephrasing the query to get better results. [Someone should do something about the excessive recall of these keyword search engines...] Anyway, the 1000-page limit is an interesting discovery, but it’s an obvious optimization. The results count given at the top of the page (results 1-10 of 40,700) is of course an estimate, again as an optimization (getting exact counts from massively distributed indexes isn’t free, and who needs an exact count at this level of recall, anyway?)

So, the limit on viewable results is very straightforwardly explained as an optimization, benign to the user experience. I don’t think there’s anything to get worked up about in only being able to see the first 1000 results for a query. That’s what query refinement is for. The interesting thing is the estimated total number of results, specifically of link queries.

When I went through Tristan’s original experiment and ran some link queries, it became pretty obvious (as if it wasn’t obvious in Tristan’s post) that there’s something weird about Yahoo’s method of estimating the total number of results. The practice of estimating the total number of results (as opposed to computing it precisely) is a necessary optimization in a search engine that wants to scale to Google or Yahoo scale, and the estimated results counts seem plausible on Google and Yahoo for keyword queries. Counts from both engines were plausible (and within an order of magnitude of each other) for most keyword queries that I tried. But for link queries, that’s not the case. Let’s look again at the estimated total results counts for pages linking to BoingBoing:

• Google, about 40,700
• Technorati, 23,358
• Yahoo! , about 1,320,000

Now, the fact that Technorati found only half as many links to BoingBoing as Google isn’t a big deal and shouldn’t give Technorati an inferiority complex. A sizeable chunk of the links may be from sites that Technorati isn’t indexing because those sites aren’t blogs or don’t use ping services that Technorati is monitoring. Also, Technorati’s index isn’t as old as Google’s and other factors like multiple links per page (to the same blog) make the comparison even more difficult. In any case, the difference between Google and Technorati is relatively small (if the Technorati team spends some time on the back end now that the new UI is up, they’ll narrow that gap). What’s interesting, however, is Yahoo’s estimate for the number of results for this particular query. At 1.3 million, it’s about 30x larger than Google’s count and 60x larger than Technorati’s. That seems implausible to me, and it looks like some wacky calculations are happening in Yahoo’s estimation of results count for this query. For several blogs I tried, Google’s results count is plausible and roughly 2-4x Technorati’s, whereas Yahoo’s is, well, out there. Here are small, medium and medium-large examples (we covered extra-large above, with BoingBoing) :

annikrubens.de:

• Google, about 52
• Technorati, 29
• Yahoo!, about 318

blog.jackvinson.com:

• Google, about 228
• Technorati, 47
• Yahoo!, about 2,760

battellemedia.com:

• Google, about 10,300
• Technorati, 1,723
• Yahoo!, about 647,000

I told you, it’s wacky. Tristan’s conclusion is that Yahoo! is more focused on indexing the blogosphere and has more data. That may be true. But these counts are so far out there that I can’t help but think there’s a problem with the way they’re calculated. So there. Fix it. I may check

And, to end on a nitpicky note … as I mentioned somewhere above, if you add or subtract www to the link: query, both Google’s and Yahoo’s total counts jump around like crazy. I don’t know about you, but in this context of searching WWW content, I think www should be treated as a special hostname and equivalent to the domain, i.e. www.domain.com == domain.com.

Search engines are so powerful. And they are so pathetically weak.

He goes on to lament how ill-suited today’s KWS engines are at answering questions. His use case is trying to find figures for California’s school spending in their historical context and/or relative to other states’ school spending, and he finds no satisfaction from “normal search tools”.

… When I finally called an education expert on Monday, she gave me the answer off the top of her head … after I’d wasted what seemed like hours over the weekend with normal search tools …

Fallows casts the problem in terms of automated question answering and cites several projects working in that area, one of them a federal intelligence project named ACQUAINT, and the others web-wide efforts ranging from shallow question-answering technologies like Ask.com (now augmented with search refinement tools) and meta-search clustering like Clusty.com.

But there is another way to cast the problem, an alternative metaphor to the web as giant library or file cabinet. The web’s billions of pages of content are uploaded into the giant file cabinet by people, including an astounding number of experts in a vast array of subjects. Even if the web as seen through the scratched lenses of a search index can’t find the answer, or even if the answer isn’t even out there, there’s an expert out there who knows the answer, as Mr. Fallows realized.

Bloggers make up a rapidly growing population of experts who contribute content to the giant library. There are now somewhere between 9 million and 12 million blogs, dealing with a mind-boggling diversity of subjects, and with real depth in many of those subjects. We (our industry and our government) should push full-speed ahead in research on automatic question answering (our security may depend on it, as Mr. Fallows points out), but we should remember that the experts are already there. We just don’t know høw to find them yet, unless we have “education experts” in our address books.

Blog search is heading in the direction of web search, using the file cabinet metaphor and adding on top of that the stream of web pages metaphor, which is how RSS is most often treated and indexed. The unique nature of blogs, e.g. the 1-to-1 relationship of author to content and the historical record of expertise amassed by each blogger over the course of many posts, should be mined to provide ways to find answerers, not just answers.