August 06, 2007 | Monday

Yahoo’s open source move against Google

By Jackie Danicki - Blogger  in News |Search Engines |Google |Yahoo |Search Technology

Tim O’Reilly, a prominent figure in the tech industry (he popularized the term Web 2.0), has some interesting analysis of Yahoo’s recently announced dive into open source. The project is called Hadoop, and Yahoo is throwing its weight behind it in a big way, even bringing in big tech name Doug Cutting to support Hadoop. What Hadoop is exactly - “a Free Java software framework that supports distributed applications running on large clusters of commodity computers that process huge amounts of data” (named after its creator’s child’s cuddly toy) - is less important than what Yahoo’s support for it signifies.

First, it indicates a kind of competitive tipping point in Web 2.0, where a large company that is a strong #2 in a space (search) realizes that open source is a great competitive weapon against their dominant competitor...If Yahoo! is realizing that open source is an important part of their competitive strategy, you can be sure that other big Web 2.0 companies will follow. In particular, expect support of open source projects that implement software that Google treats as proprietary.

Hadoop is also expected to give Yahoo vital “geek cred” with the brainiacs they need to attract as employees in order to thrive.

But according to Cutting, web search isn’t Hadoop’s raison d’être.

“Where Hadoop really shines,” says Doug, “is in data exploration.” Many problems, including tuning ad systems, personalization, learning what users need—and for that matter, corporate or government data mining—involve finding signal in a lot of noise.

For those of you who are technically adept enough to want to know more about the details of Yahoo’s involvement with Hadoop, Yahoo blogger Jeremy Zawodny has written about it here. For the rest of us, it is enough to note that Yahoo’s confidence in the importance of open source is obviously high - high enough to be employed as a tactic to compete with Google. Things will get really interesting when, as O’Reilly predicts, other big Web 2.0 companies follow. It is anyone’s guess as to whether Google can look to the future and watch its back at the same time.

Add your opinion

Name: Email: (will be encrypted, to protect against email harvesters) Location: URL:
Save your details in a cookie

Notification of follow-up comments

Enter the security code