The Powerset Demo Day
Ero Carrera (ero) <erocarreragmailcom> Friday, June 29 2007 08:58.00 CDT


I just had the luck of being invited, together with forty other people, to the first public demo of what the guys at Powerset are building up. The demo was in their headquarters in San Francisco. It was fairly impressive, to put it mildly.



They showed demos of all what theyve been blogging about, like this or this and some other new applications.

I also had a chance to talk to some of the core Ruby developers, the search ranking engineers and the linguists working there and its really an incredible team. One can smell the excitement floating in the air about what they do.



Steve Newcomb and others commented on different sides of their technology and the company itself. Mark Johnson introduced a just out of the oven Powerlabs, to give a taste of whats coming in September. Itll be possible to generate mash-ups using their natural language processing and understanding technology which, in my humble opinion, I think its going to truly open the doors to a new generation of clever semantic tools. They aim to being really open about their system and to let people interact with it. Another side in which they also want to be open is in their contribution back to the open source community from which they build a lot of their infrastructure.



From seeing their demos, it really looks like they are are taking search, among other things, to a new level by not indexing keywords but actually indexing "concepts" that they extract by semantically parsing the searchable data. If they find "dog" in a sentence they will associate also "mammal", "animal", "pet" allowing for real abstraction when performing searches.
They combine that with normal search ranking techniques to obtain impressive results.

It really creates new ways of interacting with the data to be searched. Queries like "What politician died from a disease?" or "What disease killed a politician?" work flawlessly even when there are no references to die or kill in the text. Their natural language engine understands that "died from" equals to "killed by" and relates to "deceased" or "pass away". It really knows about those concepts abstractly and the semantic relations in the search query.

Powerlabs, the social site aimed at letting people play with their technology that will be launched in September, already has more that 10000 members signed up , so definitely theres a growing community interested in their developments.

During the Q&A; sessions some very interesting topics popped up, like support for multiple languages and detection and resistance to spam (or text created by different models in order to appear human generated). Also, the "understanding" that they obtain from parsing a sentece could allow to better spam filtering, not just by spotting more or less likely-to-be-spam words, but actually detecting incoherent meanings or just uninteresting topics... just imagine having messages in your inbox automatically clustered by their real meanings, without having to specify a single rule (emails dealing with this, emails dealing with that...). The applications are endless...

Im dying to play with it...

Comments
Posted: Wednesday, December 31 1969 18:00.00 CST