Actually, I started thinking about this a while ago when this email ended up in my inbox. It references a New York Times article on the subject and suggests some potential research topics.
I'm curious if anyone has thought about these ideas/functionality, and if any work has gone into this with respect to LJ. I think it would be neat to fiddle with.
With the newest versions of mySQL there are now a pretty good way to do fulltext indexing, and it should only get better as this feature is deveoped. If one were to combine this with some interesting ranking metric you could probably build a pretty useful search engine.
What say you?