You might've also noticed a lot of CVS activity dealing with "memcached".
You might've put the two together. :-)
Yes, we're now running a bunch of memory cache daemons. In the LJ code, we always try to get things first from memory, then go to the database only as a last resort. (and when we do, we put it in memory)
The API is essentially dictionary: set and get. (there's some stuff like add/replace/delete/get_multi/etc...)
The servers have bounded memory usage, and discard infrequently used items when memory is needed.
Any memory server can go up or down at any time, and the LJ code automatically adjusts.
I wrote the initial protocol, client and server, but the Perl server was way too inefficient. avva rewrote the server in C, using a 2.4 backport of Linux 2.5's epoll() system call, Judy, and my constant nagging about nitpicky performance things. End result: it consumes hardly any CPU at all.
So, the server works. The APIs work. (oh, and it lets you transparently store objects in addition to regular scalars, so maybe we need more buzzwords: "distributed, fail-safe, object caching system".)
The next step is making more stuff use it. So far we cache:
-- logtext, talktext
-- logprops, talkprops
-- user login sessions
-- translation strings
-- userpic widths/heights/owners
This is just a start.
My end goal is to cache everything possible. I want pages to serve entirely from memory whenever possible.
We'll be buying as much memory as needed to keep cache hit rates high. (and luckily it's damn cheap.) We'll also be converting database servers we no longer need (due to this change) into memory cache servers.
server is in wcmtools:memcached/. client APIs are in livejournal:cgi-bin/LJ/MemCache.pm.
Update: I spoke too soon. Our machine with 12 GB of RAM died, exposing a shortcoming in our dead-host detection, making the site worse than no memcache. Fixing.