January 5th, 2002


I've been running the script which modifies the database for clusters on a copy of the real LJ data. I knew it'd take a long time, but it's painful actually watching it run. I've been making it faster, but it's still going to take hours to convert.

Which means when we're ready to run it, there's going to be a lot of downtime.

We need two versions of the code very soon here:

Old code/schema + Read-only aware
Any place we write to the database, check first if $LJ::READONLY is set and complain, or just don't make available the option to write anything in the first place. I figure instead of a "Site down" message for a few/many hours, we can make available a read-only copy of LJ.

New code/schema
Once the db conversion script is done, we replace all the old code with the new code which knows about the new schema, then sit back and relax. This new code could optionally also support $LJ::READONLY, though that won't be as critical. Yet another advantage of clustering the databases is that any necessary ALTER TABLEs in the future are fast: instead of doing one operation on n units of data, you do the operation on s clusters in parallel, each with n/s units of data. If we keep each cluster small (as we will), alters are relatively painless.

I'm putting cluster patches at:


the first one did most the work in perl-space. clusterfun.2 is faster, doing more on the db side, but mysql or DBI drops the connection in the middle of the 17GB logtext.MYD alter, so I'm switching that back to be more perl-ish, doing it more in chunks.

If you're interested in working on either the $LJ::READONLY stuff or converting areas of the site to the new DB schema, let me know.

initial/mandatory friends

Mark proposed a long time ago about having two things in ljconfig.pl:


All new accounts will get the initial friends ("news", etc, etc...)

Mandatory friends (more controversial) would be a subset of those which couldn't be removed, unless you had the "removemandatory" capability (which paid users would have). So basically, Mark wanted a way to force free users to read at least "news", or something like a "freeusers" community.

Ignoring the idea of whether or mandatory friends is a good idea, could somebody implement this? At least the initial friends if you're extremely opposed to aiding in any way the mandatory friends?

Shouldn't take long to do, but I have another project at the time. Clean place to put it: ljlibpl: LJ::create_user.