?

Log in

No account? Create an account
May 15th, 2003 - LiveJournal Development — LiveJournal [entries|archive|friends|userinfo]
LiveJournal Development

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

May 15th, 2003

stuuupid robots (Teoma) [May. 15th, 2003|10:43 am]
LiveJournal Development

lj_dev

[boobay]
I just launched my own Travel Community based in parts on LJ and in the last couple days Teoma crawled thousands of pages from the Months view for months/years which do not exist, e.g. "GET /users/moonwha/1846/05/

today it went down to the 18th century even (17xx), and for some users even up into the 22th century, e.g /users/rigsby3/2161/10/ - the more users signed in the worse the problem got (right now teoma still crawls 1-2 of these urls per sec) google+grub don't seam to do that
The problem is that htdocs/view/index.bml always shows arrows for the prev + next months - even if there are no entries anymore for a user, I created a quick "fix" to:
a.) only show arrows if there are more (prev|next) entries and
b.) to skip months (e.g. jump from Jan directly to May if there are no entries in between.

in htdocs/view/index.bml:89

my $prev = "";
my $next = "";
my $db = LJ::get_cluster_reader($u);
$sth = $db->prepare("SELECT year, month from log2 where journalid=$u->{'userid'} and ((year<$year) or ((year=$year) and (month<$month))) order by eventtime desc limit 1");
$sth->execute;
if (($prev[0], $prev[1]) = ($sth->fetchrow_array)) {
$prev = "";
}
$sth = $db->prepare("SELECT year, month from log2 where journalid=$u->{'userid'} and ((year>$year) or ((year=$year) and (month>$month))) order by eventtime limit 1");
$sth->execute;
if (($next[0], $next[1]) = ($sth->fetchrow_array)) {
$next = "";
}
link5 comments|post comment

navigation
[ viewing | May 15th, 2003 ]
[ go | Previous Day|Next Day ]