I thought a better way to keep it random but also prevent any one server from getting really loaded was to randomonly throw out 1 or 2 of them, using their load as a weight. (i.e... the most loaded server is most likely to be thrown out of consideration to handle a request when it comes in) This ensures that even the most loaded machine is still doing work when it's the most loaded.
I wrote the function to do this, ran it for awhile, it seemed to kinda work, but then httpds on the master started eating lots of CPU. So, I think I have a bug. Brownie points to whoever finds it!
/* Candidacy module for mod_backhand that returns everything but arg of the servers given, throwing out arg of them randomonly, weighted by their loads. */ #include "mod_backhand.h" int byDitchByWeightedLoad (request_rec *r, ServerSlot *servers, int *n, char *arg) { int remain = *n; static int loadsum = -1; int i; int tokill = arg ? atoi(arg) : 1; if (loadsum == -1) srand(time(NULL)); loadsum = 0; for(i=0; i < *n; i++) { loadsum += serverstats[servers[i].id].load; } i = 0; while (tokill && remain) { int kill_odds = serverstats[servers[i].id].load * (RAND_MAX >> 5) / loadsum; if ((rand() >> 5) < kill_odds) { servers[i] = servers[remain-1]; tokill--; remain--; } if (++i >= remain) i = 0; } *n = remain; return remain; }