Evan Martin ([info]evan) wrote in [info]lj_dev,
2003-08-25 16:59:00
  • Previous
  • Memorize
  • Share
  • Next

AOL blocking LiveJournal image loading

Here’s the scoop:

Find a random image on an AOL member’s page.
Retrieve it with a Referer of something other than LiveJournal.com:
% curl -s -i -H "Referer: http://www.livejournalx.com" 'http://hometown.aol.com/dvyfacs/myhomepage/old+friends+1.jpg' | head -1
HTTP/1.1 200 OK
Yay, we got the image!

Now try it with LiveJournal:
% curl -s -i -H "Referer: http://www.livejournal.com" 'http://hometown.aol.com/dvyfacs/myhomepage/old+friends+1.jpg' | head -1
HTTP/1.1 404 Not Found

Hey, where’d it go?



...and what can we do about it? :(

I guess it’s their jurisdiction, but it’s a pretty low thing to do, especially because it appears to their users like it’s our fault. More interesting is that other Referers (for example, using evan.livejournal.com) doesn’t trigger it.

  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    switch

    Your reply will be screened

    Your IP address will be recorded 

     
    Help
  • 118 comments
Previous
← Ctrl← Alt
  • 1
  • 2
Next
Ctrl →Alt →

[info]dottey

August 25 2003, 17:05:10 UTC 8 years ago

I would hope it is just because many other websites have blocked such offsite linking, but it is mostly obvious this has more to do with the new AOL Journals feature they've just announced.

*sigh*

[info]redfarmer

August 25 2003, 17:50:16 UTC 8 years ago

There have been several support requests saying they could load images on uJournal but not LiveJournal. I think it has more to do with a specific grudge against our site rather than against all journaling sites. Besides, if they were to block all journaling sites, the journal sites (LJ, uJournal, Blogger, etc.) could file a class action law suit for AOL attempting to create a monopoly (like they don't already).

[info]krellis

8 years ago

  • Loading...

[info]amokk

8 years ago

  • Loading...

[info]akeller182

8 years ago

  • Loading...

[info]leora

August 25 2003, 17:05:37 UTC 8 years ago

It's worse than that. Most of their low level tech Support people don't seem to be aware that they blocked LiveJournal. So, they're often telling AOL users it is our fault. Their upper level tech Support does know though.

Fun, fun, fun.

And now we have a request from an AOL user that suddenly they stopped getting LJ emails. They say AOL did just add some new spam filters, so that may relate.

It almost makes you think that they don't like us...

[info]burr86

August 25 2003, 17:17:27 UTC 8 years ago

Well, we didn't really like THEM, so ... :p

[info]pne

8 years ago

  • Loading...

[info]bery

8 years ago

  • Loading...

[info]leora

8 years ago

  • Loading...

[info]mr_dark

August 25 2003, 17:19:26 UTC 8 years ago

That begs a very interesting question:

What effect will it have on LJ if AOL decides to basically blackball us from their systems? More importantly, what recourse do we have if they decide to do so? I mean, it isn't illegal as far as I know to simply disallow e-mail, image linking, etc. from a certain domain as an ISP.

They've certainly tried to block alternative chat clients like Trillian more than once.

[info]leora

August 25 2003, 17:24:05 UTC 8 years ago

My guess. Some AOLers will leave LJ and some will leave AOL.

But they've pulled this before with other sites, and in my experience, it's not all that harmful to the other sites.

LiveJournal is big enough and strong enough that it could survive with no AOLers. Plus, it's not trying to make as much money as possible by any means.

So, if some AOLers leave, where's the big harm? LiveJournal will continue and still be popular.

Personally, if I were an AOLer (which I'd never be, I admit), I'd be furious at them for cutting off some of my services without telling me. But that's up to individual AOLers to decide. And for them to decide if they want to take action against their ISP for it.

[info]mart

8 years ago

  • Loading...

[info]evan

8 years ago

  • Loading...

[info]ibrad

8 years ago

  • Loading...

[info]volantwish

8 years ago

  • Loading...

[info]leora

8 years ago

  • Loading...

[info]volantwish

8 years ago

  • Loading...

[info]akeller182

8 years ago

  • Loading...

[info]leora

8 years ago

  • Loading...

[info]akeller182

8 years ago

  • Loading...

[info]leora

8 years ago

  • Loading...

[info]ziemkowski

8 years ago

  • Loading...

[info]seattlesparks

8 years ago

  • Loading...

[info]deahamlet

8 years ago

  • Loading...

[info]lisanys

August 25 2003, 17:38:32 UTC 8 years ago

With Opera, I have referrer logging turned off (always have and will).

The image you pasted loaded just fine.

I said:
GET /dvyfacs/myhomepage/old+friends+1.jpg HTTP/1.1
User-Agent: Opera/7.20 (X11; Linux i686; U)  [en]
Host: hometown.aol.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Cache-Control: no-cache
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers

hometown.aol.com said:
HTTP/1.1 200 OK
Date: Tue, 26 Aug 2003 00:37:16 GMT
Server: Apache/1.3.26 (Unix) Resin/2.0.5
Keep-Alive: timeout=10, max=700
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: image/jpeg

[info]jtek

August 25 2003, 17:41:37 UTC 8 years ago

Same here. Works great when people post images from GeoCities (who block ALL offsite linking) on their journals.

[info]uncaring

8 years ago

  • Loading...

[info]jackola

8 years ago

  • Loading...

[info]lisanys

8 years ago

  • Loading...

[info]gwferguson

August 25 2003, 17:52:50 UTC 8 years ago

Well, that explains a peculiar problem I've been having: I'm unable to link my own images or websites on CompuServe, which is now owned by AOL, to LiveJournal.

Poopie-bears to CompuServe!

[info]westerdrumlins

August 27 2003, 11:48:31 UTC 8 years ago

and as of last night, i wasn't getting any of my comment replies to my compuserve address, so i've had to start using my hotmail one, which i'd rather not do as my cs address i use for all "internet" things such as lj and other mailing lists...

[info]vfc

August 25 2003, 17:56:25 UTC 8 years ago

Does anyone think it might have something to do with the fact that AOL is starting its own journaling service?

[info]flashfire

August 25 2003, 18:08:45 UTC 8 years ago

Yes, I think that thought's probably crossed the minds of a few people.

[info]skyknyt

8 years ago

  • Loading...

[info]insaint

8 years ago

  • Loading...

[info]skyknyt

8 years ago

  • Loading...
...and 3 more comments... from [info]akeller182, [info]arturus_magi, [info]thatsmytrunks Expand

[info]mvivei8828

August 25 2003, 18:17:42 UTC 8 years ago

aol is sucha crock of shit. their service is slow, and unreliable and the email filter is a piece all on its own. im glad that i switched to msn

[info]insaint

August 26 2003, 05:54:51 UTC 8 years ago

The lesser of two evils...

[info]diffrentcolours

8 years ago

  • Loading...

[info]insaint

8 years ago

  • Loading...

[info]diffrentcolours

8 years ago

  • Loading...

Deleted comment

[info]mvivei8828

8 years ago

  • Loading...

[info]bradfitz

August 25 2003, 18:37:39 UTC 8 years ago

I hearby propose that we publicize and distribute a fre personal proxy for Win, Mac, and Linux that does nothing but remove all Referer headers on requests that look like images. It'd be a good mix of letting people load images from anywhere and still give sites navigation path stats.

I have a Perl to Win32 exe compiler license (ActivePerl), so this is all really like 20 lines or so. Mac OS X and Linux can just run the Perl.

Evan, Whitaker... we can work on this tomorrow.

[info]kaoslord

August 25 2003, 18:48:26 UTC 8 years ago

slickness. i'd love to test.

[info]bobert225

8 years ago

  • Loading...
...and 17 more comments... from [info]darkpixel, [info]mart, [info]meowpurrr… Expand

[info]uncaring

August 25 2003, 18:45:12 UTC 8 years ago

...and what can we do about it?

Haven't thought through this too much, but...

1. Set up a system to proxy image requests (let's call it imageproxy),
2. Before serving up a LiveJournal page that links to an image on an AOL server, re-write the image URL to be something like http://imageproxy.livejournal.com/get/RealURLToImage.
3. Have the proxy strip out any referer headers (which I've never set up but am assuming is possible).

Downsides:

1. Easily defeated by AOL -- block all requests from imageproxy.
2. The LJ network is now consuming more bandwidth by sucking down all these AOL images and then serving them to your users.
3. Others?

Upsides:

1. Transparent to users (e.g. they wouldn't have to modify URLs to their images).
2. Makes AOL images viewable to LJ users (i.e. solves the original problem).

Thoughts?

[info]knew

August 25 2003, 18:47:34 UTC 8 years ago

Now would you believe it, Netscape's (now, who would they be owned by now) "My Webpage" site is also doing the same thing.

[info]flashfire

August 25 2003, 18:48:19 UTC 8 years ago

Anyone think this would be reasonable to bring up in [info]news, in addition to the [info]lj_biz mention? I'd think this information should be spread around quickly just so more people understand what's going on.

[info]darksoul

August 25 2003, 19:19:00 UTC 8 years ago

the time for playing "cowboys" is long since past...

Am I the only person who thinks "so what"?

All of the free hosting services have blocked remote loading of images. Why? Because of the massive drain on their servers that LJ has posed. Sure, they blocked remote loading to anywhere, but it was still done mostly because of LJ users.

I think that going off half-cocked and helping users be able to circumvent AOL's ban on remote-loading to LJ will only serve to disintegrate any notion that this is a legitimate business run by an intelligent and legitimate businessman (Brad). Instead, the owner and his employees, not to mention the users, will look like a bunch of hacker hoodlum degenerates who want nothing more than to "screw the man" and "hack the planet".

If AOL wants to block remote loading from LJ, let them. It's their perogative as a business. When users come to us asking why their images won't load, explain to them that AOL is blocking images from remote loading to LJ and to request further information from their higher level tech support people as to why. That's the sensible, business-minded route.

[info]leora

August 25 2003, 19:26:15 UTC 8 years ago

Re: the time for playing "cowboys" is long since past...

There are some merits to that.

The difference between free sites blocking remote loading and AOL doing so is that the AOL users are paying for a service they are not receiving.

But really, they should be complaining to AOL and either demanding the services they paid for or finding a new ISP. It is, of course, up to each individual what they choose to do.

[info]darksoul

8 years ago

  • Loading...

[info]leora

8 years ago

  • Loading...

[info]luminifer

8 years ago

  • Loading...

[info]the_dibbler

8 years ago

  • Loading...

[info]darksoul

8 years ago

  • Loading...

[info]the_dibbler

8 years ago

  • Loading...

[info]darksoul

8 years ago

  • Loading...

Deleted comment

[info]darksoul

8 years ago

  • Loading...

Deleted comment

[info]darksoul

8 years ago

  • Loading...

[info]jaybonci

August 25 2003, 20:33:08 UTC 8 years ago

URL shortening sites might solve your problem.

There are a pile of URL shortening services out there, and a referrer from there should solve your problem, correct?

The Yahoo directory on this, along with CPAN's WWW::Shorten which is an interface to a pile of these sites.

That should solve the problem, at least in the short term, but then yes, it's an arms race.

--jay

[info]mart

August 26 2003, 04:43:45 UTC 8 years ago

Re: URL shortening sites might solve your problem.

I don't think this can work. When a server responds with a “redirect” response the client requests the new URL using the same referrer as the initial request, since redirecting isn't really referring, it's saying “actually, what you're looking for is over here…”

[info]quirrc

August 25 2003, 22:23:02 UTC 8 years ago

i'd like to note that aol is blocking not just images but all files with livejournal referrer like clicks from livejournal pages. for example here http://www.livejournal.com/support/see_request.bml?id=147731

[info]knew

August 25 2003, 22:35:52 UTC 8 years ago

now that's just totally unacceptable. blocking images i can understand, but blocking everything is completely unreasonable.

this isn't the only thing that AOL has been up to recently either. it seems they've completely blocked all email from a particular hosting company, and ignored a temporary restraining order requiring them to stop the blocks. more on this can be found at http://www.hostreview.com/news/news/030822CIHost.html

[info]bradfitz

8 years ago

  • Loading...
...and 2 more comments... from [info]the_dibbler, [info]_glittergirl Expand

[info]dawn_key

August 25 2003, 23:49:57 UTC 8 years ago

AOL is being dumped by Time-Warner. They have lost over a million users in the past year and have no veteran managers left. This is just one more indication that they are desperate and floundering and illustrates why the stock is taking a nose dive.

AOL is seen on Wall Street as "training wheels" to the Internet. Most people don't need or want training wheels any more. People are switching to high speed connections and AOL's relatively poor content isn't worth keeping for the price they charge. So if it makes anybody feel better, they are likely not going to be the biggest kid on the block for too many more years.

[info]suppafly

August 26 2003, 09:43:03 UTC 8 years ago

haha.. great troll

[info]fate_steps_in

August 26 2003, 00:00:36 UTC 8 years ago

*sigh* So that's why my LJ background isn't working.

[info]peachiegirl216

August 26 2003, 07:41:20 UTC 8 years ago

cute user pic. :)

THat is exactly why your background won't show up. I asked about my pictures not showing up on the Compuserve forums and they are clueless basically. But someone did post the link to this thread. I say we all contact AOL/CS and complain. Everything works fine on my Blurty and my mindsay journals. It's just here that things don't work.

[info]cranehaven

8 years ago

  • Loading...

[info]petfish

August 26 2003, 09:48:48 UTC 8 years ago

Heh. It's not really that I don't care, it's just that it's kinda amusing to see AOL fucking up again.

Speaking from experience, AOL has a tendency to pull shit like this. I can think of several occasions where they decided all Earthlink email was spam. One time it started with the dialup users, so I could see it as a script going awry at first, but then it expanded to everyone on the earthlink.net domain, and that was kinda wacky. I seem to remember something similar happening at another isp I worked for, that did hosting. Once it was email, but a few times it was blocked sites.

Every time, the blocking didn't last for more than a few days. A few calls from angry Abuse departments at other ISPs, a couple hundred calls from their own users, maybe the threat of a lawsuit or two, and they bring it down and never admit they did it in the first place. This situation could be different, but I wouldn't be at all surprised if it blew over like that.

[info]tau

August 26 2003, 14:59:42 UTC 8 years ago

I was told by an AOL staffer that if we change the image url to ftp://members.aol.com/username/photo.jpg (instead of http) that the images will come up. This seems to have fixed the problem.

I'm still annoyed enough that I'm leaving AOL and moving all my images over to a paid domain instead.

[info]insaint

August 26 2003, 21:54:59 UTC 8 years ago

I'm still annoyed enough that I'm leaving AOL and moving all my images over to a paid domain instead.

Good move.

[info]kutsuwamushi

8 years ago

  • Loading...

[info]evan

8 years ago

  • Loading...

[info]kutsuwamushi

8 years ago

  • Loading...

[info]bimonsradley

August 27 2003, 00:52:01 UTC 8 years ago

I hear with AOL 9.0 they are coming out with their own journal type thing, if this is the case this could be there reasoning behind blocking access.

[info]seanachie

August 27 2003, 07:15:54 UTC 8 years ago

You can troubleshoot here. This might help. http://webmaster.aol.com/

[info]jennilee_rose

August 27 2003, 10:07:39 UTC 8 years ago

Ok stupid question time:

Is it safe to assume that those using mywebpage.netscape.com (because they're affiliated with AIM users) is also a part of this blocking? I have my images there and mine aren't loading either.

Jennilee

[info]evan

August 27 2003, 10:10:08 UTC 8 years ago

Yes, they are doing it too.

[info]seanachie

August 27 2003, 12:39:35 UTC 8 years ago

[info]jettdelirium uses AOL to webhost her LJ photos I think. I'm confused. Did she find away around your problem?

[info]evan

August 27 2003, 16:11:19 UTC 8 years ago

It looks like our workaround is working for at least one person...

[info]cranehaven

8 years ago

  • Loading...

[info]mizanchan

August 28 2003, 14:56:50 UTC 8 years ago

I don't know...maybe I'm just hopelessly naive, but I would think if a site is going to block another site from image loading, it would actually contact the other site and explicitly tell them why this course of action was required.

I work in a computer store, and part of my job is garnering AOL signups (much as I loathe AOL). If you could see all the responses I get when I mention signing up for a free trial offer of AOL as we're instructed to, your heads would spin. Believe me. I could spend my whole shift asking people, and only get one or two signups, if that. Their PR sucks royally.

I believe that if AOL's trying to stifle its biggest competition by conveniently blocking image loading while it plans to come out with its own journal service, it's starting to turn into another Microsoft.

[info]pfak

August 29 2003, 16:44:29 UTC 8 years ago

Easy to fix.. "liveimage" service.



Simple, removes headers. You could even implement this directly into LiveJournal code with a simple regex.

youd do http://liveimage.livejournal.org/?url=http://evil.aol.com/site

Simple, and fixes your problem completley.

[info]kagekonjou

October 26 2003, 05:11:51 UTC 8 years ago

Re: Easy to fix.. "liveimage" service.

When you do a Location forwarder from PHP, it sometimes will send referrer tags. Same can be said for Perl, as well as any other web scripting language. The only TRUE way to go around this is to preform a GET procedure, grab the image, then paste its raw data.

[info]ac

August 29 2003, 17:50:50 UTC 8 years ago

There I am, reading Slashdot, when what do I see but...

[info]evan, I just wanted to be one of the first to say congratulations for making it on /. with such a well-written (and succinct) dissertation on the problem. Of course, based on the comments, it sounds like most people either didn't read it or didn't understand it, but I suppose that goes with the territory.

Hopefully, this public discussion of the problem will result in something from AOL. I guess we'll just have to see...

A.C.
******
Previous
← Ctrl← Alt
  • 1
  • 2
Next
Ctrl →Alt →
Create an Account
Forgot your login or password?
Facebook Twitter More login options
English • Español • Deutsch • Русский…