Brad Fitzpatrick (bradfitz) wrote in lj_dev,
Brad Fitzpatrick

Proposal: Syndicating RSS

LiveJournal should let you add an RSS channel as a friend.

Conceptually this is easy. ("just do it!") Mark's been proposing this for awhile.

Here's a plan for implementation:

  • User wants to add an RSS/RDF/FOO-XML channel. Instead of an "add friend" page they go to an "add channel" page. (we'll need a new protocol mode for this, or the "add friend" username will be allowed to start with http://)
  • User enters URL of XML data source in text field or selects from a list of popular channels and recently added channels.
  • Server checks its new syndicated table to see if we're already syndicating that URL. The URL will be indexed, from which it can find the LiveJournal username. (Remember that our "user" table is really a "journal" table)
  • The syndicated table will keep track of the source URL, time last checked, time last modified, and other info, like a summary column of the number of people that list it as a friend (I say 'summary' since that data is always available by joining on the friends table... but we could index the summary field to find popular channels)
  • If user's syndication URL doesn't exist, we grab the XML, see if it's valid, and create a new syndicated user on the fly. The usename will be automatically generated from the title of the source channel probably, or we'll let the user pick, perhaps. The user journaltype will be "S".
  • We have a background process that polls RSS files every so often as needed, sending If-Modified-Since headers and all that jazz, being HTTP friendly, and checking to see if new <item>s (or whatever that schema uses) exist. If so, we generate new posts for that user and record seeing that item so we don't generate a new post in the future.
  • User goes to their friends list and there it is ... the syndicated info. And hell, we could get fancy and generate a userpic for the syndicated user too from the image URL in the RSS file.
I'll write this all abstract enough that we can support any syndication method that pops up in the future. Initially we'll support all flavors of RSS and RDF.

Objections? Questions? Better ideas?


  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded