Anatoly Skoblov (skoblov) wrote in lj_dev,
Anatoly Skoblov

How to download all posts from journal

I try to make local copy of my friends posts for 2012 using flat API.

There are two ways to do this:

1. Get posts ID using mode=syncitems, and get posts later using mode=getevents. This method works only for my journal and communities. If I try to request data (mode=syncitems) from other journal, server answers "Client error: Don't have access to requested journal"

2. Downloads posts using mode=getevents & selecttype=syncitems & lastsync=.... This method works for any journals, but I can't find source of good value for lastsync time. If server returns first 100 posts, I need to continue using modification time of last received post. But LJ does not include it in answer (it present in mode=syncitems answer only)
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 1 comment
I've always found that the easiest way to pull that sort of data is to create a custom S2 style which renders a Friends Page (or an individual entry) in the form of an XML document, then use PHP to grab the data and chuck it into a MySQL database..

Granted, I've only done it using my own journal and communities (first by rendering the Month View in a way that gives me a collection of entry IDs, then grabbing each entry in turn), which gets around any limitations on the extent to which one can go "back" through successive friends pages, but it ought to all be possible..

Also allows you to grab comments too, if you feel so inclined..