Brad Fitzpatrick (bradfitz) wrote in lj_dev,
Brad Fitzpatrick
bradfitz
lj_dev

MyLJ: syncitems mode

Okay, I created a new table (syncupdates) to keep track of when new items are added and modified, and the indexes are setup on that table to make it really quick to say, "SELECT ... FROM syncupdates WHERE userid=$userid AND atime > $date". (I just realized that indexes don't show up in the schema browser... hmmm...)

Anyway, I then changed the protocol modes 'postevent' and 'editevent' to keep the syncupdates table updated.

Then, I added the "syncitems" protocol mode, which returns a list of the items that have been modified since a given date. (the date of your lastsync).

Currently all items are prefixed with "L-" and then the itemid number. However, in the future we can sync to-do items, comments, etc.. all using this same table and protocol mode. So your logic in your program should be to ignore prefixes which you don't understand. If version 1 of your sync client doesn't understand T- prefixes, don't worry about it...version 2 of your client might understand them, and it could then send the syncitems request later with an empty date to get all the T- items and download them.

Anyway, here's a URL that does the new mode:

http://www.livejournal.com/cgi-bin/log.cgi?user=test&password=test&mode=syncitems

There is only one parameter to "syncitems", the "lastsync" date. However, in the above example it's excluded which means everything. However, it's not really everything... it'll give you up to sync_count items ... if sync_count == sync_total, then you're done (see the bottom of the output from the above URL)

The reason it's limited is because running it on my journal gave back a half meg response.... I want this this to run nicely even on users with modems. Plus, the client has a lot of work to do now, getting those 500 items anyway. (I actually had the limit at 1,000, but the test account only has 760 items so I brought it down to 500 for testing)

Now, say our client got those 500 items. Now we'd do:

http://www.livejournal.com/cgi-bin/log.cgi?user=test&password=test&mode=syncitems&lastsync=2000-07-16+11:45:24

And there are the remaining 260. sync_count==sync_total so we're done. (well, after you fetch them all)

Your client's store shouldn't update its "lastsync" date until after it has done getevents and got the items.

Also, each store should have a separate lastsync date for each item prefix type... that way when version 2 of your protocol runs, it knows it hasn't ever downloaded any comments or todo items, so it can rerun the sync from the beginning only caring about those new prefixes.

Next thing I have to do is just add a new select type to getevents to let it accept a bunch of arbitrary itemids to return.

Um, this is already long. I'm going to stop.

Questions?
Subscribe
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 6 comments