July 29th, 2008

pirate ember

syncitems response times out if the list is too long


I was trying to figure out a problem with the LJ-SEC tool, and I found something interesting.

I have nearly 8,000 entries. When the LJ-SEC tool tries to use syncitems with no start date, the call times out.

EDIT: After investigation, the problem was not the number of entries, but a result of using the LJ built-in Edit Journal Privacy tool. For more details please refer to this post.

Just wanted to report this behavior somewhere in case it's undesirable. In the meantime I'm looking into other, more incremental, ways to get a journal's history.
pirate ember

Problem calling syncitems with no start date on an older journal

This is a revision of a previous question I submitted, with more information.

IMPORTANT EDIT: I found that the problem is not what I thought it was, and the error should now be easy to reproduce. The problem occurs when the built-in edit journal privacy tool is used. I have edited this entry to reflect this fact, and removed misleading references to the problem occurring on "large journals."

When the LJ-SEC tool tries to download my journal history (via a syncitems request) the server never responds.

I created a simple test journal to determine the cause: oak_and_sage. This journal has only seven entries. I did many things to this journal, and tested LJ-SEC after each one; the syncitems call began timing out immediately after I used the built-in edit journal privacy tool to change all public entries to friends only.

I've done a little debugging (since LJ-SEC is open source) to determine what's going on. The server is failing to respond to the initial syncitems call (when it has never done a sync before).

Here is the LJ-SEC code to transmit a web call to LJ:

Collapse )

I wrote a log of the timed-out transmission (on sithjawa) to a file. Here's what I found.

Collapse )

Is this expected behavior? What should be done to prevent it?

EDIT: I found the following in the protocol guide: (link to actual page in the guide)

syncitems. If you are trying to download someone's entire journal, this is the mode to use. This mode is the only way you can account for edits that the user has made to their entries without using your client. This is also the most efficient way of downloading entries, because the server will send you a bunch at a time (say, 100). This mode is used in conjunction with the appropriately titled syncitems client protocol mode.

Looking at the LJ-SEC code, it appears to be following the procedure outlined in the pseudocode example for properly downloading all items in a journal. This makes the timeout even more puzzling.