January 18th, 2004

S1 in C

I rewrote S1 to be in two halves: generating a data structure with everything to be shown, and flattening it down based on their style (template) into the final result.

That by itself was minimal code... replacing fill_var_props() with a constructor for either an S1::Node or S1::NodeArray. Or a simple node which is just a scalar holding text, like $lastn_page{name} = LJ::ehtml($u->{name}.

Anyway, I then rewrote the flattening part 4 ways in Perl, and one way in C.

The 4 ways in Perl were:

-- via lots of regexps, like the old way.

-- with dynamically generated code blocks, one per template element

-- a mix of the two above, depending on if if the flattener was called recursively from a NodeArray's flattening. (generating the code block was expensive, so I figured it only made sense to do it if it was to be used more than once... being inside a NodeArray was the perfect heuristic)

-- Perl code that could be easily ported to C. (the prototype before I dived into C) This involved a first pass to calculate the total length. Whenever an attribute was used with a transform, the attribute had to be flattened first, if it wasn't a text node. Then that's cached away inside the node, so the second stage could use it. After the total length is calculate, a scalar buffer of that length is allocated (my $buf = "-" x $len) and the recursive stage begins, always taking $buf and a position into it to copy its node. It returns the position it ended at. Lots of paranoia about actual lengths not matching expected lengths from the previous stage.

Then the C way, as described above in the last Perl version.

The C version was the fastest, but only 7-9% faster than the old code. Not sure if it's worth it.

I had fun, though... I'd never written any Perl/C code before, and my code compiled and worked on its first run. It's definitely not as hard as I'd always expected.

The experimental version of ljviews.pl is:

Attached to this bug about using C to make fast versions of hot functions:

It contains all the different variations. Search for "render_node", "gen_node_code", "node_length", "render_node_recurse", etc. That's the new stuff.
amused, happy
  • mart

Rudimentary Offsite User Auth Mechanism

There was a recent post about ways to validate if a given person is the LJ user they claim to be. One that intriegued me was temporarily adding a GUID to the interests list and then parsing the userinfo page. This leads me to the following proposal, which is a less hacky version of that.

The mechanism is simple, really. The external site provides the user with a link to a URL like http://www.livejournal.com/tools/auth_add.bml?token= followed by some identifier which is just an arbitrary string generated and stored by the originating site. LiveJournal will generate a page with a button to confirm the token and, if the user confirms, the token will be stored on the server related to the userid of the remote user.

The external site can also (optionally, I guess) provide a parameter for a URL to return to once the process is complete, which LiveJournal would link to and take the user back to the originating site.

There is then another URL which does not require a remote user which is called with a username and token and simply responds “YES” if the given token is stored for the given user, and “NO” if it is not. Presumably the URL supplied by the originating site above would cause the software at that site to request this URL and see if the token was added to the given account.

The above is the basic outline. It will probably end up being a little more complex than this, for example having the tokens expire after 30 minutes. Perhaps we could also allow the site to restrict the journaltype too so that they can require that only journaltype P can auth with them, although I suspect that would be a separate information-retrieval API rather than part of this system, since sites would probably want to validate that an account exists and is of a given type before they create the account on their system, rather than creating it and not letting it validate later.

The only real problem I see with this system is that LJ may get stuck supporting it even after something better comes along. However, I suspect the usual transitional period would be provided before this protocol was ditched after the advent of a superior system. It's got to be better than sites pulling down userinfo pages!