Reenigne blog

One Argulator feature I was, at one point, absolutely convinced that I needed was a web page cache/archive. The trouble is that one often wants to link to other sources of information when making statements (for example to cite a source or backup some assertion). On the web the natural way to do that is to use a link to a URL elsewhere on the web. But URLs are fickle things - the resource they point to can change with time or disappear altogether (something I've always tried to avoid happening with this site - I believe pretty much every reenigne.org URL that was ever valid is still valid today and points to the same information, even if it has been tweaked a bit over the years).

If you're referring to a URL in an Argulator statement, though, that's a problem because a statement's meaning is never supposed to change once it's been created. If it could change, you're putting words into the mouths of all the people who have expressed an opinion on that statement. So I wanted a way to make sure that web pages mentioned in Argulator statements never changed or disappeared, which meant bringing them under the control of the Argulator server. The idea was that when someone mentioned a URL, the Argulator server would fetch that URL, along with the graphics and CSS required to make the page display properly, just as if it was a web browser. Then it would store these files on the server and show it's cached copy (much like Google's cache or the Internet Archive does) when a link is clicked.

But that turns out to be surprisingly difficult, and opens up a whole cannery of worms. Security is a big problem. To ensure nothing bad is ever served from the Argulator site we need to strip out all executable content (script, Java, ActiveX, flash, XUL, Silverlight etc.) from the pages. Which means parsing all the CSS and tag attributes. After that treatment the page might not render at all (especially if it contains things like protection from ad-blockers). Such a cache should also respect the directive and refuse to cache pages whose authors do not want them cached.

Then what do we do if someone requests we take down a cached page (maybe it contains illegal content, or the content owner complains on copyright grounds)? We don't want to create work for ourselves, but on the other hand we don't want to make it too easy for people to sabotage statements they disagree with by getting that statement's sources removed.

I considered the possibility of just linking to pages on the Internet Archive but that may stifle discussions of current events since the IA doesn't serve any pages less than 6 months old. But the idea of linking to an external site instead of keeping the pages on Argulator got me thinking. Such a cache may be useful for other things besides Argulator, so maybe the cache should be a separate (but associated) site. Realizing that such a site would be useful in its own right got me to wondering why nobody had done that before, which got me to realizing that maybe they had. A few google searches later I found several sites which did exactly that. I then realized that Argulator should stick to its core competency and leave the web caching to the experts. Statements can link to any URL anywhere on the internet, but it is recommended for the sake of stability to link to one of these caching services. I hope Argulatists will realize that URLs pointing elsewhere might not have the same contents as they had when the statement was originally created, and take such statements with the appropriate amount of salt.

One thing I wanted to do with Argulator but didn't quite manage was to eliminate the login barrier - to allow people to use any part of the site without creating an account first. The idea was to essentially put all the site's data under multi-headed version control, and then put each temporary user's changes in its own branch, so that they wouldn't affect logged-in users or other temporary users. When a temporary user did decide to create an account, then those changes would be merged into the main branch.

The trouble with that is that arbitrary merges are very difficult. What happens if two temporary users happen to create identical statements (or statements which are similar enough that they would be duplicates if they had both been created by logged-in users)? What happens if a statement is deleted in one branch and created in another? Adding UI for resolving arbitrary merge conflicts would be difficult enough, but forcing users to go through it when they create an account would be downright user-hostile.

I've therefore come to the conclusion that the principle of "you can do anything without creating an account" only works for sites where users' actions don't really affect other users (such as a shopping site).

One interesting thing I had to think about for Argulator was what to do with error messages. There are three types of errors that can occur in applications such as these when classified by the action that needs to be taken.

The first type of errors are things that real users can run into, and which are not problems with the program itself. For example, visiting a statement's page and finding that it has been deleted. These deserve user-friendly error messages and are usually pretty straightforward.

The second type are validation errors, for example someone trying to attempt SQL injection. There's no point giving a user-friendly error message for these as long as we're sure real users can't run into it, for example if we're doing the same validation on the client side, so these are generally handled just with a "die" call.

The third type are programming errors on my part (assertion failures, essentially). These also deserve user-friendly error messages (without details, just a generic "uh oh, something went wrong, we're on it") but they also send me email with some details so that I can try to figure out what's going wrong.

The trouble is it's not always easy to tell what is the second type and what is the third type. Some things seem like they would have to be programming errors but maybe there's some validation missing which means that they could be triggered by invalid input. Some things seem like they could only be triggered by invalid input but perhaps there's a programming error which could cause them to be exposed to real users.

I hope I haven't made too many mistakes classifying these error messages. When choosing whether a particular error is a type 2 or a type 3, I've tried to err on the side of type 3 because the consequences of getting it wrong are less severe (a type 2 misclassified as a type 3 just results in me getting spam, a type 3 misclassified as a type 2 results in me not getting notified, and the user getting an unfriendly error message). Please let me know if you see any error messages in Argulator that don't seem particularly friendly (for example, just text on an empty page instead of a popup div with a button on top of a normal page).

Some other little bits of Argulator that I'm quite proud of:

• Pieces of UI that would be represented as a modal dialog in a desktop application (such as the search results, or the create user dialog) use an HTML div with absolute positioning, that "floats" above the rest of the page. Behind this is a window-filling div with a semi-transparent background, which causes the rest of the page to darken, focussing the user's attention on the dialog. The popup "window" can be dragged with the mouse and scrolling still works, so the rest of the page can still be read if one needs to refer to it during the dialog interaction.
• When there is a script error, it is caught and transmitted to the server with an AJAX request. The server then emails it to me so that I can fix the bug. If this fails, we do nothing in order to avoid an infinite loop. I wish more sites would do this - there is so much broken Javascript out there it's pretty much impossible to surf normally with Javascript debugging switched on. This means I either need to switch my browser between "debugging mode" and "surfing mode" when starting/stopping work on Argulator, or use a different browser for debugging than I use for surfing.
• Argulator uses 96-bit random character strings (encoded as 16 characters of 6 bits each) all over the place, for cookies, pseudo-cookies (for preventing cross-site request forgery attacks), salts and email authentication. These use few enough characters that a URL containing one easily fits on one line of an email message, but have enough entropy to make them unguessable for security purposes. They are generated using /dev/urandom if available, otherwise they are generated with a cryptographic hash, an entropy pool, and the current time.

Speaking of cryptography, Argulator's main cryptographic shortcoming is that the user's password is sent across the wire in plaintext (in a POST request, so it doesn't show up in server logs or caches). I thought seriously about hashing it on the client side (implementing a cryptographic hash function in Javascript). But in the end I decided that this was too much effort for too little gain. The danger of having your password stolen comes mostly from malware and keystroke loggers on the client side, which this does nothing to protect, or having your password stored in plain text in the database (it isn't - we salt it and hash it before storing it). Lots of other sites also send passwords in plaintext. And even if I did implement this, most users would not even be able to tell - really the way to solve this is to use https instead of http for authentication, which I will do if Argulator gets sufficiently popular that I can justify the expense. Then you'll get the little padlock icon in your web browser and you know you're secure. Until then, just don't use the same password for Argulator that you use for important things like your bank account.