Net neutrality

August 26th, 2011

Lots of things have been written online about net neutrality. Here's my contribution - a "devil's advocate" exploration of what a non-neutral internet would mean.

From a libertarian point of view, a non-neutral internet seems like quite a justifiable proposition. Suppose people paid for their internet connections by the gigabyte. This wouldn't be such a bad thing because it would more accurately reflect the costs to the internet service provider of providing the service. It would eliminate annoying opaque caps, and heavy users would pay more. Even as a heavy user myself, I'd be okay with that (as long as it didn't make internet access too much more expensive than it currently is). There would be a great incentive for ISPs to upgrade their networks, since it would allow their customers to pay them money at a faster rate.

Now, some services (especially video services like YouTube and NetFlix) will require a lot of bandwidth so it seems only natural that these services would like to be able to help out their users with their bandwidth. Perhaps if YouTube sees you used X Gb on their site last month and knows you're with an ISP that costs $Y/Gb they might send you a cheque for $X*Y (more than paid for by the adverts you watch on their site, or the subscription fees in the case of NetFlix) so that you'll keep using their service. Good for you, good for YouTube, good for your ISP. Everyone's happy.

Next, suppose that that $X*Y is sent directly to the ISP (or indirectly via the intermediate network providers) instead of via the consumer. Great - that simplifies things even more. YouTube doesn't have to write so many cheques (just one to their network provider) and everyone's happy again. Your ISP still charges per megabyte, but at different rates for different sites.

The problem is then that we have an unintended consequence - a new barrier to entry for new internet services. If I'm making an awesome new video sharing site I'll have to do deals with all the major ISPs or my site will be more expensive to users than YouTube, or I'll have to write a lot of bandwidth refund cheques (which would itself be expensive).

There's also the very real possibility of ISPs becoming de-facto censors - suppose my ISP is part of a media conglomerate (many are) and wishes to punish competing media conglomerates - all they have to do is raise the per gigabyte price across the board and then give discounts for any sites that don't compete with them. Once this has been accomplished technically, governments could lean on ISPs to "soft censor" other sites that they disapprove of. Obviously this is enormously bad for consumers, the internet and free speech in general.

We can't trust the market to force the ISPs to do the right thing because in many areas there is only one broadband option. Perhaps if there were as many choices for an ISP as there are choices of coffee shop in Seattle, having a few non-neutral network providers would be more palatable (non-neutral ones would probably be very cheap given their low quality of service).

As I see it there are several possible solutions:

  1. Force ISPs to charge at a flat rate, not per gigabyte (discouraging infrastructure investments).
  2. Forbid sites from offering bandwidth rebates to customers (directly or via the ISPs).
  3. Forbid ISPs from looking at where your packets are going to end up (they can only check to see what's the next hop that they need to be sent to).

I think pretty much anything else really works out as a variation on one of these three things. The third one seems to be the most practical, and should be considered by the ISPs as a penalty for having insufficient competition.

Legalize all drugs

August 25th, 2011

Some people who know me in person might be surprised to learn that I think drugs should be legalized. After all, I'm not a user of illegal drugs or a die-hard libertarian (though I am finding that I have increasing sympathies for some libertarian points of view as I get older). In fact, this is something that I've changed my mind on in the past - in secondary school I thought they should remain illegal because it makes it easier for impressionable teenagers (like myself) to say "no" to them (however, I did somehow manage to avoid trying cigarettes in secondary school.)

Some things I've learnt since then which have contributed to me changing my mind:

  • To get people to stop taking drugs, treating addiction as a medical conditional rather than a crime is much more effective.
  • To get people to avoid taking them in the first place, making sure everybody is informed about their effects would much more effective.
  • The cost in terms of police, courts and prisons is much greater than the costs to society in terms of loss of productivity and medical costs related to drug usage and addiction.
  • Making drugs illegal creates an enormous black market, leading to a great increase in crime and a flow of wealth to unscrupulous individuals (drug kingpins certainly don't want drugs to be legalized - it would destroy their monopolies.)
  • Stories of innocent people being killed in mistaken "no-knock" drug raids.
  • The existence of illegal drugs makes it very easy for dishonest police officers to frame an innocent person - just plant some illegal drugs on the person you want to imprison (or their house/car/belonging).
  • It is now de-facto illegal to drive across the US with large quantities of cash - it can be confiscated by police if found during an unrelated stop, and forfeited even if nobody is convicted of any crime.
  • The fact that the people convicted of drug crimes are overwhelmingly poor, causing the drug war to be a massive poverty trap.
  • The horrible racist and protectionist reasons marijuana (specifically) was originally prohibited.
  • The general principle that sane adults should be solely responsible for what they put into their bodies.
  • The fact that banning some difficult-to-obtain but mostly harmless drugs has created a market for easy-to-make but much more harmful drugs. Legal drugs are likely to be safer for drug users in other ways too - illegal drugs are sometimes contaminated and sometimes of unknown purity.
  • The fact that Portugal has decriminalized drugs with great success.
  • Drug laws inconvenience law-abiding people too - you can't stock up on decongestant if your whole family has colds, since (to discourage methamphetamine production) you're only allowed to buy a small amount in any given period of time. Also, shops are forbidden from stocking such "drug paraphernalia" as tiny plastic bags.
  • It seems that cannabis (in particular) is actually a very useful medicine for some conditions (such as reducing the side-effects of chemotherapy).

Since only a few of these are specific to marijuana, I'm in favor of legalizing all illegal drugs and taxing them at a rate which neutralizes as closely as possible the harm that they cause to society (thus avoiding a perverse incentive for governments to either encourage or prohibit drug use). However, there are some drugs which are so harmful that taking them should be evidence that a person is not sufficiently sane to make such decisions - people taking those drugs should be committed, not imprisoned.

If it's legal to sell harmful drugs then it wouldn't make much sense for it to be illegal to sell beneficial drugs without a prescription. So, along with legalizing currently illegal drugs I would also get rid of prescription requirements (although having a prescription from a doctor for a potentially harmful drug would still be a very good idea just as a matter of common sense). Having FDA approval for a drug would no longer be necessary in order for doctors to prescribe it or for pharmacies to sell it, but I imagine the FDA would continue to exist as a voluntary safety testing and labeling scheme, and most sensible people would avoid taking drugs which had not been declared as safe (or whose side-effects do not outweigh their benefits) except when circumstances warrant it (such as it being the last hope of curing an otherwise incurable disease). There should be some kind of public awareness campaign so that people know what mark to look for when they are buying such things. To avoid the safer drugs being more expensive, the costs of FDA labeling should be borne publicly.

Such a system would also be much more sensible for small scale food manufacturers.

Correcting injustices

August 24th, 2011

Currently the US legal system (at least, certainly in many other places as well) has a system of plea bargaining - if you are charged with a crime the prosecutor can offer to reduce the charge to a lesser one in return for a guilty plea.

I think this is a huge infringement of rights and is the fundamental cause of a great many miscarriages of justice. I would like to see the practice ended. While it does reduce the burden on the courts, this should not be the primary concern of the justice system. The one and only concern of the justice system should be uncovering the truth and protecting innocent people.

If we didn't care about protecting innocent people, the courts would not be needed at all - the police would just be able to arrest anyone they liked and lock them up for as long as they liked. Obviously that would be awful. The courts are the check on this system - making sure that only people who have actually committed crimes end up in prison.

Overworked, underpaid public defenders will often advise innocent people to plead guilty because they would have little chance of being found innocent by the court. This is an obviously disastrous state of affairs.

Plea bargains wouldn't be so bad if public defenders were actually adequate, but public defenders secure acquittals at a much lower rate than more costly lawyers. This is also obviously disastrous - there's essentially once justice system for the rich and one for the poor, and the one for the rich is much more lenient and forgiving. There's a simple way to fix this (although of course it does require spending enough money to get a functional system) - pick a sampling of publicly defended defendants at random and give them a highly-paid lawyer. If the public defenders do worse, increase the amount of money spent on the public defense system until the difference is no longer statistically significant. The randomness is important because there may be statistically significant differences between the crime rates of wealthy and impoverished people (in fact, this seems highly likely - poverty causes desperation and desperate people take desperate measures).

A third cause of massive injustice seems to me to be how long it takes to get a trial - some people languish in prison for months and even years without ever having been found guilty (the more well off can usually obtain bail). I think all trials should be set for no more than 48 hours after arrest (maybe a week tops if there are extenuating circumstances, like a key witness in a murder case being elusive). The average should be no more than 24 hours. If you don't have the evidence to prosecute someone after that time you shouldn't have arrested them in the first place. The entire bail system should be made unnecessary and scrapped.

Injustices (especially injustices like these that predominantly affect the less well-off portion of society) have the unfortunate side effect of increasing inequality - someone who is imprisoned awaiting trial can't earn money to improve his situation, and being convicted of a crime substantially reduces one's future prospects.

My proposed reforms would be expensive, but I think not unaffordable to a rich country like the US. I think eventually they will happen, as these injustices become less tolerated by society.

Secrets and leaks

August 23rd, 2011

The idea that states should have the capability to keep secrets from their citizens often seems to be taken for granted. Clearly there are cases where states should be able to keep secrets (it is doubtful, for instance, that the allied powers would have won the second world war if the cracking of the German Enigma code had not been kept secret). But lately I've been of the opinion that this privilege should be extremely limited, and should only be used in the most extreme of circumstances.

From Wikileaks, we have learned that states are keeping things secret even when it is not in the interests of citizens or justice for those things to be secret. Sometimes these secrets are kept to protect special interests, or to avoid embarrassment to those in power. Keeping such things secret is antithetical to informed democracy.

I would like to see a system of checks and balances to avoid abuse of state secrets. Wikileaks forms (informally) one such check - as long as they redact things that really do need to be kept secret (for example information that would reveal the identities of undercover operatives). As far I have been able to tell they have done this. However, it does require individual leakers to put their careers (and sometimes even their very freedom) in jeopardy and can therefore only go so far alone. A good additional balance would be to make it illegal for the government to withhold information from the public without good reason. Then if something is leaked which reveals that secrets have been kept unjustifiably, the secret-keepers could be prosecuted on that basis.

An alternative (or complementary) approach would be for a (trusted) third party to hold on to all government information, and to publicly release all the information that doesn't need to be kept secret. Determining which is which isn't free, so there would need to be some kind of penalty for revealing information which endangers operatives. Then, to prevent this organization from just redacting everything there would need to be an economic incentive to release as much (non-endangering) information as possible. Then there would have to be some process for keeping these rates properly tuned to avoid too much information being withheld or released. This tuning process would have to be done with public input (to make sure the balance doesn't swing too far one way or the other) but can't be done by the normal government (or there would be too much temptation to just turn the secrecy level way up). So it essentially needs to be made a whole new branch of government, with that responsibility and no other. Tricky.

Tinkering with the defragmentation API

August 22nd, 2011

I was looking at the Windows file defragmentation APIs recently and it made me want to write something that uses them. Not a defragmentation tool necessarily (there are tons of those, and with the advent of SSDs they're becoming irrelevant anyway). But maybe something for visualizing how directories, files and other structures are laid out on disk. At one pixel per sector, this would make for a very big bitmap (~25000 pixels square for the 300Gb drive I use for data on my laptop and ~75000 pixels square for the largest 3Tb drives) so I'd have to make a way to zoom in and out of it. And how should I arrange the sectors within the bitmap? Row major is the obvious way, but a Hilbert space-filling curve might make for nicer results. For colouring, I was thinking of using the file extension as a hint (e.g. red for images, green for audio, blue for video, grey for binary code, yellow for text, black for free space) and maybe making the first sector of a file brighter or darker so that you can see if it's one big file or lots of small ones (though weighting the sector by how much of it is used would have a similar effect). I also want the program to show which sector of which file is under the mouse pointer.

Sound synchronizer

August 21st, 2011

I want to write a little program that takes two sound files and plays them at the same time, but which adds the position of the mouse's scroll wheel to one of the playback positions so that you
can easily twiddle the synchronization between them. I'm not sure that this would be particularly useful for anything, but I think it would be fun.

Preventing cheating in online games

August 20th, 2011

In computer security, there is a general rule which says that you should never trust anything sent to your server by client software running on a user's machine. No matter how many cryptographic checks and anti-tampering mechanisms you put into your code, you can never be sure that it's not running on an emulated machine over which the user has complete control, and any bits could be changed at any time to give the server an answer it accepts.

This a problem for online gaming, though, as cheaters can give themselves all sorts of capabilities that the game designer did not plan for. This (apparently - I am not much of a gamer) reduces the enjoyment of non-cheating players.

However, games do have one advantage here - they generally push the hardware to (something approximating) its limits, which means that running the entire game under emulation may not be possible.

So, what games can do is have the server transmit a small piece of code to the client which runs in the same process as the game, performs various checks and sends the results to the server so it can determine if the user is cheating or not. The Cisco Secure Desktop VPN software apparently uses this technique (which is how I came to think about it). I have heard this small piece of code referred to as a "trojan" in this context, although this terminology seems misleading because this particular kind of trojan doesn't run without the users knowledge and consent, and is only malicious in the sense that it doesn't trust the user (the same sort of maliciousness as DRM, which is not quite as bad as illegal malware).

The trojan for an online game could send things which are very computationally intensive to compute (such as the results of the GPU's rendering of the game). Because the server can keep track of time, doing these computations in anything less than real time would not suffice. To avoid too much load on the server, the computations would have to be things that are easier to verify correct than to compute in the first place (otherwise the server farm would need to have a gaming-class computer for every player, just to verify the results). And to avoid too much load on the client, it should be something that the game was going to compute anyway. I'm not quite sure how to reconcile these two requirements, but I think it should be possible.

The system should be tuned such that the fastest generally available computer would not be powerful enough to emulate the slowest computer that would be allowed to run the game. Depending on the pace of progress of computer technology and the lifespan of the game, it might eventually be necessary to change these requirements and force the users of the slowest computers to upgrade their hardware if they want to continue playing the game. While this would be frustrating for these players, I don't have a problem with it as long as there is a contract between the players and the game company that both agree to and are bound by - it would be part of the cost of playing without cheaters. Though I would hope that independent servers without these restrictions would also be available if there is demand for them.

Special relativity game

August 19th, 2011

I think it would be awesome if somebody made a 3D, first person computer game where the speed of light was significantly slower (perhaps 30mph as in the Mr Tompkins books) and did relativistically correct rendering so that you could see the geometric distortions and Doppler shifts as you walked around. It might be necessary to map the visible spectrum onto the full electromagnetic spectrum in order continue to be able to continue to see everything (albeit with reddish or bluish hues) when you're moving quickly.

It would have to be a single player game because (in the absence of time travel) there is no way to simulate the time dilation that would occur between players moving around at different speeds.

It appears that I'm not the first person to have this idea, though the game mentioned there (Relativity) doesn't seem to be quite what I'm looking for.

I do realize that it might be quite difficult to do this with current graphics engines, but I'm sure it could be done in real time, perhaps with the aid of suitable vertex and pixel shaders for the geometric/chromatic distortions respectively.

Bifurcation fractal

August 18th, 2011

I wrote a little program to plot the Bifurcation fractal when I was learning to write Windows programs from scratch - this is an updated version of it. Unlike most renderings of this fractal, it uses an exposure function to get smooth gradients and scatters the horizontal coordinate around the window so you get both progressively improvements in image quality and can see very fine details (such as the horizontal lines of miniature versions of the entire picture).

You can zoom in by dragging a rectangle over the image with the left mouse button, and zoom out by dragging a rectangle with the right mouse button.

Checksum utility

August 17th, 2011

I couldn't find a good CRC32 checksum utility that properly handled wildcards, so I wrote my own.

I actually first wrote this a very long time ago using DJGPP. Actually, thinking about it it might have been the first 32-bit program I ever wrote. That compiler made it very easy because the command-line handling did the directory recursion for me (though it did mean you had to use the awkward syntax "crc32 ...\*" instead of "crc32 *" or "crc32 .". However, 64-bit Windows won't run DOS programs so it needed a rewrite, and once I had completed the filesystem routines I mentioned before it was just as easy.

This program is very handy for comparing contents of large directories across machines ("windiff -T" would copy everything across the network so would take forever.) Just run it on the two directories locally and compare the output.

I used to use this for weekly backups of my machine. It's nice to see exactly what's changed since the last backup so I can delete the backup files made by my text editor and anything else I didn't really mean to keep. Nowadays I use rsync to a Linux machine which amounts to something very similar but more automatically.