The Abstract Factory

The social graph is...

2011-11-17T05:03:00.000-08:00

...provided you redefine the words "social" and "graph" to mean something other than what they mean to everyone else.

M. Ceglowski is just being deliberately obtuse, or more precisely he is taking a wild excess of rhetorical license in order to make his statements seem more profound and unconventional. For example, he writes:

We nerds love graphs because they are easy to represent in a computer and there is a vast literature on how to do useful things with them. . . . In order to model something as a graph, you have to have a clear definition of what its nodes and edges represent.

Well, that's actually bullshit. In a dynamic Bayesian network, you don't have a complete definition a priori of what nodes and edges represent. Well, you do, in that the nodes represent variables and the edges represent relationships between those variables, but the weights on the edges are learned statistically from data. An edge may represent a meaningful connection, or it may mean nothing at all. The graph precedes semantics, not vice versa. Likewise with the social graph. People are connected, and you don't necessarily know what each connection means. But it's still a graph.

The labels on the social graph's edges may be subtler and more multidimensional than the simple weights you put on Bayesian network edges. And we don't have a good handle on how to learn those labels, or even what the labels should be. However, calling for the abandonment of a useful mathematical construction in an emerging field of science because it's incomplete is something that you do when you want to convince people that you're smarter than the people working in that field. It's not something you do when you want people to become better-informed.

Ceglowski also writes that the social graph is "not social" because... well, actually, I have trouble even locating a coherent argument in that part of the essay. He seems to be confusing "social" with "sociable". The social graph is social, since it describes relationships between people. Perhaps some activity involved in digitally reifying the social graph is anti-social (Note that anti-social is not the opposite of social — anti-social behaviors are social behaviors!). But that doesn't make the social graph "not social". By that standard, sociology is not a social science because sociologists spend a lot of time by themselves in libraries.

Incidentally social scientists have been modeling social connections as graphs for decades.

Here is a short list of the valid points Ceglowski makes:

FOAF relationship labels are kind of dumb and embarrassing.
Manually maintaining anything other than a very coarse-grained digital reification of a social graph is a tedious chore.
Making your social network and behavior the property of a company whose revenue model is not aligned with your long-term interests is a bad idea.

And here is a short list of other, non-terminological points that Ceglowski just gets wrong:

Social networks do "[g]ive people something cool to do and a way to talk to each other". It turns out that sharing photos, videos, and links is one of the most broadly appealing online activities, and social networking sites seem to do this better (along some dimensions) than dedicated photo-, video-, and link-sharing sites.
Judging communities by the outward-facing cultural artifacts they produce is a radically inadequate measure of value. The vast majority of communication is point-to-point, not broadcast, and the vast majority of interpersonal interactions are social grooming. Social grooming is a deep-seated primate instinct which nerds devalue at their peril. Social networks have made online social grooming far easier than their predecessors did.
People on WoW, Eve Online, and 4chan have healthier social lives than people on Facebook? Really?

Note that I write all the above as someone who dislikes Facebook and is skeptical of reductive approaches to modeling social relationships. And I've been advocating* an end to proprietary social networks for years — long before I started working at Google, and in fact before Facebook was even the predominant social network. So I'm broadly sympathetic to Ceglowski's aims. But I don't like at all the way that he goes about explaining them.

*Incidentally, rereading this old post, I realize that I completely missed the possibility that the dominant social network site would simply become a huge platform for third-party applications. I guess it never occurred to me that serious companies would bet their livelihoods on being sharecroppers in the walled garden. Go figure. I could speculate that this willingness can be traced directly to the Valley vogue for building companies to flip rather than to create sustainable, decades-long sources of enduring value — if you're just holding on until your "liquidity event" then it doesn't matter that your business is built on the fickle forbearance of your platform landlord — but I'm not sure how right that is.

Occupy .* and the Iraq War

2011-10-31T10:05:00.000-07:00

Then, as now, conservative opinion and elite bipartisan opinion was mostly contemptuous of the protesters. Well-fed, well-educated, well-salaried pundits looked on the shaggy protesters and remarked: how unsophisticated were the protesters' opinions, how disorganized their complaints! Fortunately, the nation was run by a select few who understood the harsh realities of a world where some suffering was necessary (for other people) so that the existing order could be maintained. And the march to war rolled on.

In all likelihood, the protests today will be as futile as those were. It's taken a couple hundred years, but the system of governance by elected representatives has evolved an immune system with nearly impervious defenses against street protests. Nevertheless in a society supported by the many and operated for the few, it is perhaps useful, for aesthetic reasons if nothing else, to have some people calling attention to that fact. If you, as a critic, imagine yourself on the side of the angels in damning the protesters, then you should perhaps reconsider.

Cynicism and libertarian ends (again)

2011-10-13T22:48:00.000-07:00

Why M. Yglesias is a nationally acclaimed writer and I am not, exhibit #7572: a couple of years ago I wrote a somewhat convoluted post about cynicism and libertarianism, whereas today Yglesias wrote this which is elegant, readable, and much more worth your time.

In ur base, 0wning ur dronez

2011-10-07T19:20:00.000-07:00

So it turns out that roc is (again) pretty prescient.

g++ unordered_multimap: an exercise

2011-08-03T06:50:00.000-07:00

I discovered this randomly several weeks ago while debugging something else at work, and I thought it was worth sharing since the g++ STL is widely used.

Step 1: Save the following file as mapdemo.cpp:

#include <iostream>
#include <unordered_map>

int main() {
    typedef std::unordered_multimap<int, int> int_multimap;
    int_multimap map;
    for (int i = 0; i < 10000; ++i) {
        map.insert(int_multimap::value_type(17, i));
    }
    std::cerr << *static_cast<int*>(0);
    return 0;
}

Step 2: Compile the file:

g++ --std=c++0x -g mapdemo.cpp

Step 3: Load the file in gdb and examine the buckets:

$ gdb -silent ./a.out
Reading symbols from xxx/a.out...done.
(gdb) run
Starting program: xxx/a.out 

Program received signal SIGSEGV, Segmentation fault.
0x0000000000400b4d in main () at mapdemo.cpp:10
10     std::cerr << *static_cast<int*>(0);
(gdb) p map._M_bucket_count
$1 = 15173
(gdb) p map._M_buckets[0]
$2 = (std::__detail::_Hash_node<std::pair<int const, int>, false> *) 0x0
(gdb) p map._M_buckets[15172]
$3 = (std::__detail::_Hash_node<std::pair<int const, int>, false> *) 0x0
(gdb) p map._M_buckets[17]
$4 = (std::__detail::_Hash_node<std::pair<int const, int>, false> *) 0x605080
(gdb) p *map._M_buckets[17]
$5 = {_M_v = {first = 17, second = 0}, _M_next = 0x670c90}

Yes, g++'s implementation of unordered_multimap (known as hash_multimap in pre-C++0x versions of C++) uses bucket hashing, but the size of the backing array is proportional to the count of elements in the multimap, not the count of distinct keys.

Exercise for the reader: Explain what I just did; explain why the result of step 3 is curious; and then explain why the authors might have chosen to do it this way anyway.

It occurs to me that this would have made a decent interview question if I hadn't written it up here. Oh well, I have others.

Acer Aspire TimelineX 1830T semi-review

2011-08-01T23:08:00.000-07:00

Attention conservation notice: Google-food for a gadget you will probably never need to know about.

I recently bought an Acer 1830T ultra-compact notebook (11.6" screen, 3.1 lbs, Core i5 470UM, ~$540 street price). I have neither the time nor (yet) the data to review it comprehensively, but here are a few observations that I didn't find in other online reviews, with a focus on the physical design. This review is intended to complement other information online, not replace it, and is offered in the hope that it's useful to other people who may be in the market for a very small laptop.

Size

You can read on the spec sheet that the laptop is 285 x 204 x 28 mm (11.22 x 8.03 x 1.1 inches) at its widest point, but that doesn't give you a visceral feel for its size. Here it is next to a few objects with which you may be familiar.

Counterclockwise from upper left: Kindle 2G, 12oz. Diet Coke, Nexus S, $1 Federal Reserve Note, 15" Macbook Pro (unibody), and TimelineX 1830T. You can see that it's quite small. It feels qualitatively similar in the hand to other ultra-compact notebooks I've handled, such as smaller Lenovo X series laptops, although it's not nearly as blade-thin as a Macbook Air. Below are a couple of detail on-edge shots to illustrate the device thickness.

The top shot is the same 15" Macbook Pro and Kindle stacked next to the TimelineX 1830T, with the other props on the edges. The bottom shot is the Macbook Pro alone next to the 1830T. Subjectively, I will say that its small size and light weight make it feel thinner than it is. I'll toss this in my bag as a second laptop without hesitation.

Input devices

The keyboard is OK but not stellar. Here it is, compared to the Macbook Pro keyboard, using my left hand as a reference object:

You'll notice that it's marginally smaller — and for me, this is enough that it does subjectively feel more cramped — but it's much closer than I would have expected, and about as good as I'd expect from a machine this size. It's not a Lenovo keyboard but then nothing is.

The trackpad is quite small and almost invisible, marked only by three fine raised lines on the palmrest's metal surface:

Overall the small size of the palmrest and trackpad are the biggest ergonomic shortcomings of this device. I've concluded that they shrunk the palmrest to make room for the (huge) battery on the top side of the keyboard. After a brief acclimation period, I find the trackpad acceptable for casual use, but I expect I'll still carry around a mouse when I want to do serious work.

Power adapter

The TimelineX has an interesting power adapter. First, here is a size illustration.

Counterclockwise from upper left: Nexus S, Macbook Pro power adapter, TimelineX 1830T power adapter, 12oz. Diet Coke, and $1 bill. Note that the adapter is closer in size and weight to a cell phone power adapter than a traditional laptop power adapter. Furthermore the plug prongs are detachable:

Cleverly, the prongs can be attached in either orientation:

Obviously this is useful for power strips or other situations where the area around the cord might be crowded. Most laptops deal with this problem by attaching the plug head via a separate cord, but I would like to see this design become more widespread for cell-phone-style power adapters. Rotating the prongs is a simple 10-second operation:

Miscellany

A few other brief observations:

1080p video plays perfectly fine, either downsampled on the native 1366x768 screen or at full resolution on an external 1080p television via HDMI.
64-bit Ubuntu Maverick (10.10) under VMWare Player works adequately for casual coding. I have not yet set up dual-boot and perhaps I won't need to.
Suspend from resume is quite quick in Windows; a full boot is slow but rare.
The battery is huge; the screen is small; the CPU is an ultra-low-voltage Core i5. The net result is that battery life goes all day for practical purposes, and this is the rare laptop that I will not bother to plug in during use most of the time.

Finally, here's a close-up shot of the laptop cover, which has a nice, grippy embossed cross-hatched texture.

p.s. Incidentally, having shopped for a laptop recently, I have to say that typical review sites do not pay nearly enough attention to laptops as physical objects. The physical reality of the laptop is one of its most crucial characteristics; it's not like a workstation which you just leave under your desk and hook up to the keyboard, mouse, and monitor of your choice. People who do this for a living should be able to provide more useful and objective information than "it feels light". Also, laptop review sites overall strike me as quite lazy. Why would you show useless white-background product shots from the PR kit, when digital cameras are so ubiquitous, and it's trivial to take your own much more useful photos, as this blog post shows? Grrr.

Two white dresses

2011-07-02T14:58:00.000-07:00

Striking juxtaposition (intentional?) on the CNN International home page just now:

One woman is the subject of the photo, and you see her eyes looking fearlessly at her vanquished opponent. Furthermore, she is important for having excelled in a worldwide competition of objective achievement. The other is a secondary subject, her eyes invisible but her gaze clearly directed at the primary subject of the photo, a man — whom, I suppose, she has also conquered, in a sense, although her fellow competitors for the prize are (as in the first photo) outside the frame. Furthermore she is newsworthy only because her new husband happens to be of royal birth. Wittstock is of course more conventionally beautiful than Kvitova as well.

I won't claim that sports are categorically more important than weddings of ceremonial heads of state, but someday I'd hope that the positions of these images would be reversed.

(Posted from lounge in BGI airport while waiting for a flight.)

Tree structure and comment threads (a brief observation)

2011-04-03T14:13:00.000-07:00

It has been claimed that flat, linear presentation of comments appears to work better for humans than tree-structured comment threads. Without getting too deeply into whether this is true (and if so, why), I would like to offer an observation.

Conversation is never a tree; it is a general directed acyclic graph. In reality, in the commenter's mind, every comment potentially implicitly responds to an arbitrary subset of preceding comments, not to a unique parent and its chain of unique transitive ancestors.

Tree-structured threading — sometimes (erroneously!) called "true threading" — artificially imposes a tree structure on this graph. Flat, linear comment systems do not: each comment appears after all those that precede it topologically in the DAG, and it is up to the reader to reassemble the DAG based on the comments' contents.

It is true, of course, that flat comment systems fail to reify all DAG edges as explicit metadata. However, the nature of these edges is quite subtle and capturing them all explicitly is intractable. Often a comment "responds" to previous comments in indirect ways — for example, simply by omitting some aspect of the argument that has been covered by a previous comment.

(Prompted by a TC article linked off HN.)

Ubuntu has not solved the agency problem in community software development for the Linux desktop

2011-03-27T12:37:00.000-07:00

Four and a half years ago, I wrote:

In an open source project with a dictatorial or committee-led governance structure, somebody would long ago have cracked some heads and gotten this feature implemented. In a commercial software project, open source or non-, some engineer would be assigned ownership of this feature; and goddammit, if that feature didn't get implemented and maintained, that engineer would be fired and the feature would be assigned to someone else. But KDE's headless. It's less like a mammal with a central nervous system than an enormous amoeba whose various pseudopodia ooze tropically in the direction of "developer itches" and "coolest implementation hacks" (hence the recent proliferation of "hugely ambitious infrastructure refactoring" subprojects like Plasma or Solid) rather than unsexy, annoying-to-implement features that merely provide value to end users.

I genuinely thought Ubuntu had a fighting chance of resolving this agency problem. Surely with a dictator taking responsibility for the entire desktop stack, there would be progress. When there's a bad corner of usability for a common user task, somebody will crack some heads and get it fixed.

Today, I tried to share a file over my local network between my Ubuntu desktop and my Ubuntu laptop. And I ran into this defect which has been open for a year. In the year 2011, the easiest way to copy a file from one Linux computer on your local network to another Linux computer on your local network is still either (a) copy it onto a USB stick or (b) upload it to the Internet (e.g. by attaching it to a draft Gmail message).

Meanwhile, Ubuntu's burning huge numbers of developer and UI designer cycles on stuff like Unity ("We're too impatient to fix the rough edges in our existing desktop which has a decade of developer investment behind it; therefore we will design a brand new desktop, because there definitely won't be any rough edges in that.").

It turns out that, in fact, Ubuntu has not solved the agency problem in Linux desktop software development. Sigh.

Federal budgets and applied public choice theory

2011-03-24T19:40:00.000-07:00

Atrios writes something so true you should affix it in your memory for the next two decades:

There's literally nothing that this Congress today can do to reduce the deficit 20 years from now. What you can do is sign into law legislation which reduces granny's pension 20 years from now. And, yes, given the way our system works it wouldn't necessarily be easy to reverse that decision 20 years from now depending on the politics and who is in power. But what will still be easy to do 20 years from now is cutting taxes on rich people and writing giant checks to defense contractors. Those things are always easy to do when Congress and their donors are mostly rich people. Reduce the deficit by cutting granny's pension, increase it again by cutting taxes on rich people. Rinse repeat.

. . .

. . . just to remind us of history I'm sure we all remember. That Democratic Socialist Bill Clinton got rid of the deficit. Alan Greenspan, who spent years fretting about the deficit, suddenly decided the great danger we faced was not having a deficit. And Bush tax cuts, and too and such.

Once this really sinks in, you realize that the only measure which leads to long-term balanced budgets is reform which changes the configuration of political power so that budget deficits no longer benefit the powerful. Going by today's projections, the main contributors to long-term budget deficits are rising medical costs, the Bush tax cuts, and war (and preparation for war). Therefore the principal budget-balancing methods that might work in the long term are:

health care reform that reduces the power of people and industries that make medical care expensive.
campaign finance reform and progressive taxation, which reduces the power of rich people to demand tax cuts (campaign finance reform directly, and progressive taxation indirectly by simply making rich people less rich).
reducing funding for the military and defense contractors, which reduces the political power of the military-industrial complex by reducing the number of people dependent upon it.

You may think some of these changes would be bad. That's OK; it just means these reforms conflict with your political values, and I'm not even trying to argue you out of your political values. Just recognize, then, that budget deficits will forever be the price of maintaining policies consistent with your political values. Relax; it's not so bad; maybe it's even a price worth paying; but no amount of cleverness on your part will allow you to wriggle out of paying it.

Your faction may exercise heroic effort, tenacity, and ingenuity to bring the budget into balance. And if you succeed, the current political economy of the United States simply means that the politically powerful health care sector, the politically powerful overclass of wealthy people, or the politically powerful military-industrial complex will find some way to squander your effort and throw the government back into deficit, with the balance of money going into their pockets. This is not a piece of political polemic; it is simply an observation that I offer about the world, with recent history as my evidence.

And as for the widespread conceit of upper-middle-class liberals that there's some purely technocratic way to fix long-term deficit problems by twiddling around with the retirement age and the like — that's just an exercise in hopeless naivete.

The moral case for sending money to Japan (rather than somewhere else)

2011-03-13T16:57:00.000-07:00

Japan is one of the richest countries in the world, both in absolute and per capita terms. The earthquake and tsunami have inflicted terrible damage on the country, leading most likely to thousands of deaths and many millions of dollars of property damage. However, as a nation, Japan has ample financial resources to recover from the disaster. Japan was a healthy and prosperous society one week ago, and a year from now the overwhelming majority of Japan's people will still be alive and healthy, and they will still be relatively prosperous in global terms.

By contrast, there are still hundreds of millions of other people around the globe living in conditions of persistent poverty and immiseration. Just to take one random example, one year after Haiti's 2010 earthquake, Haiti is still a complete basket case.

So, the Japanese earthquake has no doubt pricked your conscience. You have been reminded that there are people in faraway places who direly need assistance. Your moral intuitions are worthy, but if they lead you to donate money to Japanese relief, then you are probably doing something non-optimal from the point of view of improving human welfare. Donate, instead, to an international relief organization that consistently directs its efforts to the most needy worldwide: Oxfam International, Unicef, etc. It is even possible that these organizations will spend some of their resources to help Japan now; but they're in a much better position to analyze the situation and direct the appropriate quantity of resources in that direction than you are.

Accordingly, I just donated to Oxfam, and I urge you to do the same.

On the other hand, of course there are some forms of aid specific to the immediate aftermath of natural disasters for which money is not a substitute. Obama has directed the U.S. Navy to station aircraft carriers in Japan to help airlift relief supplies and such. Google has launched the People Finder for Japan. Etc. These organizations are uniquely situated to help in ways that no amount of money can purchase on the open market. And if you know of some similarly specific aid effort for which equivalents cannot be obtained via market mechanisms, then you should support that effort however you can.

A final caveat is that resources within Japan are unequally distributed. There are usually some very poor and miserable people even within rich societies. If you know of some specific subgroup within Japan which is unlikely to receive assistance due to the structure of Japanese society, then again go ahead and donate to help them as well.

But to a first approximation, the logical response to natural disasters in wealthy countries is to donate money to aid organizations generally, not to donate to aid for those countries. And yes, when The Big One hits the Bay Area (where I currently live), I'll say the same thing.

UPDATE 2011-03-16: Via MR, charity ratings organization GiveWell agrees that donations should not be sent to funds earmarked for Japanese disaster relief. See also F. Salmon, and F. Salmon again.

A simple handwave that makes The Matrix tolerable for the scientifically literate

2011-02-15T23:33:00.000-08:00

Suppose that controlled fusion power requires real-time control computations which can be much more efficiently implemented in neurological hardware than in silicon.

(That Second Law of Thermodynamics thing was bothering the hell out of you, wasn't it?)

Note that neurological hardware really is exceptionally power-efficient for certain classes of computations. Common estimates of the human brain's power consumption are 20-25 watts; this is roughly the wattage of a Mobile Intel Core i5 processor, which (as far as we know) appears to be a much less powerful computer for many purposes. By contrast Watson runs on 90 IBM Power750 servers, filling ten racks, whose power draw is something like 80 kilowatts. In other words, Watson consumed about four thousand times more power than either of its meat-based competitors.

Here in reality, I think it's unlikely that any fixed class of computations can be efficiently implemented in neurons but not in silicon — see Carver Mead and his academic descendants' work on analog silicon circuits. But positing that such computations may exist seems within the realm of acceptable science-fictional handwaving.

UPDATE: Yes this is close to the standard handwave that the humans are being kept as computing devices, not power sources. But I think you need to draw the connection explicitly to power generation; otherwise there's just too much narrative in the film and animated shorts that makes no sense.

UPDATE': Never mind, I just remembered Morpheus's exact wording from the first film's voice over and I don't think it's salvageable. Oh well.

Why iPads' role in education should be limited

2011-02-06T10:48:00.000-08:00

This morning I came across this story about a Georgia state senator proposing replacing textbooks with iPads. This strikes me as a bad idea for several reasons.

First, the iOs developer agreement means that iOs will forever be a platform that teaches children to consume rather than to create, at least with respect to my discipline (computer programming).

If you think this doesn't matter because "kids don't program", I would like to disabuse you of that notion, hard. The iPad could have been a great device for running Squeak, which includes environments like etoys and scratch. These systems teach children to be producers rather than merely consumers of computing, which I think is no small matter given the importance of computers in society today. But, as a programming language interpreter and compiler, Squeak is prohibited by the iOs developer agreement

Many adults grew up as passive consumers of technology — for example, as television watchers or video game players, rather than people who shoot videos or program games. This blinds them to the possibility that things could be profoundly different. Ubiquitous digital video cameras and video sharing sites like YouTube have already transformed the relationship between people (especially young people) and video. Video is no longer solely the thing you sit on the couch and watch. It's something everyone can and does create. Despite the proliferation of astonishing banality on YouTube, I claim that this is a positive development. Making and editing a video of your cat is still more rewarding, and exercises more cognitive skills, than passively watching almost anything on television.* Programming could be the same way. No, the average person will never build complex software from the ground up. But the average person can, with appropriate support, learn to write small programs, and to modify big programs around the edges, and (crucially) to enjoy doing so. The iOs developer agreement simply shuts down this avenue of creativity, and that's the wrong thing to be doing to children.

Well, OK, so maybe you don't care about ideological issues like "freedom" and "creativity" (although I claim that if you do not, then you have no business working in education). Let's get down to practicalities.

The general attitude of Apple towards iOs device "owners" has been that the end-user does not truly own the device — Steve is just letting you hold it for a while. Note Apple's hostility to jailbroken devices and even its use of nonstandard screws for its cases. What does this mean in the education world? A school district that buys a thousand iPads will be completely at Apple's mercy w.r.t. software upgrades, hardware repairs, and general system maintenance.

The article notes that Georgia is "currently spending about $40 million a year on books [that] last about seven years". Apple is a consumer company and builds its products to last about 2 years. Ask people with vintage 2007 iPhones how well their devices are coping with iOs 4. Furthermore this 2-year product cycle is not an accident; it's built into the foundation of Apple's current business model, which is to ride the leading edge of technology so that they can always sell a device that's shinier than their competitors' devices (and even their own devices from a couple of years ago). Have the school districts really worked out the implications of this? I doubt it.

Furthermore, the article notes: "Textbook publishers are eyeing the potential for moving their content to the digital world, enabling them to update material rapidly and include interactivity." The unspoken subtext is that school districts will move from purchasing textbooks to purchasing subscriptions to textbook content. Ask university libraries with electronic journal subscriptions how well that's working out for them. And the iPad textbook age will, in the long run, be worse than that. Electronic journals at least distribute their articles as DRM-free PDFs: if you find a paper you need, you can save it to your local hard drive and have it forever. Does anyone think that grade school textbook publishers are going to release their iPad content without DRM? I laugh mordantly in your general direction. I pity the school district that has a budget crisis and cannot afford a textbook publisher's access fees for the year. I can only hope that most districts keep the old dead-tree textbooks in the basement for emergencies.

In summary, iPads in education will

Teach children to consume programs rather than to create them.
Lock schools into a closed computing ecosystem that they do not control.
Lock schools into a continual treadmill of costly hardware upgrades.
Lock schools into electronic subscriptions to DRM-encumbered textbook content.

All that said, dead tree textbooks are expensive and heavy and troublesome to update, and there's tremendous potential for electronic educational materials to improve the situation. And the iPad, being an excellently made device with an active developer community, is a fruitful context in which to experiment with new educational software. But until/unless some of the above features change, it's extremely premature for an entire state to make the iPad the center of its educational curriculum.

*Of course, there are a few great shows on television and people still watch those. Homemade video does not replace that and never will. But great shows constitute a tiny fraction of televisual content.

The (logarithmic) calendar I want

2011-01-13T23:46:00.000-08:00

I was looking at my Google Calendar tonight and I realized that this software was encouraging a way of thinking about time which has significant drawbacks. Namely, it only allows one to look at a small, fixed time window at once: a day, one or two weeks, or a month at a time.* But the question I was asking myself at the time was, what is the shape of my year going to be?

When am I going to make time to visit friends and family in faraway places? How should I spend the 20 vacation days per annum that my employer gives me? What sort of personal projects will I realistically have time to pursue over the course of this year?

Being the morbid type of person I am, these thoughts then led me to other questions about even longer time spans. What do I want my life to look like in 5 years? By contrast, what is the realistic outcome of extrapolating 5 years from the way that my life is moving now? What do I hope to accomplish before I die, and am I going to have time to do it?

All these types of questions cannot be visualized on a month-at-a-time calendar, whether it's Google Calendar, or some other software, or a typical monthly wall calendar printed on paper. Now, the mere fact of having a proper calendar scarcely leads to satisfactory answers to the questions I'm pondering. But clearly the artifacts we use to track time influence our thoughts, our emotions and ultimately our actions. In this light, the limited scope of our calendars seems like a cognitive handicap with potentially huge effects on our lives.

So, here is what I want. I want a logarithmic calendar. I want this week to be visualized large. I want the rest of this month and the next to be visualized somewhat smaller. I want larger and more distant time intervals to be visualized as progressively smaller boxes. And I want the scope of the calendar to be decades — at least as long as my remaining life expectancy, or perhaps a bit longer so that I'm forced to think about posterity.

With a little work, I could write a bit of software that visualized time like this. There's nothing particularly earth-shattering about the code required. It is, as we say in the industry, a Mere Matter of Programming.

In fact, doing it in software is so straightforward that it's almost more fun to puzzle out how to do it with just paper. An argument can be made that paper would be fundamentally better anyway, in at least a few ways. A wall-sized poster is a tremendous display technology, with better resolution, pixel density, and variety of supported input methods than even a 30" touchscreen monitor. And a wall poster doesn't get covered up whenever you open a web browser while craving a moment's distraction. At best, you can ignore it via the old-fashioned method of averting your eyes.

So, let's do the exercise. How would you lay out a logarithmic wall calendar using only layers of preprinted paper? Here's my stab (apologies for the roughness of the sketch):

For every year, you mount a new spiral-bound calendar at the top. The spiral-bound annual calendar has ~52 leaves (give or take depending on the number of calendar weeks in the year). The week currently on top has a lot of writeable area per day, but the bottoms of the pages for the rest of this month peek out from beneath the current week. Below the current month's week pages, there are tabs for each month, which are large enough that you can write notes in them. The months go across from left to right. And beneath that, there's a large writeable area for annual goals, observations, etc. for each of the next 5 years. These are not part of the spiral-bound calendar; instead they are pinned to the wall, and when you reach a year's end, you unpin that year and pin up a new strip. And finally below that, there are undifferentiated strips for each semi-decade following the current semi-decade.

Suggested customization: next to each year, write down how old you will be when that year begins.

Admittedly, it's all a bit ad hoc. Randall Munroe would no doubt have devised some much more rigorously consistent mathematical visualization. But this is just my idle evening doodling, and at each level of time granularity I just chose something that looked good to me.

Now, to start thinking about those actual goals...

*Incidentally, there are deep architectural reasons that interactive calendar software which aims for predictable latency per user gesture will tend to offer (visualizations of) fixed-time-window queries, rather than queries over windows of unbounded size. If you're software-minded you can probably figure these reasons out, and also some ideas for working around them.

UPDATE 2011-10-31: Hacker News reactions.

Sexual desire, authenticity, and Internet business models

2011-01-08T16:29:00.000-08:00

Among the many errors in N. Vargas-Cooper's Atlantic article this month on Internet porn, one stands out as the major fallacy. The article's entire argument seems to rest on the assumption that Internet porn reflects a more authentic view of human desire than other forms of cultural production. But the prevalence of low-budget amateur porn on the Internet reflects less on the sexual stimulation that people most fervently desire than on the economic reality that it's vastly easier to build an Internet business on crowdsourced amateur porn than on any other kind of porn.*

To conclude, from the consumption of bad porn, that everyone prefers bad porn when better porn can ostensibly be had if one looks harder, is roughly like deducing from the success of McDonald's that everybody prefers Big Macs to every other type of food. Again this is a question of costs, not authenticity. Sexual desire, like hunger, is an extraordinarily robust urge and can be stimulated (and even, to some extent, satisfied) by relatively low-quality fare. Therefore, people settle. If I offered you a free dinner at McDonald's or a free dinner at French Laundry, you know which you'd pick. And yet McDonald's is a much larger business, for reasons that have nothing to do with McDonald's satisfying a more authentic or deeper hunger.

If you think I'm comparing apples and oranges with McDonald's and French Laundry, substitute In-N-Out for the latter and the answer remains the same. Even when you want a burger, you want a good one; but sometimes you settle for less, because it's cheaper or more convenient.

The analogy becomes even better if you imagine that hamburgers were culturally marginalized and frequently outlawed, such that many people were embarrassed to admit in public that they liked a good burger. Such a social and legal environment would make it hard to build a business around providing high-quality burgers, and In-N-Out or other good burger places would not exist. And cultural critics would conclude that there was some inherent property of human hunger that led us to prefer crappy burgers.

Which is why Vargas-Cooper leaps from a particular configuration of economic incentives, in a particular technological and social context, to a ridiculously broad claim about the ultimate nature of human desire.

*The proliferation of highly specialized niche porn is also a consequence of Internet economics, albeit for different reasons, which I leave as an exercise to the reader.

The types of bestselling free Kindle books

2011-01-03T07:09:00.000-08:00

Periodically I go on binges where I browse the Kindle bestsellers list and download most of the top 100 free books, more or less indiscriminately, without consideration for quality. I mean, what the hell, it's free and my Kindle 2 still has over 1.2GB of free storage (out of 1.4GB user space). Even the worst piece of formulaic pulp trash might be funny in a so-bad-it's-good kind of way; or at least there may be some anthropological interest (oh, so this is what women fantasize about?). Most of the stuff goes totally unread of course — I don't have time to even glance at a tenth of it — but I suppose I like having the option.

Anyway, it's interesting to note that as of January 2011, the top 100 free Kindle ebooks list consists of the following:

Count	Type	Example	Notes
50	Gutenberg ebooks	The Adventures of Sherlock Holmes	Kindle conversions of public domain etexts from Project Gutenberg; mostly classics.
7	Games	Every Word	IMO Amazon should segregate these in their own section.
5	Thriller/Mystery	The Perfect Woman	Mostly in the gruesome-crimes subgenre, not the sleuthing subgenre.
8	Erotica/Romance	Rough Cut	Romance readers might claim that these are two categories but I defy you to draw the line among these titles.
2	ChickLit	Stuck in the Middle (Sister-to-Sister Book 1)	Apologies for the derogatory label but what do you want me to do with a cover and title like that?
11	Christian Fiction	Fools Rush In (Weddings By Bella, Book 1)	Often disguises itself quite stealthily as other genre fiction.
6	Other Fantasy	Don't Die, Dragonfly	Mostly spirits-and-vampires stuff, not Heroic Medieval Fantasy Product.
8	Alleged Nonfiction	The Winners Manual	Includes many crappy cookbooks and the Bible.
3	Other Fiction	The Stolen Crown	Arguably the bravest authors here, as non-series non-genre fiction has the least "author stickiness" of any fiction. Which isn't to say the writing's any good necessarily.

Email, instant messaging, immediacy, and productivity

2010-12-25T06:42:00.000-08:00

Krugman's response to this NY Times trend article on the alleged death of email and triumph of instant messaging reminds me of Donald Knuth's explanation of why he doesn't use email:

Email is a wonderful thing for people whose role in life is to be on top of things. But not for me; my role is to be on the bottom of things. What I do takes long hours of studying and uninterruptible concentration. I try to learn certain areas of computer science exhaustively; then I try to digest that knowledge into a form that is accessible to people who don't have time for such study.

As it turns out, even email is insufficiently immediate for many people these days. And nobody can reasonably dispute that when you're out and about, coordinating real-world social activity, it's sometimes better to have instant messaging — "[I'm] down the street, be there in a minute, stay put" is not a useful message to receive a couple of minutes late, nor do email's strengths offer much benefit in this context.

But you choose your communications medium based, to some extent, on the type of person that you want to be. If you want to be like Krugman or Knuth, then you need long periods of uninterrupted concentration, and you must favor asynchronous communication. If you want to be like a teenager gossiping about his/her classmates, then instant messaging is probably good enough. So, the question: do you want to be more like Don Knuth or more like... well, millions of people whom you've never heard of because they never accomplished anything great? Most of us settle for something in between, of course, but this is a question of aspirations, and anyway in practice the true issue is about modifying one's behavior at the margin and not about absolute positioning.

On the other hand, it is possible to take enforced inaccessibility too far. It is worth quoting a bit from Richard Hamming's classic essay:

I noticed the following facts about people who work with the door open or the door closed. I notice that if you have the door to your office closed, you get more work done today and tomorrow, and you are more productive than most. But 10 years later somehow you don't know quite know what problems are worth working on; all the hard work you do is sort of tangential in importance. He who works with the door open gets all kinds of interruptions, but he also occasionally gets clues as to what the world is and what might be important. Now I cannot prove the cause and effect sequence because you might say, ``The closed door is symbolic of a closed mind.'' I don't know. But I can say there is a pretty good correlation between those who work with the doors open and those who ultimately do important things, although people who work with doors closed often work harder. Somehow they seem to work on slightly the wrong thing - not much, but enough that they miss fame.

eMusic: Pay again for music you've already bought

2010-11-20T14:36:00.000-08:00

One of the major advantages of eMusic in the past was that, after paying for a track, they'd remember that you'd paid for it and allow you to download it again if you ever lost the file. No more. So long, eMusic, it's been fun. In your continuing quest to court clueless businessmen at the major record labels, you've been steadily making your service worse and worse for some time now, and now I'm done with you. Amazon mp3 for me from now on.

Two steps to freeing yourself from operating system zealotry

2010-08-16T22:06:00.000-07:00

First, repeat the following mantra to yourself until you really, really believe it:

Other people's needs, habits, and experiences with software differ dramatically from mine, and are just as legitimate.

Your platform of choice may seem to satisfy your needs exactly. It may ring every single chime in the halls of your heart. But if you sincerely believe that every user who wants something different is simply mistaken, then you're presuming to a knowledge that you do not possess: namely, the knowledge of how every other user behaves.

Second, pay close attention when you're using your preferred computing device, and make a mental note every time you end up staring, slack-jawed or pissed-off or confused, while your computer does something other than what you just asked it to do. (This includes while you're waiting for some indicator to stop spinning, or when you need to stab the "Cancel" or "Back" button.) If you believe that this never happens to you, then you are not paying close enough attention, and in fact you should be frightened, because you have become so habituated to your platform of choice that you've learned to automatically edit these moments out of your consciousness. The inescapable truth is that all computing platforms suck in different ways, including yours. And until you realize this, you will not have achieved enlightenment.

Once these two conclusions sink in, you will shortly see that all OS zealotry is the futile worship of invented gods — cruel, greedy, capricious, and temporary gods to boot. Arguing with people on the Internet about the superiority of your computing bauble will seem to you like a species of insanity.

But who am I kidding. Most OS zealots are fractally wrong and it's a fool's errand to dissuade them. But maybe you, dear reader, can be saved...

Golden Promise and The Macallan

2010-08-09T22:50:00.000-07:00

Attention conservation notice: Several hundred words on whisky.

One of the signature qualities of The Macallan whisky has traditionally been its use of Golden Promise barley. It is sometimes claimed that Macallan uses 100% Golden Promise. However, in Raw Spirit (2003)*, Scottish author Iain Banks (yes, that one) writes:

Macallan uses Golden Promise barley, a variety which is out of favour with farmers these days because it produces much less yield than more recent, more productive but less tasty forms. As a result, Golden Promise has become hard to get hold of over the years and even Macallan has had to resort to other varieties, using only about 30 per cent Golden Promise since 1994. It'll be interesting to see whether the 10-year-old Macallan bottled in 2004 tastes appreciably different compared to the year before.

For all my Google-fu, I am unable to find concrete numbers on the precise barley composition of Macallan's maltings prior to 1994. It's possible that they used 100% Golden Promise before then. Certainly, the implication of the passage above is that a much larger proportion of Golden Promise was once used.

Macallan themselves seem evasive on the point. The current notes on their website avoid mentioning Golden Promise entirely, opting instead to claim that ~~they have always been at war with Eastasia~~ "The Macallan uses a proportion of Minstrel barley, a variety grown exclusively for The Macallan . . . to ensure the rich, oily character of the Macallan new make spirit". This evasiveness is understandable, given the extent to which whisky drinkers (like gourmands of other types) tend to be irrationally influenced in their judgment by untasteable qualities, such as strict adherence to tradition.

And, after all, whisky gains most of its flavor from subjecting barley to various extraordinarily refined and artificial practices — artificial not in the sense of false or disingenuous but simply in the sense of artifice as calculated construction — including malting, milling, mashing, fermentation, distillation, and barrel-aging. And therefore, it is especially irrational to develop an attachment to an exact source barley per se. In the hands of skilled distillers, one can trust that the new expression will share many characteristics of the old, and may even be indistinguishable when imbibed, even if it's not chemically identical down to the last molecule.

Nevertheless, I offer the following friendly note to anyone out there who likes a nice Speyside. If Macallan last used its original ratio of Golden Promise barley in 1993, then this year's bottling (2010) is the last 17-year Macallan made with that ratio. Likewise, 2011's bottling will be the last 18-year expression made from the same. As older Macallan tends to be an especially finely balanced whisky, and as whiskies older than 18 years tend to be exorbitantly priced**, you may wish to buy a bottle of Macallan 18 sometime in the next year to save for a special occasion. If nothing else, you will be able to impress*** your friends by telling them that this exact Macallan cannot be had any longer at any price.

*By the way, Raw Spirit is almost impossible to read from cover to cover. It may be the most self-indulgent, meandering thing Banks has ever written. On the other hand, it's entertaining to dip randomly into short passages, including some nice bits on Scotland and sundry. In many ways, the writing suffers from being incarnate as a bound codex. A whisky blog written by Banks would be a lot more fun to read. Anyway, by leafing through the book and sticking labeled Post-Its on the pages where each distillery appears, I was able to transform it into a decent introductory reference guide to Scotch, and one that's far less stuffy and pretentious than that sort of thing would normally be.

**While I'm dispensing unsolicited advice, here's some on whisky and age (this will be obvious to any serious whisky drinker, but it may be news to some readers of this blog). Some people assume that older whiskies are strictly better than younger ones. This notion is supported by the exceptionally high price of old whisky, and also pop culture artifacts like that West Wing episode when Leo rhapsodizes on the age of Johnnie Walker Blue. In fact, age changes the taste of whisky, but whether that change is an improvement is entirely a subjective matter. Younger whiskies retain more of the character of the original malting and distillation process, whereas older whiskies take on more of the character of the barrel in which they are aged. As a rule of thumb, the barrel exerts a gentling, sweetening influence. Thus some qualities, like intense peatiness, which are especially beloved of the most insufferable whisky nerds, are more commonly found in younger spirits.

***Or, more likely, annoy.

Personal digital curation: a software category that does not exist, but ought to

2010-07-22T06:00:00.000-07:00

If you're a typical computer user today, you have lots of data that you've created or participated in creating. The data takes several forms, typically including:

Personal documents: writing, drawing, photographs, home movies, etc.
Purchased media: music, movies, software, etc.
Electronic records: receipts, account records, etc.
Communication: email, chat logs, & other social media messages.

There are several things about how you keep this data today which ought to bother you.

First, it's probably inadequately backed up. By this, I mean that you don't back it up frequently enough, and when you do, it's probably either (a) on a USB hard drive or (b) on cloud storage. Taken alone, either of these is inadequate. Typical hard drive backups are inadequate because on most file systems, the data's neither pervasively checksummed nor stored with redundant error-correcting codes; as a result, all your files are subject to random corruption. There's also the small matter that you probably don't archive your backups regularly to a remote location that's safe if, e.g., your home burns down. Cloud storage alone is inadequate because (1) any compromise to your account could result in malicious deletion or corruption of your data and (2) Amazon/Google/... may seem like permanent institutions today, but on the scale of decades I am somewhat dubious; DEC and SGI and Sun were once lords of Silicon Valley; ultimately one must consider Phlebas and all that.

Second, the data's stored in a mishmash of formats, some of which will be exceptionally difficult to read in years to come. Microsoft's file formats are particularly egregious, but I also have my doubts that, for example, today's video file formats, or an iPhoto or Picasa metadata database, will be readable by commonly available software in twenty years.

Third, a lot of your data's stored in multiple related forms, and the relationship between those forms is totally ad hoc and not captured by future-proof software. For example, you might have a batch of raw photos, of which you pick a few to clean up, rescale to lower resolution, and upload to the web. So now you've got multiple versions of the photo. If you need to go trawling through this mess some years from now, you're in for a lot of curatorial tedium reorganizing it and figuring out what's redundant and discardable versus what's a pristine original that you must keep.

Conquering any one of these problems, let alone all of them, requires serious geekery today. For example, if you want to have good backups, you need to store data both in cloud storage and on multiple media, and you need software that records and verifies the checksums of all your files. The other two problems are just as gnarly, if not more so.

This may seem like a totally anorakish concern that doesn't matter to most people. Maybe most people are OK with most of their data being ephemeral, except for the rare object that they print out into physical form. Maybe it's just me, because I've been thinking about posterity a lot lately — including photos and video, my raw data generation rate has risen to something like a couple of GB per month. But I suspect I'm merely one of the people on the leading edge of this problem. Someday, everybody will generate a couple of GB per month and they really will want to share the family albums with their grandchildren without inordinate curatorial effort.

So, as far as I can tell, there's a big gaping hole in the market for personal digital curation software — software that would help you not only back up your data (there's plenty of software out there for that) but that would take care of ensuring the posterity of your data. This implies at least (1) backing it up to multiple distributed locations, (2) transcoding it into future-proof forms, and (3) remembering the relationship between different parts of your data.

This software would not be simple to build. It would have to be cross-platform. To offer a credible promise of future-proofness, it would have to be built on well-documented protocols and file formats so that if your organization went bust, someone else could write software, from scratch, that at least recovers the data. It would have to either include software that manages common file types like photos, or to hook into existing software that manages them, or compute relationships between the files after the fact (for example, it would have to either replace Picasa, or hook into it, or be able to figure out by post hoc analysis when two files were really variants of each other). It would have to be performant. It would need a nice UI.

I suppose the difficulty of building such software is one reason it hasn't been done. Much easier to just build a social networking doodad or a little timewasting mobile app or whatever the next Valley flavor of the month is. On the other hand, I think there's actually a reasonable (although perhaps tough to pitch) business case. There's probably at least tens of thousands of digital obsessives in the world who'd pay Photoshop CS-level prices for a credible digital curation package. The need to support new file formats or cloud storage APIs as they come online could provide a steady stream of upgrade revenue. If you built it right, then there's the potential for standard licensing deals where you bundle value-subtracted versions of the software with new computers, digital cameras, and other doodads.

Oh well. Anyway, add it to the list of stuff that I wish existed but does not, and also the list of things I wish I the time and focus to write but will probably not get to in my lifetime.

UPDATE 2010-08-03: Apparently you can actually learn something by blogging in ignorance and waiting until Reddit sends some commenters your way. There's an IT service category called digital asset management (DAM), and it's a big deal for enterprises (which shouldn't be surprising). (In library science, the analogous problem is called digital curation, which IMO is closer to the problem I care about.) The question, I suppose, is whether DAM can be scaled down, made sufficiently comprehensive, and encapsulated in a mixture of consumer-grade software and services so that individuals can have credible assurance that their data will be preserved on decades-long time scales. I'm somewhat dubious that Expression Media or Bridge can really offer that kind of promise (for example, those packages seem media-focused; do they back up stuff like email and source code?) but of course I haven't looked very deeply. Thanks interwebs!

Ubuntu 10.04 (Lucid) on Dell Mini 12

2010-07-20T00:07:00.000-07:00

After a recent system update, my Dell Mini 12 went on the fritz: wireless networking stopped working reliably. Obviously, that's completely unacceptable in a device like this. I guess I could have tried messing around with the configuration files and drivers, but Dell's oddball Ubuntu 8.04 (Hardy) lpia distribution has been feeling long in the tooth lately anyway. So, following my mostly satisfactory Lucid workstation experience, I decided to try upgrading my Mini 12 to Lucid.

And, once again, almost everything worked just fine.

I prepared a USB drive with the installer (actually just a memory card reader plus the card from my camera) according to Ubuntu's instructions. Then I rebooted (using F12 to bring up Dell's boot menu, and then selecting USB), and selected installation.

The installer was exceptionally sluggish — for which I blame the Mini 12's underpowered hardware — but otherwise the installation went through without a hitch.

If you try the same with your Dell Mini 12, you'll want to look at these notes before you run the installation. Two post-install tweaks are necessary:

To get acceptable graphics performance, you'll need to enable the Poulsbo GMA500 proprietary driver.
For the wireless networking you'll have to enable the Broadcom STA wireless driver from the System -> Administration -> Hardware Drivers menu.

Overall I'm mostly satisfied. Lucid both looks and feels much slicker than Hardy, from the fonts to the windows. The desktop distribution's UI works fine on the 12 inch screen. And once you perform the tweaks above, most everything in the hardware works fine, including wireless, bluetooth, trackpad, sound, and the webcam.

The fly in the ointment this time? Suspend and resume are sometimes flaky. In particular, sometimes resume either fails completely or requires that I switch virtual terminals a couple of times (Ctrl-Alt-F2, Ctrl-Alt-F7) to jog it out of its slumber. Given the way I use this device, it's actually less of a big deal than you might expect (basically, when suspend fails, I just hard-reboot and restart my web browsers and emacs), but if this matters to you a lot then you might want to hold off. I guess I could try debugging the problem, but like I said it hasn't been that important to me.

So, OK, I have to admit that owning this computer overall hasn't been a seamless experience. (But then nothing is these days, not even my Macbook Pro from work; I've had many travails with MacPorts and Fink and X11.app and...). When I bought the Mini 12, my goal was to see whether a sub-$700 computer could keep me satisfied for more than one year, which would make it more cost-effective than a $2k computer which typically lasts me 3 years. In that sense, the experiment succeeded: it's lasted over a year, and I've gotten good mileage out of it. Meanwhile, I have not cursed at my Mini especially more than I've cursed at any other computing device I've ever owned. And the biggest positive qualities — compactness, light weight, near-silent operation — remain salient even today.

Virtual cosmetics, redux

2010-07-19T23:09:00.000-07:00

OK, well, I thought they'd do faces before bodies but the basic motivation isn't far off (research paper link.

UPDATE 2011-01-09: Updated YouTube link; the original appears to have had its permissions toggled to private. Also added link to original research.

On the verbosity of Java generics and related type systems

2010-06-09T20:25:00.000-07:00

So, recently T. B. Lee tweeted:

Java's generics syntax feels really clumsy. Are there other (strongly typed) languages that do it better?

I replied as best I could in 140 characters while riding a crowded Muni bus home after work, but I mangled the explanation, so I think I should rectify this.

First, it is worth comparing Java generics to other languages that have parametric polymorphism. For example, consider ML (and more generally the Hindley-Milner family of languages). In ML, you don't need to write down types most of the time, because of type inference. Whereas in Java you might write:

<K, V> LinkedList<K> keysAsList(HashMap<K, V> aMap) {
  LinkedList<K> result = new LinkedList<K>();
  for (K key : aMap.keySet()) {
    result.add(key);
  }
  return result;
}

in OCaml you can write:

let keysAsList aMap =
  Hashtbl.fold (fun key _ rest -> (k::rest)) aMap [];;

Notice that although we had to annotate all the Java variables with types, there's not a single type annotation in the OCaml code (Hashtbl.fold is the name of a function qualified by its module name; it is not a type annotation). But OCaml is statically typed nevertheless.

So, one might ask, what gives? Why can't you just add type inference to Java?

Well, the short answer is that typechecking Java generics is a fundamentally harder problem. ML has only parametric polymorphism; Java has both parametric polymorphism and subtype polymorphism (i.e. the object-oriented kind). It is perhaps not obvious why this makes things hard until you learn that B. C. Pierce proved in 1992 that bounded quantification in F_≤ (pronounced "F-sub") — a formalization of the most straightforward and general combination of subtyping and parametric polymorphism — is undecidable.

In other words, in F_≤ it is possible to write programs for which the type checker would not terminate. This is generally held to be a bad thing (n.b. I disagree with the prevailing opinion, but that's a discussion for another day), so over the next decade or so there followed several papers by Pierce and others attempting to isolate calculi weaker than F_≤ with type systems that were both usable and decidable. The most practically relevant outcome of this work was Featherweight Java, which provided the formal foundation for (most of) Java generics. C#, Scala, etc. build on this line of work, although in Scala's case fairly indirectly.

What does all this have to do with type inference? Well, nothing directly. But for any given level of type system expressiveness, full type inference is at least as hard as type checking. And the type checking problem for Java with generics already lives close to the undecidability ceiling (in fact, the decidability of Java generics with wildcards is, AFAIK, still an open problem; proving this sort of stuff used to be a hot research subject but I think everyone's gotten bored of object calculi and moved on). Oh, and I should mention that many of the best minds in academic language design have thrown themselves at the parametric polymorphism + subtyping + inference problem at one time or another, and come up empty. Now, none of this is hard proof that much richer type inference for object-oriented languages with generics is impossible, but it all hints strongly in this direction; at a minimum any such system is likely to be exceptionally intricate and difficult to prove sound.

So, basically, I believe that it is unlikely that anyone will come up with a fundamentally more concise type system for a programming style that combines (1) objects, (2) generics, and (3) static typing. At best, people will fiddle around on the margins — using different punctuation, for example, to denote type parameters, or making other parts of the language more terse to compensate.

That said, although general type inference seems hopeless, there are clearly some things that could make Java's generics more syntactically lightweight in certain common cases. For example, it would be trivial to infer the type of a variable at any declaration site with an initialization expression. C# and Scala appear to do some of this.

Ubuntu Lucid update

2010-05-15T10:28:00.000-07:00

Following my disappointment with Kubuntu Lucid, I just got around to replacing it with the standard Ubuntu Lucid desktop. It's possible to switch desktop environments using a couple of package manager commands, but I decided to do a from-scratch reinstall.

With a little effort, I was able to make most of the Ubuntu desktop behave OK. Window management is still not up to par with KDE 3.5 + KStep window decorations, but it's enough for now. I'll probably switch my window manager to WindowMaker at some point. (NeXTSTEP-style window decorations with X11 window management gestures are the apex of desktop window management, for reasons that I could go into at length but won't today.) Visually, the new Ubuntu theme looks nice; in fact it looks and feels much better when you're using it than it does in screen shots.

However, there's one fly in the ointment. Sound didn't work. At all. Note that for all its flaws, Kubuntu, which is derived from the same base distribution, had no such problem, so it isn't simply a driver issue. I could bore you with all the details of my debugging adventure, but at the end of the day I blame PulseAudio, and Ubuntu's decision to make PulseAudio central to their desktop sound system. After a couple of hours of unproductive web searching and config file wrangling, removing the PulseAudio packages in Synaptic made sound work, sort of.

Yes, sort of. It still doesn't work quite right. When I open the System > Preferences > Sound menu, I get a dialog box saying "Waiting for sound system to respond" and nothing else. (This behavior occurred before I uninstalled PulseAudio, so that's not the cause.) Apparently a whole lot of people have run into variations of this problem since at least Ubuntu 9.10, and nobody seems to have definitive answers on how to solve it. I'd report it as a bug, but I suspect that it's one of those opaque symptoms with dozens of underlying possible causes and it's probably futile.

I want to emphasize that I haven't had a problem like this with a Linux distribution in years. This is literally a regression in behavior to Linux ca. 2005. Poking around by hand with .conf files in /etc just to get something working on my desktop is something I used to do. It's not something I expect to be doing in the year 2010.

So, anyway, I can't set my sound preferences. I guess I'll just have to cross my fingers and hope Ubuntu didn't assign any really annoying sounds to desktop events.

How to design a popular programming language

2010-05-11T21:06:00.000-07:00

This has been kicking around in my brain for at least half a decade, and if you know me well then I've probably spoken it aloud in your presence; so it's high time to get it down in writing. Here is my Grand Unified Theory of Programming Language Adoption. There are three steps:

Find a new platform that will be a huge success in a few years.
Make your language the default way to program on that platform.
Wait.

That is all. Note that none of the above steps has anything to do with the language design itself. In fact, nearly all popular languages are terribly designed. Languages become popular by being the "native" way to program a certain kind of system. All of history's most widely used programming languages fit this model — Fortran (scientific programming), C (Unix), C++ (MS Windows), JavaScript (web pages), Objective-C (Mac OS X), . . .

Or, in fewer words: Languages ride platforms to popularity.

Why is this so? Well, to a first approximation, no piece of software ever gets rewritten in another language; and once a critical mass of software for a platform has been written in one language, nearly all the rest will follow, for two reasons:

Nobody has figured out how to make cross-language interoperability work well.
The network effects from language adoption are immense. Programming is, despite appearances, a deeply social profession. To write successful software quickly, you must exploit the skills of other programmers — either directly, by hiring them, or indirectly, by using library software they've written. And once a language becomes the most popular in a niche, the supply of both programmers and libraries for that language rapidly accumulates to the point where it becomes economically irrational to use any other language.

In fact, I claim that in all the history of programming languages, no language has ever successfully unseated the dominant language for programming on any platform. Instead, a new platform gets invented and a new language becomes the "founding language" for that platform.

Well, OK, there are exactly two exceptions: Java and Python. It took me a while to figure out what happened in those cases, and the answers I came up with were surprising (to me).

Java is anomalous because although it is widely used in its primary domain (Internet application servers), it is not predominant, the way that e.g. C++ is predominant in writing native Windows GUIs. My explanation is that the web architecture has a uniquely high-quality interoperability protocol in the form of HTTP and HTML(/XML/JSON/...). Hey, stop laughing. HTTP and HTML fail all kinds of subjective measures of elegance, but they succeed in isolating clients and servers so well that it is economically viable to write the server in any language. In other words, as unbelievable as it sounds, HTTP and HTML are the only example in history of cross-language interoperability working really well.

I'll abandon this explanation if I can find, in all the annals of computing, another protocol that connected diverse software components as successfully as HTTP and HTML. The only things I can think of that come close are (a) ASCII text over Unix pipes or (b) ODBC, and neither of these provide nearly the same richness or connect components of similar diversity.

Python is anomalous because rather than riding a new platform to success, it simply seems to be displacing Perl, PHP, etc. in the existing domains of shell scripting, text processing, and light web application servers. My explanation is that Python appears to be the only language in history whose design was so dramatically better than its competitors' that programmers willingly switched, en masse, primarily because of the language design itself. This says something, I think, both about Python and about its competitors.

Incidentally, this theory predicts that all the new(ish) programming languages attracting buzz these days — whether Ruby, or Scala, or Clojure, or Go, or whatever — will fail to attract large numbers of programmers.* (Unless, of course, those languages attach themselves to a popular new platform.)

UPDATE 2010-05-15: Reddit and HN weigh in.

*Which is fine. Very few languages become hugely popular, and in fact nearly all languages die without ever seeing more than a handful of users. Being either influential (so that later languages pick up your ideas), or even merely useful to a significant user population, are fine accomplishments.

On internationalized TLDs (a contrarian opinion)

2010-05-06T20:53:00.000-07:00

The Timberites rejoice. I'm obviously revealing my North America-centric roots but I think that this is a huge amount of cost for insufficient benefit.

Great civilizations leave their stamp on the conventions of world culture. The Romans gave us their calendar; the Indians gave us the modern numbering system; the Italians gave us terms and symbols used throughout the Western world in musical notation (piano, fortissimo, crescendo, ...). The global recognizability of these signifiers is part of what makes them useful.

In the modern era, American culture predominates in computing. In practically every programming language, English words like begin, define and integer (or abbreviations thereof) have special meanings understood by every programmer in the world.

With respect to TLDs, there are two alternatives before us. Alternative one is to make everyone in the world simply learn to use ASCII TLDs. Alternative two is to make everyone in the world learn to use, or at least recognize, TLDs in every Unicode script. Alternative one is actually the simpler alternative, even for non-English speakers.

Imagine if numbers were subject to the politics of modern i18n. We would have the modern positional decimal numeric system, but also the Roman numeral system, and the Babylonian numeral system, and so on, and nobody would ever have asked anyone to standardize on any of them. After all, we have to be sensitive to the local numeric culture of the Romans!

It's not like I'm saying people should communicate in English all the time. I'm only saying that people learn to type and to recognize ASCII TLDs. This is a relatively limited set of special-purpose identifiers. There are only about an order of magnitude more ccTLDs than months in the year or decimal digits. And I would claim that it's useful for everyone in the world to recognize that, say, .uk and .com look like the end of a domain name, whereas .foobar123 does not. Pop quiz: which one of the following is a new Arabic ccTLD, مص or مصام? The reason you can't recognize it is not just that you're an English speaker — people who only speak Mandarin or Spanish or Russian are in exactly the same boat as you. And when ICANN unveils the Chinese Simplified or Cyrillic or Bengali TLD scripts, Arabic speakers in turn won't be able to make heads or tails out of those.

But, whatever, my opinion's on the losing side of history, so it's almost pointless to express it. I just thought I'd get it out there that there was a real benefit to, and precedents for, the status quo where a convention originating in one culture diffuses and becomes universal.

Kubuntu Lucid and KDE 4 reactions

2010-05-01T09:21:00.000-07:00

Speaking of how software makes you dependent on other people, the newest Ubuntu Long Term Support (LTS) release just came out. This means that in a year, support for the previous LTS release will wind down; which in turn means that Ubuntu users must upgrade sooner or later, unless they want to sacrifice security updates and compatibility with new releases of third-party software.

So, I took the plunge: yesterday I downloaded and installed Kubuntu Lucid.

This is the first Ubuntu LTS release that runs KDE 4, the latest major revision of KDE. I've been using KDE for about 11 years, ever since version 1.1. My immediate reaction was simply that KDE 4 is a mess. And after playing around for a few hours, tweaking settings, and trying to settle in, I still think KDE 4 is a mess. As I use it more, I'm not settling into it; I'm simply accumulating more irritations.

Without exhaustively listing all the details, my complaints basically break down into three categories.

First, there are pervasive performance problems. In every corner of the UI, "shiny" effects have been prioritized over responsive, performant interactivity. To take just one example, under KDE 3.5, the Amarok media player used to be super snappy and responsive; it left iTunes or Windows Media player in the dust. In KDE 4, Amarok takes a couple of seconds to expand one album or to queue up songs, and resizing UI panels is painfully slow and janky. (My workstation has a 2.13GHz Core 2 Duo and a good graphics card. This should not be happening.) Similar problems can be observed in the desktop panels, file manager, etc.

Second, in general, the UI changes seem designed to push KDE's new technology into your attention space, rather than getting out of the way so you can accomplish tasks. Again, here's just one example: in the upper right corner of the desktop, there's a little unremovable widget that opens the "activities" menu:

The upper right corner of the desktop is a hugely valuable piece of screen real estate. By placing this widget in the upper right corner, the developers are signaling that this menu contains operations which will be frequently accessed. Do they really think users will add new panels to the desktop frequently? (For non-KDE users, a "panel" is KDE's equivalent of the Mac OS X dock or the Windows taskbar.) So far, almost every time I've clicked this widget has been by accident while trying to close or resize a window.

If you're a desktop developer who wants to show off your technology, this design may sound good: you put this menu there to make sure users discover your desktop widget and "activities" technology*. However, if you're a user, then this menu mostly gets in your way, and you wish it were tucked away somewhere more discreet.

Third, the KDE 4 version of every application has fewer features and more bugs than the KDE 3 version. The "Desktop" activity no longer has a way to "clean up" icons without repositioning all of them in the upper-left-hand corner. The Konsole terminal application's tab bar no longer has a button from which you can launch different session types. The list goes on.

Anyway, of course, I don't pay for KDE, and so in some sense this is all bitching about free beer. However, suppose I did pay for KDE. Would I have any more input into the process? Windows users pay for Windows; if you don't like the direction Vista and Windows 7 are taking the UI, do you think you personally have any chance of influencing Microsoft's behavior? Mac users pay for Mac OS X; if you disagree with Steve Jobs, do you have any chance of influencing Apple's behavior? In fact, you do not, and both user populations have experienced this reality multiple times in the past decade. Mac users loved the Mac OS 9 UI but they had to give it up when Apple stopped supporting it on new Macs. Microsoft users who are attached to the Windows XP UI will likewise be forced to give it up eventually, when Microsoft stops sending security patches.

The KDE 3 to KDE 4 transition is simply KDE's version of the OS 9 to OS X transition, or the XP to Vista/7 transition. Except that those seem to have worked out OK in the end, whereas KDE 4, which was released over two years ago, seems to have lost its way permanently.

I'm writing this post not just to point out KDE 4's defects — I mean, it feels good to vent, but who really cares — but also to marshal further evidence in support of my contention that owning software doesn't mean much anymore.

Even the fact that KDE is Free Software means little in this case. I mean, what am I supposed to do now? I can't stay with the previous Ubuntu LTS release forever, unless I want to expose myself to security risks, and also be unable to run or to compile new software, both of which are deadly for a software developer. Conversely, I can't singlehandedly maintain a fork of the KDE 3 environment forever; this guy's trying but without a large and active community behind the project, it's doubtful that it will remain current for long. And frankly, I'm getting older, and I don't have enough time to invest in both hacking around with my desktop environment and also accomplishing the other things I want to accomplish in my life.

So, I can either (1) suck it up and live with KDE 4, or (2) abandon the desktop environment I've grown to love over the past 11 years, and jump ship to GNOME or something. (Right now I'm leaning towards (2).) Adopting software means making a calculated bet on the behavior of other people. And sometimes you lose.

*BTW "activities" are 80% redundant with virtual desktops and therefore hugely problematic and confusing as UI design, but I won't get into that.

In which an Icelandic volcano prompts the funniest paragraph on the Internet today

2010-04-18T18:59:00.000-07:00

Yglesias writes:

Ever since the eruption, I know I can’t be the only person who’s been wondering how to say “Eyjafjallajökull.” In principle, the Internet and its multimedia cornucopia ought to shed a lot of light on this issue. In practice, no matter how many times I click over here and hear it pronounced, I can’t come any closer to saying it myself.

I thought oh come on how bad can it be. And then I clicked through to Wikimedia and I burst out laughing.

Cynicism and libertarian ends

2010-04-18T10:17:00.000-07:00

So when I wrote that cynicism about government does not help the libertarian cause, a libertarian might regard the suggestion with suspicion, given that I'm just another big government liberal. Well, at least one libertarian agrees with me. And his NYTimes column actually contains a lot of stuff that I find pretty risible.

Computer science and the iPhone developer agreement

2010-04-11T09:27:00.000-07:00

Full disclosure: I work for Google. However, this blog reflects my personal opinions only.

Programming and computer science are not synonymous, but obviously the two are deeply intertwined. The fundamental activity of programming is the construction of abstractions. Programming language design and implementation is one of the fundamental forms of abstraction building. It is central to the field, and has been so nearly since its inception. One of the oldest and most important research conferences in computer science is named Programming Language Design and Implementation.

This suggests a particular understanding of what Section 3.3.1 means. Section 3.3.1 says: "Thou shalt not build abstractions other than those we prescribe." It bans one of the fundamental activities of programming.

This would be a mere curiosity, except for Apple's unusually influential position in the computing industry. All trends point towards mobile devices* becoming much more pervasive than all other general-purpose computing devices. Indeed, the combination of mobile and cloud computing may someday replace all other user-visible hardware except what's needed to support input and output (screens, cameras, etc.). And Apple has the credible goal of becoming the preeminent mobile device provider, setting standards for the industry and defining the entire computing experience for a huge swath of future computer users.

Section 3.3.1 therefore constitutes a direct attack on computer science, delivered by a powerful and well-funded organization that aims to transform laypeople's interface to the field. As long as 3.3.1 stands, for a computer scientist to purchase an iPhone or iPad is akin to a biologist purchasing a textbook that advocates against teaching evolution. Full stop. Go ahead and do it if you can't resist the shiny, but understand the moral weight of the decision you're making.

I can already hear people ready to trot out the standard roster of excuses. Hit the comment box if you want, but realize that I've anticipated the common objections and the only thing stopping me from preemptively rebutting them all is the fact that I'm moving soon and I have a huge number of boxes to pack. To pick just three examples:

Q: "The iPad isn't for people like you. Why do you care?"
A: "This post isn't for people who don't care. Why are you reading it?"

Q: "Apple has a right to do whatever it wants with its platform. If you don't like it, you shouldn't use it."
A: "Thank you for agreeing with me."

Q: "You can program whatever you want in HTML5 and access it through Safari."
A: "Yes, the web is an open platform, which Apple fortunately does not control.** I'm talking about Apple's rules for programming on the platform that it does control."

*A.k.a. "phones". Incidentally, I think the British slang "mobile" is more elegant and generalizes far better any of the {cell,smart,super,...}phone terms that are used on this side of the pond.

**Although I will remark that it's naive to imagine that a platform can be preeminent for very long without influencing the market of content and applications to which non-participants in the platform regime have access. There's a reason Hulu used to work on Flash only. But that's a post for another day.

Button order in web browsers and phones

2010-03-09T07:19:00.000-08:00

Disclaimer: I work for Google; however I do not work on the Android team and this post reflects my personal opinions only.

Do you have an Android phone? Look at your Chrome toolbar. Now look at your Firefox toolbar. Heck, look at your Internet Explorer or Safari or Opera toolbars, if you have those handy. Now look at your phone's face buttons. Notice anything?

The Nexus One gets it right:

Likewise the Droid:

A smartphone is an Internet access device, and as such its interface should strive for consistency with the most widely-used Internet access interface: the web browser. Maybe the Menu button could go to the right of the Home or Search buttons, but it seems obvious to me that on a smartphone with Back, Home, and Search buttons, those buttons should appear from left to right in that order. And therefore, for example, the Back button should not be on the right side of the phone.

Alas, via the gadget blogs in my RSS reader, I find that many Android smartphones permute button order in violation of this principle. One might speculate that button order is a manufacturer-specific quirk, except that the layout changes even within a single manufacturer's devices.

What's going on here? Does this parallel not occur to the designers of these phones? Do they disregard consistency for the sake of trivial product differentiation? Or is it just that I'm the only person bothered by this?

Q: Why does string get tangled up in knots?

2010-02-26T06:55:00.000-08:00

A: Because it can.

This is not a joke answer. The question is very nearly tautological. A knot is, by definition, a conformation of string which resists untangling. If you perturb a length of string randomly, then at any given time it may either become tangled in a knot or remain free. If it is in a knot, it will resist becoming a non-knot; if it is not a knot, it is free to change. If a knot is possible, one will eventually emerge. The only way that a knot could fail to emerge is if it were completely impossible for an unknotted string to tangle into a knot.

A very similar question would be why a shaken ratchet eventually turns forward. Of course, nobody would ask that question. The lower dimensionality means that the answer is transparent.

There is a similar result in evolutionary population dynamics which states that as time goes to infinity, given a fixed population cap and a randomized chance to reproduce, every species goes extinct. Intuitively, if you roll k dice infinity times, then eventually they all come up sixes.

The many, many applications of this principle are left as an exercise for the reader.

Tab completion for meta-bang shell commands (Wednesday Emacs blogging)

2010-02-24T06:36:00.000-08:00

You probably use M-! to run a quick shell command now and then, when you don't want to be bothered with a full M-x shell. But if, like me, you're a former XEmacs user, then you probably find FSF Emacs' default lack of tab-completion for files in the minibuffer rather annoying. Well, your pain ends here:

(if (not (string-match "XEmacs" emacs-version))
    (progn
      (defadvice read-from-minibuffer
        (around tab-is-pcomplete-in-minibuffer activate)
        "Bind TAB to pcomplete in minibuffer reads."
        (let ((keymap minibuffer-local-map))
          (define-key keymap "\t" 'pcomplete)
          (ad-set-arg 2 keymap)
          ad-do-it))))

Ta-da. Now when you M-! mv LongAnnoyingFileName.java LongAnnoyingFileNameFactory.java, you'll be able to tab-complete the filename.

Incidentally, who knew that elisp supported aspect-oriented programming? Apparently it does. Astonishing. I owe this tip to a co-worker, who I'd name except that I doubt he'd want to be associated with the other content on this blog. (p.s. AF, if you ever run across this post and don't mind being credited, I'll happily add your name.)

Meta-slash performs dabbrev-expand, and you require this knowledge (Wednesday Emacs blogging)

2010-02-17T05:10:00.000-08:00

If you don't know this one already, then go to an emacs window right now, open up any source file (~/.emacs works fine), navigate to any function, and type the first couple of characters of a nearby identifier. Then type M-/. Ta-da!

Details: By default M-/ is bound to dabbrev-expand, which triggers the dynamic abbreviations facility. This dynamically compiles a dictionary from nearby identifiers in the source file, and offers matching identifiers as completions for the current token, preferring identifiers closer to the cursor over more distant ones.

For bonus points, type M-/ multiple times to cycle among recent matches, or use C-M-/ to pop up a list of matching completions in another buffer.

dabbrev-expand isn't as sophisticated as the semantically aware tab-completion available in many IDEs. Conversely, however, it works with no modification in almost every buffer type under the sun, so you can use it when editing code in elisp, Java, or the language you invented this morning. It even completes reasonably well when editing English prose (although since token prefixes are much less unique within an English document, it's only worthwhile for longer words).

Meanwhile, because dabbrev-expand's algorithm is so simple, it doesn't require a heavyweight background process to scan all your project files and keep an in-memory database up-to-date. This is, of course, a typical IDE pitfall. You'll never be waiting for emacs to repopulate the dabbrev-expand database after you refresh all the files in your project checkout.

I'm a little embarrassed to admit that I only learned this keyboard shortcut a couple of months ago. Yes, that's right, I've been typing all my identifiers manually (or using M-w/C-y to copy-and-paste) for my entire freaking career. I estimate that my long-term danger of RSI declined dramatically the day one of my teammates mentioned this feature.

(On the other hand, my incentive to keep names short has been reduced slightly, and I wonder what effect this will have on the code that I write. It seems to me that although names that are too short can be cryptic, it's good to keep code as concise as it can be, consistent with maintaining clarity. Along similar lines, I suspect, for example, that IDEs which make it too easy to extrude large volumes of boilerplate code, or to import functions from many different modules, result in looser, less organized code.)

The state of Kindle backups and data portability, February 2010

2010-02-16T07:00:00.000-08:00

I recently plugged my Kindle into my workstation's USB port for the first time. In ordinary operation, there's no need whatsoever to do this, but I wanted to try backing up my ebooks. Also, since writing this I wanted to confirm my suspicion that the current Amazon DRM scheme is more akin to Apple's FairPlay "speed bump" than a serious playback control technology.

In short, it is.

The Kindle connects as an ordinary USB mass storage device with a simple folder structure, containing four root-level directories:

Audible: audio ebooks? (empty in my case)
documents: ebooks
music (empty in my case)
system: Not exactly what it sounds like — it doesn't actually contain the operating system, only auxiliary data files used by system software. I suppose it's sensible enough not to let some clueless user bork their OS by accidentally dragging this file to the trash. (I suspect that there's a backdoor code that will mount the OS/firmware as well; at least, that's how I'd design this device if I were a developer and wanted to debug it.)

For each ebook, the documents folder contains at least one .azw, .azw1, or .tpz file, and usually a .mbp or .tan file that stores some auxiliary data. Your "clippings" file (containing excerpts that you highlight or note) is stored as a plain .txt file (yay).

Free samples and free public domain ebooks from Amazon are not DRM-restricted. Purchased books, of course, are.

Incidentally, no technology in the Kindle device prevents copying. As noted, the Kindle mounts as an ordinary USB mass storage device, and it is inherent in the filesystem abstraction that you can do simple things like copy the entire contents of the documents folder onto your hard drive. You can do it once or a thousand times, and no technology even tries to stop you. This is an inherent function of the type of device that Amazon has made.

What the files' DRM prevents, in theory, is "playing back" the files' content on some other device after it has been copied. But of course, it doesn't really do that in practice. Without going into details, there are downloadable programs on the Internet, widely available in source and executable forms, which can extract the contents of a restricted AZW file.*

So, in short, it's trivial for you to back up your ebook library. If you're a programmer, it's also pretty easy to write a script that will harvest your entire ebook library, shuck off the obnoxious DRM enclosure, and transcode the contents into some other format. Nontechnical users, unfortunately, don't have easy access to the DRM removal/transcoding step, although this may change as the transcoding software matures and distribution channels route around the legal jurisdictions where this software is banned.

Anyway, as I wrote in my earlier post, I'm hoping that the content production cartels will eventually realize that DRM serves Amazon's interests, not theirs, and abandon even the "speed bump" DRM currently in place. In the meantime, I've found that the value of having a dozen unread books in my bag at any given time, and being able to buy and read a book instantly at midnight on a Sunday, is sufficiently huge that I'm willing to make the compromise.**

*Even if this weren't true, people determined to infringe copyright for monetary or other gain will do so. DRM does not prevent the widespread, willful, uncompensated distribution of copyrighted content. The only thing that DRM does is prevent legitimate paying customers from getting the value from their books that they have been promised by electronic booksellers' use of the phrase "buy this book".

**A compromise, incidentally, that I was never willing to make with iTunes DRM. I suppose that the value I get in my life from reading greatly exceeds the value I get from music.

Q: How is Spock like a fortune cookie?

2010-02-15T06:38:00.000-08:00

A: His major lines are vastly improved when you add the suffix "in bed". For example, in last year's Star Trek movie:

(To Uhura): "I need everyone to continue performing admirably. In bed."
(To himself): "Do yourself a favor: put aside logic, and do what feels right. In bed."
(To himself): "As my customary farewell would seem oddly self-serving, I will simply say: Good luck. In bed."
(To Kirk): "I will not allow you to lecture me about the merits of emotion. In bed."
Kirk: "You know, traveling through time, changing history... that's cheating."
Spock: "A trick I learned from an old friend. In bed."
Spock: "Furthermore, you have failed to understand the purpose of the test."
Kirk: "Enlighten me again."
Spock: "The purpose is to experience fear, fear in the face of certain death, to accept that fear, and maintain control of oneself and one's crew. This is a quality expected in every Starfleet captain. In bed."
Bones: "You know, back home we have a saying: 'If you wanna ride in the Kentucky Derby, you don't leave your prized stallion in the stable.'"
Spock: "A curious metaphor, doctor, as a stallion must first be broken before it can reach its potential. In bed."

Electronic goods markets: end-to-end wins?

2010-02-11T00:03:00.000-08:00

Hypothesis: As industries of cultural production adapt to digital distribution, content publishers in each industry will follow, with minor variations, the four-phase pattern set by the music industry:

Denial: Publishers pretend that digital distribution does not exist, attempting to salvage business models based on distribution of physical media. In some cases, publishers use the legal system to try to make this fantasy a reality. Regardless of the legal outcomes, this proves unsustainable in the long run. During this phase, publishers may make halfhearted forays into digital publishing, which invariably fail because they are deeply and deliberately user-hostile.
Faustian Bargain: A technology company designs a system which disguises computers' fundamentally general nature with a fig leaf of DRM. The disguise allows this company to strike a deal with major content publishing cartels to distribute content. Because a technology company has taken control of the technology, the system finally works in a way that doesn't make customers want to tear their hair out. The DRM system fails to prevent widespread copyright infringement, but it provides a hook for the technology company to build a vertically integrated stack which is somewhat inconvenient for customers to exit.
Clash of the Titans: Publishers realize that they are in a weakening bargaining position with respect to the technology company, which has acquired considerable monopsony power due to its control of the platform. Publishers butt heads with the technology company over prices and other contractual terms. This, too, proves unsustainable.
End-to-End Wins: Publishers realize that architectures which embed control in the distribution mechanism put more power in the hands of middlemen than endpoints. Conversely, end-to-end architectures, wherein the endpoints negotiate the transaction and any number of interchangeable mechanisms carry data between them on a best-effort basis, place power in the hands of endpoints rather than middlemen. Publishers furthermore realize that publishers and customers are the endpoints; that in the long run both are best served when the customer can purchase a bundle of data which is not bound (even weakly) to the sales channel, the software stack, or the physical device, all of which are intermediaries between the content and the customer. Publishers finally offer their content in a portable format via multiple sales channels.

This is just a hypothesis. I'm not sure I believe it. However, as evidence that expecting the final stage is not laughably utopian, I offer Sony and Warner's deals with eMusic and the introduction of MP3s on iTunes as evidence that stage 4 is already happening for music.

Detailed application of the above model to current hoopla in the e-book market left as an exercise to the reader. However, I will note that one reason I bought a Kindle is that I thought book publishers were so ornery, retrograde, and technophobic that they'd never progress to stage 4 unless they had an obnoxious would-be monopsonist (viz., Amazon) to frighten them through stage 3.

(A counterpoint to the above argument would be to observe that certain goods, like streaming video and computer games, appear to be evolving in the direction of fairly strong architectures of control. Neither Netflix streaming nor Steam give you much freedom w.r.t. your "purchase". It's unclear whether this means their respective markets haven't progressed far enough yet, or there's something fundamentally different about these media.)

J. Blow: Games as Instruments for Observing Our Universe

2010-02-08T22:40:00.000-08:00

You might be dissuaded from listening to this talk by Jonathan Blow because it's distributed as a PowerPoint presentation and a couple of MP3s, or else because it's nominally about the much-maligned artifacts of human civilization commonly called "video games".

You would be making a mistake.

Jonathan Blow is a minor genius, and this talk is worthy of attention from anyone interested in science or art or really any creative activity. I have previously mocked video game apologists for viewing games as a failed (or at least not-quite-successful-yet) aspirant to "interactive cinema" — the teleological destiny of gaming, by this aesthetic, being the creation of an action movie in which You! Are! The! Hero! — and Blow is perhaps the most articulate proponent of the opposite view.

E. W. Dijkstra famously said that "Computer Science is no more about computers than astronomy is about telescopes." He was suggesting that there are properties of the universe — viz, certain mathematical truths — that can only be inspected by studying algorithms, which humans can only do through the construction of computing devices. Per Dijkstra, the devices are not the point, or at least not the only point.

In practice, most of computer science amounts to cleverly engineering around messes that humans have created; but sometimes you do glimpse something which appears to be a property of the broader universe. This is a point that is mostly unappreciated by non-computer-scientists, who assume that the essence of computer science is fiddling around with gadgets.

Similarly, the word "game" applies, in the broadest sense, to any system of rules with which one or more agents interact. Blow's basic point is that the generative systems of rules that we call games can be profound devices for exploring truth, just like the generative systems of rules we call algorithms. But that's a pretty inadequate summary of the talk. You should really listen to the talk itself.

(The Q&A is longer and somewhat more inside-baseball w.r.t. the Game Industry as it actually exists today, and therefore less interesting overall, although there are some good bits there too.)

Accenture ad, SFO gate 41

2010-01-01T12:30:00.001-08:00

Do you think this ad was made before or after?

On the changing nature of software ownership; with a digression on horniness and the troubling interdependence of human beings

2009-11-30T07:31:00.000-08:00

Sometime in the late 1990's, I realized that "ownership" of software had become meaningless.

Well, OK, I didn't grasp the full implications at the time. But the seed was planted when I was downloading the third or fourth or tenth macro virus protection patch for Microsoft Word 95 or Word 97. Maybe it was for my family's computer, or a friend's; or maybe it was for mine. I don't remember exactly.

However, I do remember being troubled by the implications of what I was doing. Ownership carries connotations of permanence, and the reassurance of physical possession: You own this, so no one can take this away from you. And I could indeed purchase a disc, containing a piece of software, which no one could take away from me. But this software had bugs, which made it vulnerable to security exploits; and I still needed to open files created by other people. This, in turn, implied that I needed to continue downloading updates for as long as I used the software. Even if I saved all updates available at any given time onto another disc, I would still be vulnerable to new exploits discovered later unless I returned to the mothership for more.

Therefore, unless I was willing to live a completely isolated existence, never accepting anyone else's data, I would always be dependent on the software's manufacturer.

And how long would they continue to provide updates? The answer was clearly "not forever". A few years later, a more precise answer came, in the form of Microsoft's Support Lifecycle Policy, which guarantees security patches for a certain number of years depending on the type of the product.

So, when I "bought" software, I was really paying for an implicit, limited-time support contract with the manufacturer. Without this ongoing relationship, my software would rot to the point of uselessness, as surely as a tomato on a blighted vine.

And the turn of the century also brought a new and particularly blighted vine for your software to rot on: the broadband Internet connection. Whereas before, viruses spread through casual sharing of floppy disks and occasionally file shares on your local network, a broadband Internet connection literally exposes your machine continuously to the cleverest and most malicious hacker in the entire world. And so it is that today, a vulnerable machine becomes infected within minutes of being connected to the Internet (on a non-firewalled network).

Lest the Mac or Free Software weenies (n.b. I am both) start gloating about the inferiority of Microsoft software, I hasten to point out that every nontrivial software system has a similar update schedule. How often must Ubuntu users run sudo apt-get update? How often does Firefox issue a point release? How often do Mac or iWhatever users get told to run Apple Software Update?

Therefore, software updates are an inescapable, relentless fact of Internet-connected life.

And security exploits are just the most obvious and unavoidable source of bit rot. Consider what happens when the organization — company or community, formal or informal — behind your favorite software decides to move on to the next version of a data format or protocol, and you decline to upgrade. New web pages stop being viewable; you stop being able to watch videos, play music, read documents, chat with friends, or really do anything involving other people.

Of course, you can choose to live like a virtual hermit, never sharing anything with anyone. But this comes with the same drawbacks as being a hermit in the physical world. Gains from trade and specialization of labor aren't just abstractions. Buying a jug of milk at the grocery store takes far less effort than raising and milking a cow. In some cases the difference isn't even a matter of effort but of possibility: it is impossible for an individual to manufacture an airplane, or even something as humble as an aluminum can, without relying on a massive infrastructure of manufacturing, transport, and trade provided by society. And likewise it is impossible for you to singlehandedly fork the Firefox source code, in any nontrivial way, if Mozilla takes it in a direction you don't like. (You might be able to publish an initial fork, but there's no way you'd be able to keep up with Mozilla; eventually they'd be adding critical features faster than you, and your fork would fall into the dustbin of history.)

And then there's the fact that your hardware will wear out or go obsolete, and your software will need to be ported to new hardware.

In short, your software lives in a continuously changing ecosystem of data. And you are inescapably dependent on other people to evolve that software in response to environmental changes in that ecosystem.

Therefore, while you may still "own" your software in some formal sense, that ownership means little in any practical sense. What matters is the organization that maintains your software, and your ongoing relationship to them. You're not (merely) paying for access to a passive bundle of data; you're actually making a calculated bet on the future behavior of a group of people.

At this point I'll show my colors and claim that, in the long run, open source projects with open governance structures offer more credible maintenance promises than proprietary systems. But that's a subject for a whole other set of posts. In the meantime, I offer two final nuggets, in the way of provoking thought.

First, yes, this post is a stealth followup to my previous post on ChromeOS punditry.

Second, I offer the following excerpt from Neal Stephenson's Cryptonomicon, concerning autistic World War II cryptographer and Naval xylophonist Lawrence Waterhouse:

Waterhouse has been chewing his way through exotic Nip code systems at the rate of about one a week, but after he sees Mary Smith in the parlor of Mrs. McTeague's boarding house, his production rate drops to near zero. Arguably, it goes negative, for sometimes when he reads the morning newspaper, its plaintext scrambles into gibberish before his eyes, and he is unable to extract any useful information.

Despite his and Turing's agreements about whether the human brain is a Turing machine, he has to admit that Turing wouldn't have too much trouble writing a set of instructions to simulate the brain functions of Lawrence Pritchard Waterhouse.

Waterhouse seeks happiness. He achieves it by breaking Nip code systems and playing the pipe organ. But since pipe organs are in short supply, his happiness level ends up being totally dependent on breaking codes.

He cannot break codes (hence, cannot be happy) unless his mind is clear . . . Clarity of mind (C_m) is affected by any number of factors, but by far the most important is horniness, which might be designated by σ, for obvious anatomical reasons that Waterhouse finds amusing at this stage of his emotional development.

Horniness begins at zero at time t = t₀ (immediately following ejaculation) and increases from there as a linear function of time:

σ α (t - t₀)

The only way to drop it back to zero is to arrange another ejaculation.

. . .

Now, when he was at Pearl Harbor, he discovered something that, in retrospect, should have been profoundly disquieting. Namely, that ejaculations obtained in a whorehouse (i.e., provided by the ministrations of an actual human female) seemed to drop σ below the level that Waterhouse could achieve through executing a Manual Override. In other words, the post-ejaculatory horniness level was not always equal to zero, as the naive theory propounded above assumes, but to some other quantity dependent on whether the ejaculation was induced by Self or Other: σ = σ_self after masturbation but σ = σ_other upon leaving a whorehouse, where σ_self > σ_other, an inequality to which Waterhouse's notable successes in breaking certain Nip naval codes at Station Hypo were directly attributable, in that the many convenient whorehouses nearby made it possible for him to go somewhat longer between ejaculations.

. . .

If he had thought about this, it would have bothered him, because σ_self > σ_other has troubling implications . . . If it weren't for this inequality, then Waterhouse could function as a totally self-contained and independent unit. But σ_self > σ_other implies that he is, in the long run, dependent on other human beings for his mental clarity, and, therefore, his happiness. What a pain in the ass!

I leave the connective tissue between the above passage and the rest of this post as an exercise for the reader.

On "tricks" and science

2009-11-24T04:50:00.000-08:00

Are people really claiming that the word "trick", when used by climate scientists to describe a data analysis technique via a mailing list for other climate scientists, indicates some nefarious conspiracy of deception? Why yes, they are, and Acephalous has the rebuttal:

Global warming skeptics are attacking climate scientist Phil Jones for encouraging trickery in an email recently stolen off the webmail server at the University of East Anglia in which he wrote:

I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline.

Over at RealClimate, the skeptical response to the word "trick" is to treat it as a colloquial:

Trick:
“a cunning or deceitful action or device; “he played a trick on me”; “he pulled a fast one and got away with it”
“Something designed to fool or swindle; ”
“flim-flam: deceive somebody; “We tricked the teacher into thinking that class would be cancelled next week”

. . .

Schmidt obliges:

. . . It's mostly used in mathematics, for instance in decomposing partial fractions, or deciding whether a number is divisible by 9 etc.etc.etc.

The skeptics rejoinder:

This is nonsense. Both are examples of teaching or explaining concepts to lay people. The first intentionally places “tricks” in quotations marks to emphasize its non-technical use.

The problem with nonspecialists reading the private correspondence of experts is that their ignorance transforms all the technical points into nefarious inkblots. To continue with the example above, skeptical nonspecialists encounter the word "trick" and ask for clarification. Schmidt provides evidence that the word is innocuous, but because nonspecialists can interpret neither the context of the original nor that of the further examples, they redouble their efforts: now the rhetorical situation in which the word "trick" is uttered matters; now the appearance of quotation marks matters, etc. They are convincing themselves that those black blobs represent what they insist they represent, and when experts inform them that those are not Rorschach blots to be subjectively interpreted—that they are, in fact, statements written in a language that skeptics simply do not understand—the nonspecialists look over them again and declare that it could be a butterfly, or maybe a bat.

For the programmers out there, this is a little like finding a comment in some random piece of program source code

// Hack: just disassemble the whole tree for now

and concluding that the author of the software is attempting to "hack" your password.

The word "trick", like the word "hack", is a term of art with an esoteric meaning different from its lay meaning. And no, "trick" does not denote either a deception or a way of "teaching concepts to lay people". See, for example, the tricks on Terence Tao's blog. Can anybody seriously believe that this amplification trick is "a way of explaining math to laypeople"? Well, I guess they can, if they're shooting their mouths off without having the least fucking clue what they're talking about.

So, OK, what does trick mean? Well, C. Shalizi's review of Tao's book gives a reasonable definition:

Tao’s third theme is tricks: patterns of establishing results that replicate across many situations, but in which any one result is too small to be a theorem in its own right, while the general pattern is too vague. These are an important part of how math actually gets done, but by their nature they tend not to have a recognized place in the curriculum, getting passed down by oral tradition, or by being absorbed by those who are lucky enough not only to run across a paper using the trick, but also to guess that it will generalize. There are numerous tricks throughout the book, and one of the nicest chapters, 1.9, expounds a family of tricks for improving inequalities, which Tao calls amplification.

Although, to be more precise, the climate scientists in question do not seem to be using the word "trick" in its narrowest mathematical sense, but rather in the more general sense of a useful technique — however, again, usually one too small to be publishable on its own [0] [1].

Incidentally, this sort of irresponsible misreading shows how little most climate change "skeptics" on the Internet know about math or science. Why the heck would I lend credence to people who don't know jack about scientific practice, and cannot be bothered to learn, and still feel comfortable dismissing thousands of working scientists as frauds? And especially why would I believe them over scientific professionals who have dedicated their lives to studying the data?

I mean, yesterday I got a comment on my previous post on climate change denialism saying:

Models that can exhibit large errors tend to exhibit them in the direction that the modellers would prefer. That's why real sciences use double-blind testing.

Anyone who doesn't know what results climate "scientists" are looking for isn't paying attention.

"Real" sciences use "double-blind testing"? O RLY? No wai! Blind testing, of either the single- or double- variety, is irrelevant to most experiments in the natural sciences. I would like to know what "double-blind" even means for a microarray assay; are we not informing the microscopic dots of DNA whether we've hybridized them or not? Are we getting them to sign little microscopic release forms and giving them little microscopic placebo pills?

If most researchers in the natural sciences even had to single-blind every experiment they do, they'd never get anything done. Natural science labwork is often incredibly laborious and grad students do not have time to, like, close their eyes and spin a little roulette wheel of test tubes every time they pipette a drop of reagent. The usual way to weed out observer bias is by (1) designing experimental methods which are relatively robust to observer influence; (2) repeating your experiments; and (3) describing a procedure in sufficient detail for other sufficiently trained people to reproduce the result. Blind experimentation is reserved for certain types of experiments where observer or subject bias is especially dangerous or probable.

As for the email the commenter links to, I find it hard to see anything suspicious about it. Here's an excerpt:

The Soon & Baliunas paper couldn't have cleared a 'legitimate' peer review process anywhere. That leaves only one possibility--that the peer-review process at Climate Research has been hijacked by a few skeptics on the editorial board.

. . .

[In quoted reply:] I looked briefly at the paper last night and it is appalling - worst word I can think of today without the mood pepper appearing on the email ! . . . The phrasing of the questions at the start of the paper determine the answer they get. They have no idea what multiproxy averaging does. By their logic, I could argue 1998 wasn't the warmest year globally, because it wasn't the warmest everywhere. With their LIA being 1300-1900 and their MWP 800-1300, there appears (at my quick first reading) no discussion of synchroneity of the cool/warm periods. Even with the instrumental record, the early and late 20th century warming periods are only significant locally at between 10-20% of grid boxes.

I'll freely admit that I don't know what the jargon means, but "they have no idea what X does" is not a phrasing I'd ever have wanted to read in a review of a paper of mine. This is not prima facie evidence of anything but that some journal published a bad paper.

And subversion of publication venues by cranks is actually an everpresent danger in the sciences. Do climate change denialists really believe that concern over a journal's publishing trash is evidence of a groupthink conspiracy? No doubt they'll be taking up the heroic cause of M. S. El Naschie next.

[0] The notion of "trick" seems loosely related to what the C.S. community calls a "pearl", except that pearls are perhaps more rigidly described, and computer scientists publish them in some venues regardless (e.g. ICFP). Truthfully the "unpublishability" of tricks and other semi-formal knowledge seems like a flaw in scientific publishing, albeit one that blogs and other Internet-based venues may be correcting.

[1] It's not even unheard-of for laypeople to use the word "trick" in roughly this sense. See: Clemenza's recipe from The Godfather. I look forward to climate skeptics' exegesis of the diabolical agenda behind Clemenza's spaghetti sauce.

Pro tip for ChromeOS punditry

2009-11-22T15:09:00.000-08:00

Full disclosure: I work for Google, although this blog reflects my personal opinions only.

Please don't pontificate about ChromeOS until you grok the long-term implications of Web Storage, Native Client, Open3D/WebGL, Courgette, and a large local disk cache. Oh, and, of course, Moore's Law.

And if you can't work out the implications, at least talk to someone who can, before you hit the "Post" button. You might still think that ChromeOS is a bad idea, but at least you'll be critical in a more clueful way.

Software patents have tangible costs for innovation, and for you

2009-11-03T07:32:00.000-08:00

I have a friend who's been working extremely hard on a small software startup for the past few years. He and his partner developed a genuinely innovative, original technology which solves a useful problem for end-users and probably has significant commercial value. The technology has been integrated into a website that is awesomely functional and and even fun to use. (I'd point you there, except that I'm going to discuss legal matters shortly and I think it's better not to identify the parties by name.)

His startup recently got sued for patent infringement by a company that independently developed a product that performs a vaguely similar function. This other company's product is much less sophisticated, and their user-facing site is an ugly, user-hostile pile of crap. The term "search arbitrage" would be a kind word to apply to this other company's product. And there is absolutely no sense in which my friend's work builds on any of this other company's technology.

Now, my friend and his partner have consulted multiple IP lawyers and they've said, "Yep, the law is probably on your side." They have also said, "You're still screwed." The trial would take forever, the legal fees would be ruinous, and in the meantime nobody will invest in a company which has a litigation cloud hanging over it.

So, this sucks for my friend and his partner. More importantly, this sucks for you, because, having seen the product, I am 100% convinced that you, or someone you know, would love to have this technology acquired and integrated into a major site that you use.

This sort of story is not at all uncommon in the software industry. I've been meaning to tell it for a couple of weeks now, because it made me think of this post by Tim Lee about "libertarian political philosopher" Richard Epstein's bold claim in an amicus brief* that:

The credible threat of a published patent’s right to exclude acts like a beacon in the dark, drawing to itself all those interested in the patented subject matter. This beacon effect motivates those diverse actors to interact with one another and with the patentee, starting conversations among the relevant parties.

In response, Tim writes:

There’s nothing beacon-like about software patents. Software companies do not use patents as a mechanism for finding technologies or business partners. Patents tend to be written in unintelligible legalese, they’re not well indexed, and they issue years after they’re filed. They’re completely irrelevant to the day-to-day process of product development in the software industry. I’ve never met a software developer who regards the patent database as a useful source of information about software inventions, nor can I think of an example of a software company (Intellectual Ventures doesn’t count) that uses patents as a central part of its product-development strategy.

Completely true, except that Tim does not go nearly far enough. At any software company with competent legal counsel, developers are instructed in the strongest possible terms never, ever to look at a patent, because the tiniest amount of documented influence could be used as ammunition in a lawsuit. The only time a sane software developer reads a patent is when your company's lawyers specifically ask you to help them prove you're not infringing on one. If you ever get wind that there's a patent even vaguely related to your work, you stick your fingers in your ears and run in the other direction. In short, software patents facilitate "conversation" about as well as poison gas bombs do.

One thing that I find extremely frustrating about many legal scholars' and economists' approach to patents is that they make two false assumptions. The first assumption is that transaction costs are acceptable, or can be made so with some modest reforms. The second assumption is that patent litigation is reasonably "precise"; i.e., if you don't infringe on something then you'll be able to build useful technology and bring it to market relatively unhindered. As my friend's story shows, both of these assumptions are laughably false. I mean, just black-is-white, up-is-down, slavery-is-freedom, we-have-always-been-at-war-with-Eastasia false.

The end result is that our patent system encourages "land grab" behavior which could practically serve as the dictionary definition of rent-seeking. The closest analogy is to a conquistador planting a flag on a random outcropping of rock at the tip of some peninsula, and then saying "I claim all this land for Spain", and then the entire Western hemisphere allegedly becomes the property of the Spanish crown. This is a theory of property that's light-years away from any Lockean notion of mixing your labor with the land or any Smithian notion of promoting economic efficiency. And yet it's the state of the law for software patents. Your business plan can literally be to build a half-assed implementation of some straightforward idea (or, in the case of Intellectual Ventures, don't build it at all), file a patent, and subsequently sue the pants off anybody who comes anywhere near the turf you've claimed. And if they do come near your turf, regardless of how much of their own sweat and blood they put into their independent invention, the legal system's going go off under them like a land mine.

It is hard to think of a more effective mechanism for discouraging innovation in software. I mean, I suppose you could plant a plastic explosive rigged to a random number generator under the seats of every software developer, and that would be slightly worse.

* To be fair, the amicus brief is not completely Epstein's work; it is the sworn work of one Dr. Ananda Chakrabarty and coauthored with lawyer F. Scott Kieff. I don't really know how these things work, but I assume that Epstein agrees with the argument laid out even if he's not the lone progenitor of it.

Gross Pointe Blank is a great movie

2009-10-26T23:55:00.000-07:00

So I'm watching this again, and I can't believe I ever thought it was merely OK. The skill with which it deftly weaves together irreverent humor, pathos, and even wisdom has rarely been equaled in turn-of-the-century film; maybe Eternal Sunshine of the Spotless Mind comes close, although the latter ultimately takes itself so seriously that it loses some of its dexterity.

A partial list of virtues of GPB:

The scene where Martin Blank (John Cusack) revisits his old home address to the tune of Guns 'n Roses' Live and Let Die never ceases to be hilarious.
The awkward tension between Martin and Debi (Minnie Driver) hits this perfect compound of emotional veracity and factual implausibility. No meet-cutes or other boilerplate infrastructure of romantic comedy here; we just get an international hit man coming home to chase down an old ex-girlfriend. Boom, there it is. Deal with it. And yet it feels more real than any number of superficially more "realist" entries in the genre.
Dan Aykroyd. Hit man. And somehow, against all odds, funny, for maybe the last time ever.
The degree of care evident in virtually every line of the script is astonishing. The dialogue is as dense and snappy as the best Coen brothers films. Even throwaway lines are frequently exceptional. (Waitress at diner: "...there's Gatsby's 'West Egg' Omelette..."; you might not even notice the line, yet observe how exactly this item, offered by the waitress at Martin's Midwestern (!) hometown diner, echoes the broader themes of the movie. Rather than ordering this, Martin opts for an omelette without filling.)
Two years before The Sopranos started turning the "emotionally disturbed criminal in a shrink's office" into a cliché, we get a criminal-in-a-shrink's-office setup that's funnier and more plausible than basically anything in The Sopranos' entire run, and also doesn't overstay its welcome.

I could go on and on, but basically, you should just go watch this movie again, with attention.

(However, I will admit that Debi's character, as written, veers close to being the mere projection of Martin's psychological needs, rather than a character in her own right. This is perhaps redeemed by Driver's performance.)

J. Holbo on hate crimes

2009-10-16T00:11:00.000-07:00

At CT, J. Holbo kicks off a discussion of the moral and legal justifications for hate crimes legislation, and how calling it "thought crime" is basically ridiculous. Worth reading because (a) Holbo is uncharacteristically terse and (b) the comments thread's decent enough that I have little to add.

One can parse out a distinction between hate crimes legislation and other uses of mens rea in criminal law but, as the thread illustrates, it's an exceedingly fine line, and a much subtler one than is commonly supposed by the rhetoric of hate crimes legislation opponents. It's pretty clear, once you think this through, that the debate over hate crimes legislation is more "haggling over price" than some bright shining line of moral principle.

Online dating and race at OKCupid

2009-10-13T08:20:00.000-07:00

This recent post on the OKCupid blog has been widely linked, but since I covered this previously, at some length (also related), I feel obligated to post it.

If you have any interest in this subject, you'll no doubt read the whole thing, but if you need any further suasion, here are the key figures. Reply rates by race when men send messages...

...and reply rates by race when women send messages...

Read it and weep. (Or rejoice, I suppose, if you're in the favored classes and you guiltlessly enjoy racial privilege.)

BTW yes, they controlled for algorithmic match rate, which is essentially flat across all race/gender combinations (see the "Match % by Race" figure in the original post).

As usual, I find the result itself sad — I think this goes far beyond a moderate amount of understandable homophily (look at the diagonals!) — but what I find much more sad is the degree of self-deception that people engage in when discussing these results. Unfairness is annoying; deception about unfairness really brings on the facepalm.

See, for example, the 459-comment Metafilter thread, where about half the educated, literate, liberal MeFi crowd doesn't seem to get the following simple proposition: although diversity of aesthetic preferences, including preference for racially marked features, may be a simple personal choice, systematic statistical skew in aesthetic preferences across a large population strongly indicates socialization to racially biased standards of attractiveness.

Note, by the way, that racial preferences don't mean merely visual discrimination. The degree of racial discrimination is considerably stronger and more widespread for women than men, even though (as folk wisdom has it) women are less visually focused than men. (Personally, I think folk wisdom overstates this sex difference, but I do think it's real.) I think this implies that part of the racial discrimination effect — possibly even the dominant part — is due to people making assumptions about personality or character based on race, rather than preference for a certain physical appearance alone. Which is even more damning.

Standard caveats w.r.t. all such social science analyses apply blah blah blah. On the other hand, the fact that this result essentially replicates, at finer granularity, the results of the Hitsch et al. study I blogged previously, as well as anecdotal evidence gathered from friends and acquaintances, does not incline me to skepticism.

Coase and Pareto optimality illustrated

2009-10-12T08:03:00.000-07:00

The Coase Theorem states that absent transaction costs, a Pareto efficient outcome will be arrived at regardless of the assignment of rights in an economic transaction. This theorem and its implications are a common source of confusion, which I attribute to the relatively dry examples typically used to illustrate the theorem --- commercial property easements, etc. Here is a more vivid illustration, which may clarify matters.

Suppose I want to shoot you in the face. With a gun. Suppose I value this experience at X dollars. Suppose, also, that you value your face at Y dollars. The Coase theorem predicts that both the outcome and the global welfare thereof will be the same regardless of whether

I have the legal right to shoot you in the face.
You have the legal right not to be shot in the face.

You can prove this by breaking it down into cases:

Suppose Y > X; that is, you value your face more than I value shooting you in the face. Then consider the two possible assignments of rights:
1. Suppose I have the right to shoot you in the face. Then you will pay me $X plus a penny not to shoot you in the face. Your welfare will be $Y - ($X plus a penny), and my welfare will be $X plus a penny. The total welfare will then be $Y - ($X plus a penny) + ($X plus a penny), or $Y.
2. Suppose you have the right not to be shot in the face. Because Y > X, I will not pay you enough to let me shoot you in the face. You will have your face, which is worth $Y. I will have nothing. The total welfare between the two of us is then $Y.
Notice that in both cases, you do not get shot in the face, and the total welfare is $Y.
Suppose, on the other hand, that Y < X; that is, I want to shoot you in the face more than you value not being shot. Now consider the two subcases:
1. Suppose I have the right to shoot you in the face. You will not be willing to pay me $X, because you only value your face at $Y. I will shoot you in the face. Your welfare will be -Y dollars. My welfare will be X dollars. In this case, the total welfare is $X - $Y.
2. Suppose you have the right not to be shot in the face. Because Y < X, I will pay you $Y plus a penny to let me shoot you in the face. Your net welfare will be a penny, because you got shot in the face, but got paid $Y plus a penny. My welfare will be X minus Y dollars plus a penny. The total welfare between the two of us is $X - $Y, plus a penny minus a penny, which is $X - $Y.
Notice that in both cases, you get shot in the face, and the total welfare is $X - $Y.

So, the outcomes are the same, and the total welfare is the same. Clearly, it does not matter whom we assign the rights to, right?

Right. Back here on Earth, any thinking person will be prompted to make a few observations.

First, although total welfare is the same, the balance of welfare may be dramatically different. Consider the difference between A.1 and A.2: if I have the right to shoot you in the face, you have to pay me not to; but if I do not, then you have to do nothing and you get to keep all your money. So the Coase Theorem says nothing whatsoever about distributional justice.

Second, you may observe that I have made an unwarranted assumption that the person being paid off in A.1 and B.2 will accept a mere penny extra to change his/her behavior. In fact, the person being paid off might observe that the welfare surplus (the difference between $X and $Y) can be larger than a penny, and bargain for a larger fraction of that surplus. Logically, the "buyer" in each case ought to be willing to pay the maximum price minus a penny. For example, if I value shooting you in the face at $X, I ought to be willing to pay $X minus a penny for the privilege of shooting you; in fact, every payment between $X minus a penny and $Y plus a penny is Pareto optimal and the Coase Theorem has nothing to say about how much money will actually get transferred. In practice, of course, this would be determined by social norms, unequal distribution of prior capital, and other "social" stuff that economists usually prefer to ignore the existence of.

Third, we have made no mention of "human" or "natural" rights. And indeed the Coase Theorem is a completely amoral observation about (an idealized model) of bargaining. I hope you believe it would be monstrous for society to give me the right to shoot you in the face, with your only recourse being to pay me off not to do it. The Coase Theorem tells us that both assignments are "efficient" (Pareto optimal). But of course there's a world of difference between the two.

So basically, the Coase Theorem is a cool little mathematical widget that, by itself, offers almost no guidance in real world policy questions. In combination with other concepts, it's no doubt a useful widget for economists. But when it appears unadorned in laypersons' rhetoric, its most common purpose, as far as I can tell, is to obscure fundamentally normative disagreements about distributional justice, social inequality, and moral principle. Mathematical concepts like Pareto optimality are objective --- they assume no particular moral values or social context --- but neither does a gun, and yet when somebody points a gun at your face there's no mistaking the malign intention.

Why are so many IT workers climate change denialists?

2009-10-08T07:12:00.000-07:00

Maybe it's just my imagination, but it seems that climate change denialism is even more common among programmer and sysadmin types than among engineers and applied scientists more generally. Without diving into climate science*, here are a few brief hypotheses.

Computers teach you to think in logic. Climate change modeling relies on the synthesis of a large number of statistical correlations rather than crisp rules of inference. Although logic and statistics are related (probability strictly generalizes logic, in a precise mathematical sense), a mind too narrowly conditioned to thinking in syllogisms may find it hard to reason statistically.
Computing workers spend a lot of time on the Internet, and are disproportionately likely to be white, male, and libertarian. Climate change denialism is seen as not only respectable but intellectually heroic by (a significant faction of) the tribe of white, male, libertarian Internet users. (Arguably this merely begs the question, however, as the direction of causality may go the other way.)
Computing workers are more socially isolated than people belonging to the same socioeconomic class. Because they spend less time around an intelligent, well-educated peer group, they are less socialized to defer to the knowledge of others.
Computing workers spend all their time around intricate machines which (a) they understand better than the general public, and (b) the general public has become heavily reliant upon. This breeds arrogance, and arrogance breeds disrespect for expertise in general. Disrespect for climatologists is simply a special case of this phenomenon. This, however, is true of almost any profession involving specialized knowledge, from plumbing to physical therapy to nursing; so this factor might not prove decisive, except for the next bullet...
Computing workers are, on average, more "autistic" and less "empathetic" on the autism/empathy spectrum. That is, they are unusually incompetent at modeling the mental and emotional states of other people. As a result, they fail to place themselves in the shoes of professional climatologists. That is, they do not imagine that most professional climatologists have worked hard to become experts in an esoteric and demanding (which is to say nerdy) intellectual discipline; might be driven by passion and curiosity and a desire to get it right; might along the way have been exposed to vast volumes of knowledge with which the lay observer is not familiar. In short, it is much easier to view literally thousands of scientists worldwide as a species of fools, liars, and conspirators when one assumes that they are nothing like oneself. (I strongly suspect that fewer IT workers would be climate change denialists if they realized that climate scientists are natural science geeks like them, whereas the primary beneficiaries of climate change denialism are corporate suits who were probably shoving geeks into lockers in high school.)

*About which, er, you can say whatever you like, but I'm going to listen to this guy.

Cynicism about government does not help the libertarian cause

2009-10-07T07:24:00.000-07:00

TLF has been much less interesting since Tim Lee left, and this piece by J. Harper is the sort of thing that is reducing my desire to keep the TLF feed in my Reader. I'm not going to address the ostensible main thesis of the piece, but I just want to comment on one aspect that strikes me as dramatically misguided:

I’m a person who notices premises, and Lessig sets up an interesting premise indeed: What he calls the “naked transparency movement”—unvarnished access to government data—”is not going to inspire change. It will simply push any faith in our political system off the cliff.”

Yes, Lessig has “change” and “pushing faith in our political system off the cliff” in opposition. So, the only thing that qualifies as “change” is improving faith in our political system? This pegged my bs detector.

First, there's an elementary error of logic here. The proposition

"change" and "pushing faith in our political system off the cliff" are in opposition

does not in any way entail that

"improving faith in the political system" is the "only thing that qualifies as change"

Political change and faith in the political system are variables in a very high-dimensional space, and the observation that they converge at the origin does not entail that every single increase in one is correlated with an increase in the other. Or in less math-metaphorical terms, even if you grant Lessig's premise, there might be a large number of changes which slightly reduce faith in our political system, or hold faith in our political system constant, and this would still not contradict Lessig's statement. (N.b. I say this as someone who thinks that "faith" in our political system is about as misguided as any other kind of faith.)

But never mind that.

The point I really want to make is that Harper seems to think that cynicism about the political system is in some way helpful to the libertarian cause. I'm not exactly a libertarian but this strikes me as completely wrong. Cynicism breeds two things: (1) apathy and (2) shameless exploitation. A population that is truly cynical about politics leads to even greater disengagement plus even greater corruption than prevails in the ordinary course of government.

Libertarians are not cynical about politics. A true cynic would adopt the essentially nihilistic stance of today's Republican Party leadership, whose political strategy is basically

Use the government to enrich one's political allies.
Sabotage any attempt to use government as an instrument to enhance the general welfare.
Distract people from the former two points by a systematic campaign of deceitful propaganda relating to irrelevancies.

On the contrary, principled libertarians are in fact extremely idealistic about the possibility of government being reformed in ways that either enhance the general welfare (in the case of utilitarian libertarians) or are more respectful of the libertarian conception of natural rights (in the case of natural-rights libertarians).*

In short, true cynics either become players in the game or stop playing; libertarians hate the game. And hating the game requires a certain belief in the possibility of a better game. Basically, Harper seems to be deeply confused about the distinction between procedural and substantive liberalism.

Oh, all right, I'll say a little bit about the primary thesis of the piece. I don't read TNR and I don't want to start, but my view on transparency is more or less in line with Aaron Swartz's: exposure is necessary but it's not going to change things without the development of social practices and norms which increase civic engagement.

*Of course I disagree that actually existing libertarianism does either of these things but that doesn't have anything to do with how libertarians justify their own beliefs.

T. B. Lee on journalism and the Innovator's Dilemma

2009-08-27T07:34:00.000-07:00

T. B. Lee (not T. Berners-Lee) has a newish blog and it has proved excellent enough* that it exceeded my expectations (which were not low to begin with; & I'm not saying this because we're slightly acquainted). For example, this post on newspapers and journalism makes a point which I've never seen stated as clearly anywhere else (well, OK, just once, but it bears repeating):

If I ran the world, no one would be allowed to opine about the decline of the newspaper industry until they’d read The Innovator’s Dilemma. The web is so clearly a disruptive technology, and the newspaper industry is so clearly following the trajectory Christensen describes in his book, that it’s hard to think clearly about the process if you haven’t grasped Christensen’s key insights. To review, the key attribute of a disruptive technology is that when it’s introduced into the marketplace, it is cheaper but also markedly inferior to the incumbent technology, as judged by the criteria of the dominant technology’s customers. Internet-based news clearly fits this pattern. As newspaper people never tire of reminding us, Internet-based news outlets rarely have the resources to staff expensive foreign bureaus, conduct in-depth fact-checking, fly sports reporters to away games, hire teams of lawyers, and so on. If the Huffington Post or TechCrunch were judged as newspapers, they would be pretty lousy ones.

What we learn from The Innovator’s Dilemma is this state of affairs is completely normal when a disruptive technology invades an established industry. . . . So newspaper partisans are absolutely right to point out that newspapers continue to be superior to online news sources in a number of respects. But they’re completely wrong to think this can save them.

More along those lines in the full post. Plus a neat coda on how C. M. Christenson was unfortunately kind of a sellout (or at least naïvely optimistic), which probably isn't said often enough.

*N.b. this does not mean I agree with Tim all of the time, or even most of the time. But the relevant mark of quality here is that even when I disagree with the argument, I usually find it both (a) thought-provoking and (b) neither stupid nor hackish.

Brill on New York City public schools

2009-08-26T22:34:00.000-07:00

In this week's New Yorker; read it all and weep. Now, I'm pretty damn liberal, and sympathetic to unions in general — I once marched in a picket line as a member of UAW Local 4121 — but this piece had me frothing for the blood of New York City teacher's unions. As of this moment, I am a supporter of rapid expansion of charter schools everywhere; the sooner the better.

The most maddening thing about the whole conversation is not even the blatant terribleness of the teachers profiled, or the waste of taxpayer money, or the harm to students, but the disingenuousness of the union representatives*:

[Former United Federation of Teachers president] Rudy Weingarten . . . always tries to link the welfare of teachers to the welfare of those they teach — as in "what's good for teachers is what's good for children."

[New York City Department of Education deputy chancellor, Chris] Cerf's response is that "this is not about teachers; it is about children." He says, "We all agree with the idea that it is better that ten guilty men go free than that one innocent person be imprisoned. But by laying that on to a process of disciplining teachers you put the risk of the kids versus putting it on an occasional innocent teacher losing a job. For the union, it's better to protect one thousand teachers than to wrongly accuse one."
. . .

Should a thousand bad teachers stay put so that one innocent teacher is protected? "That's not a question we should be answering in education," Weingarten said to me. "Teachers who are treated fairly are better teachers. You can't have a situation that is fear-based. . . . That is why we press for due process."

Notice how Weingarten completely dodges the question of the cost/benefit tradeoff. Apparently accountability for teachers is synonymous with "fear", and that's unacceptable, full stop.

Sorry, but around the time I became an adult, I realized that, although being held responsible for my actions can produce a variety of emotions, including occasionally fear, this was simply the nature of holding responsibility.

And although it may be better for ten or even a thousand guilty people to go free than for one innocent person be imprisoned, there is surely some number of guilty people going free at which the balance tips. No reasonable justice system can be infallibly free of false convictions; therefore, by allowing a justice system to exist at all, we implicitly acknowledge that some number of wrongful convictions is a price worth paying for protecting society. And that's when the price of a false conviction is putting people in jail. When the cost is merely forcing people to find another job, the balance certainly tilts towards removing more guilty people and occasionally harming an innocent teacher.

It would be one thing for Weingarten to make an argument about where the balance lies, as an empirical matter, in the case of teachers. But she doesn't do that. She simply denies the premise of the question.

And, of course, the willful denial of transparently obvious logic is a huge red flag in any argument. People deny logic when they fear its conclusions, which is to say that they fear truth itself, which is to say that they are both holding an indefensible position, and also aware, on some level, that they're holding an indefensible position. The only remaining question in such cases is whether, in addition to misleading their audience, they're deceiving themselves as well.

Incidentally, this all leads me to wonder if intransigence will ultimately prove counterproductive for teachers' unions in the long run. Treating one's audience the way Weingarten does is insulting to their intelligence; and insulting people is not, as a rule, a good strategy for gaining their support.** I mean, I'm now sufficiently incensed that I'd donate a decent chunk of change annually to any organization that could credibly promise to accomplish nothing at all besides undermining the political power of teachers' unions.

Finally, I should add that the existence of bad teachers is no secret among actual classroom teachers. I was at dinner a couple of weeks ago with a couple of teachers in the Oakland, CA school system and they know who the goofballs are. Talk to a good public school teacher for about twenty minutes about the other teachers at their school, and see if you don't see them roll their eyes about someone or other. Maybe they don't believe that all borderline teachers should be fired, but among good teachers (particularly younger ones) I suspect that you'd find a fair amount of support for booting bad teachers with a much less ridiculously onerous process than currently prevails in New York City.

p.s. If you harbor some suspicion that Brill's article is unfairly slanted due to his haute-bourgeois disdain for the unionized classes, see this Village Voice article which examines a small slice of the same issue. The Voice ain't what it used to be, but anti-liberal it is not. And the union doesn't really come out smelling like roses there either.

*Yes, all of these things are more objectively harmful than the union representatives' disingenuousness; I'm just saying that the latter just pushes my personal buttons more.

**This observation may seem ironic because I insult people on this blog all the time. However, I'm mostly indifferent to the support of those whom I insult.

Shorter Betsy McCaughey

2009-08-22T08:32:00.000-07:00

The government should not enforce contracts.

URL redirectors: the other shoe drops

2009-08-10T10:29:00.000-07:00

Well, it was only a matter of time. (Previously.)

False advertising

2009-08-08T21:51:00.001-07:00

SFO breakfast receipts (en route to EWR)

2009-08-07T10:39:00.001-07:00

Spot the funny.

How destructive was Ozymandias's bomb in Watchmen (the movie)?

2009-08-07T06:55:00.000-07:00

Attention conservation notice: nitpicky analysis of a detail that you probably don't care about, from some comic book movie.

Finally saw Watchmen on video (missed it in theaters). I noticed something curious about the final explosion. It is clearly centered on Times Square, and produces a perfectly spherical blast; but in a later shot, the Empire State Building is not only standing but largely intact.

From this, we can infer that the explosion's radius of destruction was less than a dozen or so north/south blocks.

View Larger Map

This got me thinking: how big is this explosion, compared to an actual nuclear bomb blast?

According to HYDESim, a 25-kiloton nuclear weapon detonated at Times Square would exert just enough overpressure at the Empire State Building site to demolish a concrete skyscraper. For comparison, according to the the carloslabs.com Ground Zero simulator, the "Little Boy" nuclear weapon detonated over Hiroshima sixty-four years ago was 15 kilotons; if detonated over Times Square, it would have blown the windows out of the Empire State Building, but the structure would probably not be knocked down.

So Ozymandias's bombs appear to create a pressure blast about as powerful as that from the atomic bombs dropped on Japan in World War II.

The nuclear weapons possessed by the U.S. and U.S.S.R. in the 1980's were, of course, vastly more destructive, on the order of hundreds of kilotons; and would have been delivered as batches of multiple simultaneous warheads.

Wikimedia Commons image by U.S. Army; source

Now, regarding Ozymandias's bombs, it's possible that radiation killed many people outside the direct blast radius. But there's no indication in the movie that Dr. Manhattan's electromagnetic emissions are harmful to human life. In my opinion, if one takes the movie strictly on its own terms, Ozymandias has been careful to engineer a relatively small mass-destruction event: only on the scale of a small fission bomb.

Of course, step outside of the movie's fiction, and it seems equally likely that director Zach Snyder simply assumed that most viewers would not be too familiar with the geography of New York. So he had Ozymandias set off the bomb in Times Square, because it's instantly recognizable, and then he left the Empire State Building standing so that viewers would recognize it instantly in the aftermath shot.

ibuffer: If you do not use it, you are insane (Wednesday emacs blogging)

2009-08-05T07:12:00.000-07:00

In short, use ibuffer, now; the relevant .emacs magic:

(global-set-key (kbd "C-x C-b") 'ibuffer)
(autoload 'ibuffer "ibuffer" "List buffers." t)

This is so much better than the regular buffer list it's not even funny. The first thing you'll notice is font-lock colorization (welcome to the 21st century!); but the killer feature is the wealth of buffer management keyboard shortcuts. On your first trip to ibuffer, you'll want to spend a little time reading through C-h b to learn the keyboard bindings; a small sample of just the marking functions:

% f     ibuffer-mark-by-file-name-regexp
% m     ibuffer-mark-by-mode-regexp
% n     ibuffer-mark-by-name-regexp

* *     ibuffer-unmark-all
* /     ibuffer-mark-dired-buffers
* M     ibuffer-mark-by-mode
* e     ibuffer-mark-dissociated-buffers
* h     ibuffer-mark-help-buffers
* m     ibuffer-mark-modified-buffers
* r     ibuffer-mark-read-only-buffers
* s     ibuffer-mark-special-buffers
* u     ibuffer-mark-unsaved-buffers
* z     ibuffer-mark-compressed-file-buffers

Once you start chaining these together, you'll wonder how you ever got along without them. For example, I commonly do * s * r t D y: mark all "special buffers" (*shell*, *scratch*, etc.); mark all read-only buffers; toggle marks (marking ordinary read-write buffers); delete marked buffers; confirm. This is handy since I often work on projects via multiple emacs instances, switching between an X11 emacs instance and a tty instance running under screen. When I switch, I want to close all the files I have open for editing (even if there are no unsaved changes), but leave *shell* and dired buffers alone.

Buffer management has historically been a real bottleneck in emacs productivity. Unlike most IDEs, emacs makes it trivial to have dozens or hundreds of buffers open simultaneously. This works great, except that working with all these buffers can become troublesome. For example, to switch to a buffer, you typically C-x b and type-complete the name; but when you have a half-dozen dired buffers all named client, it gets hard to remember whether the one you wanted was client<2> or client<5>. And, of course, the C-[left click] buffer list gets ridiculous with many buffers — you might as well be using Eclipse or something. ibuffer doesn't completely solve all these problems, but it certainly mitigates them.

The puzzle of American yogurt...

2009-08-04T21:34:00.000-07:00

...is this: Every national cuisine has a vernacular yogurt which is superior to American yogurt.

Think about it. Greek*, Indian, Swiss, Korean (n.b. many Koreans are lactose intolerant!), the list goes on: all better than American yogurt. And I'm not talking about handmade "artisanal" yogurt; this is all mass-produced stuff.

Now, of course, in our globalized world, one can obtain foreign yogurts for pretty reasonable prices, so it's not like I'm suffering here. What I want to know is: who keeps Dannon in business? Haven't they noticed that American yogurt has the unappetizing consistency of snot, and is also excessively sour, which the manufacturer typically tries to cover up (clumsily, unsuccessfully) with excessive sugary flavoring? Or do people just not know any better?

Or maybe it's just the advertising.

*My poison of choice: Fage 2% + fresh pineapple chunks. This may be the only vaguely healthy foodstuff which actually makes me feel satiated when I eat it after exercise.

B. DeLong: Arnold Schwarzenegger was right about one thing

2009-07-30T21:55:00.000-07:00

DeLong writes:

Changing sedentary, high-cholesterol, high blood pressure, high blood sugar fat people into more active, low-cholesteral, normal blood pressure, normal blood sugar fat people certainly does improve their health.

The governor of California is incompetent at budgeting, but these words of his are well worth listening to:

This exercise is extremely effective for your lats[issimi dorsi] and your upper back. Stand with your feet on either side of an open door and grasp the doorknobs with both hands. Slowly sink away from the door so that your back jackknifes and your arms extend fully and lock. Now pull yourself back up to the starting position. Let your arms, not your legs, complete the motion. I will count out thirty repetitions. Beginners should do 10, intermediates 20, and advanced the full amount. LET'S DO IT! 1... 2... 3... 4, AND STRETCH YOUR BACK!... 5... 6... 7, DON'T USE YOUR LEGS!... 8... 9... 10... 11... 12... 13... 14... 15... 16, JUST USE YOUR ARMS!... 17... 18... 19... 20... 1... 2... 3, CONCENTRATE ON YOUR BACK!!... 4... 5... 6... 7, THREE MORE!... 8... 9, AND NOT LAST ONE!... 30... WE'RE DOING FIVE MORE!!... 31, 32... HA! HA!... 33, 34, 35. Next we have in our program a wonderful leg exercise, the lunges. This exercise develops the front part of your thighs...

Hard to argue with.

p.s. Maybe this goes without saying, but M. McArdle is a completely intellectually irresponsible dingbat, and if DeLong doesn't convince you, T. Levenson and E. Klein are here to perform intellectual garbage pickup.

A business model for reporting

2009-07-27T06:27:00.000-07:00

I would pay $50 a year for Seymour Hersh's reporting alone.* So, suppose I did that. Then suppose that there are twenty thousand people in America like me; this does not seem farfetched. That's a million-dollar budget just to fund one man's reporting. Aside from paying himself, he could easily fund travel, online distribution, and a handful of apprentices and assistants out of that budget. So I would not be funding Seymour Hersh, the individual; I would be funding the Seymour Hersh news team.

There are a handful of other reporters — off the top of my head, Elizabeth Kolbert, James Fallows, and Matt Taibbi come to mind — for whom I'd pay a similar amount. In fact, I can probably afford to spend a thousand dollars a year to patronize twenty journalists whom I actually respect. This is considerably less than I donated to miscellaneous humanitarian organizations last year. I say this not to brag about my personal charity, which is actually below median for my income, but to remark that once normal people get jobs in the professional class, they start donating this rough magnitude of money to charity.

So, suppose that there are one hundred reporters in America like Hersh. That is a one hundred million dollar domestic industry dedicated to pure reporting.

Notice that this industry would be mostly unconstrained by the need to seek eyeballs. I do not want to give Hersh money because I think his articles will get more page views than the latest article on trends in reality television (the center article on Sunday's NYTimes front page). I want to give him money because he discovers things that I want discovered.

This seems like a much more rational business model for news reporting in the public interest than the current one, where news is funded by supermarket coupons and advertisements for department stores and used cars.

Note that you can generalize the model. I picked 20,000 people donating $50 a year because that's roughly what I'd pay. Clearly, you can stretch the model in either the direction of higher prices or more people. At one extreme, there's the individual-patronage model: Bill Gates can just pay a million dollars a year to his pet journalist. At the other extreme you could fantasize about charging twenty million people a nickel a year. (This seems dubious to me. Due to transaction costs and coordination issues, I think it's much easier to wrangle larger sums of money out of smaller numbers of people. On the other hand, ideally you'd want a large enough cadre of donors so that no individual has too much pull with the reporter.) If you look at the whole spectrum, it's hard to believe that there's no working point on that spectrum. Maybe it's 4,000 people donating $250 a year. (That's about as much as a family membership to the Monterey Bay Aquarium, or a medium-sized pledge for the local NPR affiliate.) Maybe it's something else. The point is that there are three hundred million people in this country and for a nationally recognized journalist it's not hard to imagine that there's enough patronage out there to fund his or her work.

Now, no doubt there are many practical challenges one would have to overcome in order to reorganize the news industry as a whole along these lines. And the details definitely wouldn't work out exactly as I've painted it**. But revolution's never a sure thing. I, for one, would love to see someone try it.

*On the other hand, I would not pay any amount of money to support Thomas Friedman*** or David Brooks. It is a curious artifact of the current newspaper industry that I cannot give money to writers whom I respect without also subsidizing hacks. The distribution of my news dollar to people who write stuff is determined by the whims of people like Bill Keller and Marcus Brauchli. These guys do not, as far as I can tell, try to produce a product which describes reality; they aim to achieve "balance" as defined by the political sensibilities of their social networks. In fact, it is not clear to me that they even understand the difference between these two things.

**Random guesses: (1) journalists like to be around other journalists, so individual teams would rent offices or share coworking space together; (2) the economics of risk would lead to agreements to share revenue and other resources among teams. The end result might be something like a current newsroom but with more distributed authority and a different revenue stream.

***Actually, I would pay negative money for Thomas Friedman. That is, you would have to pay me to read Friedman regularly, and I would probably pay you money if you could credibly offer to stop Friedman from writing.

An objectively incorrect Buddhist belief

2009-07-26T12:12:00.000-07:00

Pratītyasamutpāda states that

A human being's existence in any given moment is dependent on the condition of everything else in the world at that moment, but in an equally significant way, the condition of everything in the world in that moment depends conversely on the character and condition of that human being.

For "world" substitute "universe". But, of course, we know that this is false. One's condition can be affected by, and can conversely affect, only objects within one's light cone, not everything else in the universe at that moment. More generally, two objects only affect each other insofar as their light cones intersect.

Image by Wikipedia user Deibid; source.

Given everything else that's arbitrary about Buddhism, and religion more generally, it may seem sort of random to quibble with the details of this corner of Buddhist metaphysics. But this is just one minor example of how religion gets all kinds of stuff wrong — not even in the things that most people will notice, or that have much practical impact on the practice of the religion — just stuff that's casually wrong, wrong in the way that people will be wrong when they make stuff up without modern knowledge and modern standards of evidence. Religion is fractally wrong, and so when you turn over any random rock you'll probably find something wrong under it.

Buddhism is often portrayed as the most rational of religions. This is probably true, but that's a bit like saying that the pika is the least rabbity lagomorph.* It may not be very rabbity, but there are plenty of things even less like a rabbit.

Image by Wikimedia user Justin.Johnsen used under CC BY-SA 3.0; source

Squeezing truths from religion is like studying the ruins of collapsed ancient cities to learn how to build skyscrapers. In the ruins you'll find a lot of interesting history and occasionally some genuine beauty, but when you want to get stuff built, you'd do much better to just study modern architecture.

*OK, this is a terrible analogy, but I wanted to link to a pika because pikas are cute.

What do polyamory and hexadecimal have in common?

2009-07-26T11:39:00.000-07:00

The title of this MeFi post reminded me of two related linguistic curiosities: both polyamory and hexadecimal are hybrid mashups of Greek and Latin roots.

"Poly" is Greek for "many"; but Greek for romantic love is "eros". "Amor" is a Latin root. A more fiddly linguist would have coined the term polyerotics (Greek) or multiamory (Latin).

Likewise, "hex" is Greek for "six"; but the Greek root for ten is "deka". "Decimal" is Latin. In this case, more consistent coinages would have been either hexadecadic/deca-hexadic (Greek) or sexadecimal (Latin).

Interestingly, in both cases, at least one of consistent formations (polyerotics*, sexadecimal) is more sexually suggestive to the modern English speaker than the hybrid coinage. Coincidence?

*Actually, "polyerotic" seems to have been adopted by online polyamorist communities as a designation for specifically sexual polyamorous relationships. This is OK, I suppose, although I think the Greeks had it right that romantic love and sexual desire (which they denoted with the same word) cannot be cleanly cleaved in two.

The other pie in My Blueberry Nights (a correction)

2009-07-22T02:08:00.000-07:00

So, I've been slowly working my way through all Wong Kar-Wai movies, and tonight I watched My Blueberry Nights.

It's not a great movie. In many ways it's not even a good movie (although considered purely as a visual artifact, nobody who appreciates film could fail to be seduced). In many ways, it would work better as a long-form music video, and at times that is what it becomes. If you want to read a more considered opinion of the movie, then Dana Stevens's Slate review is reasonably astute. But I'm not posting this to review the movie per se. No, I have more serious business.

I have run across a couple of articles online suggesting that Jeremy (played by Jude Law) is eating a lemon meringue pie while Elizabeth (Norah Jones) is eating her titular blueberry pie. And since nobody appears to have corrected this*, I suppose it falls to me.

Does that look like lemon meringue to you? It is topped with an elegantly browned meringue no doubt, but the base cannot possibly be lemon jelly. No, this is a fruity filling of a more chunky consistency. Although it is definitely a yellow fruit.

Given that this is a Wong Kar-Wai film, the pie in question is clearly a pineapple meringue — Wong's sly, anorakish way of tying Jeremy explicitly to the protagonists of Chungking Express and Fallen Angels (both of which are far superior films, and among my favorite films ever).

Thus endeth the lesson.

Oh, all right, one more thing. If you ever get the chance, watch the scene with Chan Marshall (a.k.a. Cat Power) from this movie in a format where you can really see her face.

There's something about her slightly off-kilter line readings — "maybe you're just sen-ti-mental" — that suggests a whole history. You can hear the barbs of old emotions, catching at the rhythms of this character's speech even though she won't say them in words. Of course, who knows, maybe that's just how Chan Marshall talks. Or maybe I'm just reading too much into the awkward performance of an inexperienced actress. But regardless, the readings transcend the clunkily metaphorical lurch of the screenplay's lines here, and I'll remember this scene long after I've forgotten everything else about this movie but the pie.

*Note that Stevens herself is careful merely to aver that it is a "meringue" of unspecified provenance. Bravo.

Dandy Warhols promise unfulfilled

2009-07-14T22:54:00.000-07:00

"...we're covering Blackbird".

Sarah Palin's career: A concise explanation

2009-07-12T07:58:00.000-07:00

As with so many other things, the answer can be found in Bill and Ted.

She was a deeply thoughtless and incoherent woman, but she was briefly the anointed representative of a tribe; and amidst her incoherency, she occasionally uttered the ritual phrases of her tribe; and for these things they blessed her.

(This is the last thing, I hope, I will ever write about this person. But see Dahlia Lithwick's astute Slate piece if you really crave more.)

Matt Taibbi on "everyone else was doing it"

2009-07-07T23:23:00.000-07:00

Taibbi's article on Goldman Sachs has been getting a lot of attention. Today, he struck back at some of his critics in a most excellent fashion:

. . . even if it is true that “everyone else was doing it”: so what? Who cares? To me this response is highly telling. We published a piece accusing Goldman Sachs of systematically ripping off pensioners and other retail investors by sticking them with rafts of toxic mortgages it knew were losers, of looting taxpayer reserves to cover its bad bets made with AIG, of manipulating gas prices to massive detrimental effect, of helping to explode an internet bubble that caused over $5 trillion in wealth to disappear, and numerous other crimes — and the response isn’t “You’re wrong,” or “We didn’t do that shit, not us,” but “Well, Morgan did the same stuff,” and “Why aren’t you writing about Morgan?”

Why didn’t we write about Morgan? Because we didn’t. Because it’s your turn, you assholes. Maybe later someone will tell the story of the other banks, but for now, while most ordinary people are only just learning about the workings of the financial innovation era that blew up in their faces last year, the top dog in that universe is going to be first in line to get the special treatment. That might be inconvenient for Goldman, but it doesn’t make the things I or anyone else say about them untrue.

I find Taibbi's analysis uneven sometimes, but his writing and reportage are consistently excellent, and you should be reading his blog if you aren't already.

Japantown, SF, 08:36 a.m.

2009-07-07T08:37:00.001-07:00

On "listserv"

2009-07-03T07:39:00.000-07:00

Hypothesis: usage of the word "listserv" to mean "mailing list" generically is a shibboleth for non-technical Internet users of a certain age.

LISTSERV was the first mailing list management software, so you'd think that everybody who used the Internet before 1997 or so would call mailing lists "listservs". However, I don't think I've ever heard a programmer use the term "listserv" in the generic sense — at least not since the 90's, when LISTSERV itself was still in widespread use. Even back then, I think programmers and sysadmins mostly restricted its usage to mailing lists managed by LISTSERV specifically (as opposed to majordomo, lyris, mailman, or a plain sendmail alias).

(Speculation as to the reason for this distinction: non-technical users have a greater tendency to conflate general classes of software or protocols with specific instances of them. Example: the belief that a blue "e" is the icon for the Internet.)

Conversely, of course, kids who started using the Internet after web-based social software supplanted email and Usenet obviously don't even know what a "listserv" is. Unless/until they start working with some old fogies who use the term, I suppose.

Pretty close to how I feel about human social interaction these days

2009-06-25T21:24:00.000-07:00

The symbols differ but the sentiment is similar.

The strange thing is that I really like all my friends and co-workers but I still have this strange and inescapable distaste for "people" as a category.

Oblique reaction to an ending

2009-06-25T21:23:00.000-07:00

From Great Jones Street by Don DeLillo, pp. 181-182:

"The group broke up. We no longer exist as a group. Of course there wasn't any real hope once you left. Still and all it's frightening. Nobody was really prepared for it. But it happened. We no longer exist in the old sense."

"As of when?"

"I heard it on the radio coming in from the airport. When I left L.A., things were still in flux. Nothing was decided to the point where we could come out and say we've reached a decision. But I guess we broke up because I heard it on the radio. It sounded pretty official. Who has final word on these matters?"

"The radio," I said.

"A lot of it was my doing," he said. "I got heavily involved in black music. Not performing or producing. Just listening. That old showcase stuff with everybody in shiny clothes and pomaded hair. Brushed drums, piano, sax breaks. 'Baby don't you know that I love you so.' I'm into that sound, Bucky, and I can't get out. After all these years I realize that's the only sound I really love. So I neglected the band and now we no longer exist as a group. The little dance routines they do. Hands flashing out, feet gliding, bodies whirling so smoothly. Romantic soul music done by immortal groups. The Infatuations. The Tailfins. The Splendifics. 'It's a hurtin' pain you give me, babe, but I'm fightin' for my love.' It's all love and sorrow, Bucky, and it just about destroys me emotionally. The crude dumb emotion, it's so incredibly beautiful. Sorrowful ballads with occasional falsetto passages. And even when I'm just listening to records I can see them moving on stage, doing the little whirls and gliding steps, flashing out their hands. Shiny bright hair. Custom tuxedos. Fantastic teeth and fingernails. And the cheap emotion behind the lyrics just wrecks me. The Motelles. The Vanities. The Willows. The Renditions. The Flairs. Nate Pearce and the Hydromatics. 'Baby can't you see how you're upsettin' me, shoo-eee, shoo-eee.' Everything is there, Bucky. There's nothing else I want or need."

Busting prostitution: An efficient use of police resources

2009-06-20T09:52:00.000-07:00

So, today I ran across this from A. Lapite (original source at Gotham Gazette):

Robert Pinter, a 52-year-old gay man who was arrested for prostitution at the Blue Door in the East Village on Oct. 10, spoke at the town hall meeting. He said a young man — a 29-year old undercover cop who, Pinter said, looked even younger — cruised him in the store. He was "charming and persistent, and we agreed to go home for consensual sex, but as we were leaving he said, 'I want to pay you $50 [to have sex].' I didn't respond, but I thought it was strange," Pinter recounted. As the men left the store, Pinter said, a group of men who did not show police identification pushed him against the wall.

"I thought I'd been set up by a gang," he said. "I asked them why they were doing this to me. I was totally clueless. They handcuffed me and said, 'Why the f--- do you think we're arresting you — loitering for the purpose of prostitution.'"

and this, also via Lapite (read the whole thing, it's short):

It was Rocio Palacios who first noticed the woman who appeared to need help.

It was 8 a.m. when she and her husband, Erasmo, dropped their 6-year-old daughter off at school and had picked up their 22-year-old daughter to go out for breakfast when they saw the woman waving her arms at 53rd Street and Kedzie Avenue last November.

The Palacioses, of Chicago, claim the woman approached their car, parked outside Manolo's restaurant, leaned in to the passenger side where Rocio was sitting and asked Erasmo if he wanted oral sex for $20 or sex for $25.

The couple laughed, realizing this wasn't a woman in distress after all.

But within seconds, Chicago police swarmed the family car, hauling Erasmo Palacios out in handcuffs. He was charged with solicitation of a prostitute.

So, in case you haven't guessed by now, the title of this post is sarcastic.

Now, these two incidents don't prove anything by themselves. But both stories suggest that the police departments in question had adopted questionable policing strategies; and note that the NYPD and CPD are the two largest municipal police forces in nation, and doubtless among the best-funded in the world, with budgets of roughly $3.9 billion and $1.2 billion respectively. If the vice units of these highly professionalized forces are doing stuff like this, what happens elsewhere?

Decriminalizing prostitution may be politically unrealistic, but it seems to me that we ought, at a minimum, to be focusing enforcement effort on its genuinely pernicious aspects: coercive trafficking, prostitution-related violence, and underage prostitution. But, of course, busting a trafficking ring requires hard investigative work, whereas paying an individual undercover cop to "solicit" a consenting adult requires minimal skill and energy.

(Which — sigh — is exactly why we should legalize prostitution itself, and criminalize that which is genuinely harmful. Police have a finite budget of time and will allocate that time efficiently by arresting those lawbreakers who can be most easily apprehended. I'd argue banning prostitution per se makes life easier for trafficking rings by giving vice cops something to do besides busting trafficking rings. But whatever, I'm shouting into the wind here. The gates to prostitution decriminalization are guarded by the three-headed dog of social conservatives plus radical feminists plus the large majority of people who simply find prostitution "icky" and I'm pretty convinced that nothing's ever going to fix this problem in the U.S.)

On honesty and interpersonal relations

2009-06-19T07:15:00.000-07:00

So, a friend of mine has embarked on a project that involves a lot of dating, and on her most recent outing the man told her this:

I said, "So, Colombia, are you full of shit or what?" After a long pause, he looked me straight in the eye and said (seriously), "I've lied to women, but I've never lied to a lady."

Does this sort of line actually work? Because my immediate thought on reading this was, "Wow, hope he never decides you're the wrong kind."

Honest people are not honest because they think you deserve it; they're honest because it hurts not to be.

But, of course, we don't usually want to be around those people. Lying is an essential social skill. People who are very bad at deceiving others tend to cause awkwardness and don't get invited to many parties.* The unvarnished truth of human existence is that we are agents competing for scarce resources and our interests never align perfectly with those of other people. Lovers, husbands and wives, parents and children, brothers and sisters — to say nothing of friends or acquaintances or strangers you meet socially — all are engaged in a tug-of-war over who does the chores, who gets Dad's approval, who gets the girl, who's the center of attention at the party. Social life is a war for priority in the eyes of other people. Lies are the lubricant which allows us to pretend otherwise.

To genuinely forego participation in this game takes unusual will, perversity, obliviousness, narcissism, or some combination of these.

(As for me, I basically play the game, however ineptly, and I think this is what is turning me into a misanthrope.)

*Note that the converse is clearly not true: people who are socially awkward are not necessarily more honest. I think my friend's skepticism at her date's smoothness betrays a false belief that if he were more awkward, then he would be more trustworthy. Actually, I knew her ex; he was pretty awkward and he wasn't trustworthy at all. Most often, people are socially awkward simply because they lack the skill to be otherwise. P. Graham has interesting things to say about this, although I would add that Graham is being both too self-congratulatory and too optimistic: being socially deft does not require the sacrifice of one's intelligence; and as far as I can tell, success in the adult world seems to be most positively correlated with being integrated into the social networks of power, not with objective achievement in some discipline.

Iranian elections (random thoughts prompted by)

2009-06-14T13:59:00.000-07:00

I don't have much of value to add to what other, better-informed people are saying about the Iranian election, but I strongly recommend keeping your eye on this continuously-updated HuffPost roundup.

I also want to record some random thoughts that occurred to me as I was sitting at the coffeeshop waiting for my laundry —

While watching the protest footage, I was suddenly reminded of a moment, long ago, while I was traveling in Greece with a bunch of college classmates. Our Greek tour guide was saying something along the lines of "Ancient Greece had the first democratic government. Of course, you may wonder how the birthplace of democracy could keep slaves, but..." etc. There was no hint of irony in her utterance of this sentence. One guy in our group leaned over to me and said, in a low voice, "Does she realize she's talking to Americans? Slavery at the birthplace of democracy? Isn't that how it's done?" I surmised later that the tour guide's patter was probably designed with pan-European audiences in mind.

I don't know what exactly about the Iranian elections made me think of this, except possibly that both remind me of how thoroughly imperfect real-world democracies are, and how many different pieces have to fit together just exactly right to make this form of social organization function to modern standards.

I usually find Huffington Post annoying, but their post seems to be a better guide to what's going on than anything else I can find right now. This sort of real-time but human-curated index synthesizing links to news"paper" stories, blog posts, Twitter posts, and embeddable user-uploaded videos really does seem quite powerful.

It is also a type of media object which traditional outlets are currently ill-equipped to create (and possibly even ideologically opposed to creating). Compare the HuffPost article with the NYTimes equivalent: the latter has less variety, slower updates, and more focus on "official" sources. And this makes the NYTimes version worse, not better.

If Obama determined American foreign policy by listening to everything Joe Lieberman says and making sure to never, ever do that, then he would have a pretty good foreign policy.

Review: Dell Mini 12 with Ubuntu

2009-06-07T08:53:00.000-07:00

A few months ago, my old Thinkpad T42 died after 4 years of faithful service. I realized that I usually buy a $2000 laptop every 3 to 4 years; and as an experiment I decided to instead buy one between 1/3 to 1/4 of that cost, and see if it would last me over a year. I also wanted to switch from a 5-pound laptop to something lighter. So, I got a Dell Mini 12 with Ubuntu for about $550 (1.6 Ghz Atom processor, 80GB hard drive); including the second battery I spent $700.

After a few months of regular use, I feel ready to render a verdict.

The keyboard is OK, but not great. Most of the keys are fine, but the comma, period, and slash keys are half width, which is pretty annoying for a Linux-using touch-typing programmer. I've retrained my fingers to adequate speed under this arrangement (I now hit all three of these keys with my ring finger), but it took some time to get used to, and I'll never be as fast as with a full-sized keyboard.
You will hit the touchpad accidentally while you're typing. This is impossible to avoid. The touchpad is astonishingly sensitive. I could swear that it picks up tracks when my palm's not even touching the pad.
The screen is 1280x800, glossy, and about average quality for a notebook screen (which is to say, inferior to a Thinkpad or Macbook Pro screen, but fine for most uses).
Performance is not impressive. This is an ultralight with compromises. You can't watch Hulu videos full-screen with this computer, and there's noticeable lag when switching among multiple Firefox instances. Lately, however, I spend most of my mobile time either checking email/RSS feeds, or hacking LaTeX and OCaml in emacs while running Pandora in the background. For these purposes, it's perfectly adequate. (I suspect that if I were running Eclipse or some heavyweight development environment I'd be much less happy.)
Having a super-light computer is a definite pleasure. One of the most underrated human-computer interfaces is the one between your bag and your shoulder. The Mini 12 and its power adapter together weigh about 3 pounds. It feels qualitatively different from, say, a Macbook Pro or Thinkpad T-series.
This computer runs cool enough to both (a) stay in your lap indefinitely and (b) not require a cooling fan. The hard drive is extremely quiet as well. As a result, the computer's almost perfectly silent. YMMV but I find this extremely pleasurable. I don't think I've ever used a computer this quiet.
I got the model with preinstalled Ubuntu 8.04 (Hardy Heron). It's a huge bonus for me that I didn't have to reformat or repartition my hard drive and install Linux myself — something I've had to do with almost every computer I've bought in the past decade. Wireless networking, sound, Flash, sleep-and-resume, etc. all just work. The Mini 12 comes with the dell-lpia architecture Ubuntu distribution, rather than x86, but it mostly behaves like any other Ubuntu. (Exceptions: (1) when I first got my Mini, it would occasionally crash under heavy load, but recent kernel updates seem to have fixed the problem; (2) a few packages, such as smlnj, aren't yet compiled for dell-lpia, but this hasn't been a deal-breaker.)

So, the bottom line is that when I don't need to be on my employer's VPN, I throw this thing into my bag more often than my Macbook Pro from work. Take this with a grain of salt, of course, keeping in mind that I'm a fairly atypical computer user. But I, at least, am fairly happy with the purchase.

Shorter Tyler Cowen

2009-06-06T09:47:00.000-07:00

"If you adopt a fringe philosophical view on interpersonal versus intertemporal welfare comparisons, then everybody who disagrees with me about health care is wrong."

Cowen enjoys this kind of rhetorical maneuver, I suspect largely because there's nothing that pleases his ego more than to think of himself as not only smarter than liberals, but smarter in a much more unconventional way than liberals. By this standard, Cowen's post of this morning probably caused him to spontaneously ejaculate on his keyboard.

But anybody with an iota of ability to connect abstract ideas to the real world should intensely question a definition of welfare by which a "typical 23-year-old lower-middle-class immigrant has a higher real endowment than does Warren Buffett". In fact, I think you can find this sentence in the dictionary under reductio ad absurdum

And yes, I know what Cowen's referring to when he talks about McKerlie and egalitarianism. I do not find McKerlie convincing. It's the height of arrogance for Cowen to blithely assume that those who disagree with him are either confused about his argument, ignorant of the background, or irrationally rejecting a sound argument for emotional reasons, rather than simply disagreeing about the highly arguable philosophical conjecture which provides the foundation of his argument. Cowen's usually worth reading, but every once in a while all of his personal and intellectual flaws come crashing together in one horrible post that makes me want to get him banned from every restaurant on the planet.

Supreme Court 2009: the big picture

2009-06-05T00:22:00.000-07:00

No particular opinion about Sotomayor. She'll most likely replace Souter, Republican whining notwithstanding, and actually that's not very interesting. Souter's a liberal; Sotomayor's a liberal; the balance of the court will be unchanged.

At the risk of being excessively morbid, here's what matters, in roughly increasing order of importance:

Breyer: 70 years old.
Stevens: 89 years old (!).
Ginsburg: 76 years old.
Kennedy: 72 years old.
Scalia: 73 years old.
Life expectancy at birth for American men: 75.15.*
Time to Election Day 2016: roughly 7 years and 5 months.

It's a cliché by now to remark that the most enduring political successes of Bush 41 and Bush 43 were their Supreme Court nominations. Thomas, Roberts, and Alito are not only uncompromisingly hard-right conservatives; they're all 60 or younger and will probably stay on the court into the 2020s. The ascension of Alito to replace O'Connor was a particular disaster. Gonzales, Meredith, and Ledbetter were only the beginning. We're going to see many more ugly 5-4 decisions in coming years.

But if Obama gets re-elected in 2012, there's a non-negligible chance that he'll get to nominate five or six justices to the court in total, swinging the balance back to a liberal majority — maybe even a 6-3 majority.

Of course, that's assuming a number of coins all come up heads. It's far from certain that Obama's going to be re-elected; it's far from certain that Senate Democrats won't lose the majority or simply cave to Republican demands for conservative nominees; and I wouldn't be surprised if Scalia manages to hang on by his fingernails until he's 110. But the point is that it wouldn't take a miracle for such a dramatic transformation to occur within Obama's Presidency.

Is this surprising? Upon reflection, not especially. On scotusscores.com, the recent 1994-2005 stretch stands out as an unusual period of stasis — as J. Toobin remarks in The Nine, it's the longest interval the court's ever had with the same nine justices. But when justices stay together for a long time, it means that they're also from the same age cohort**, and therefore that they'll also reach retirement age together. O'Connor's retirement and Rehnquist's death were just the leading edge of a generational turnover that will run its course over the next decade or so. And since both parties are now savvy enough to nominate young justices, we can look forward to another relatively long period of stasis after that.

Incidentally, this also means it would be wise for the other liberal justices — Breyer, Stevens, and Ginsburg — to follow Souter's lead and retire as soon as possible, while it's certain who will be nominating and confirming their successors. But reading Toobin's book has convinced me that Supreme Court justices aren't unusually wise, so it's anyone's guess what they'll actually do.

*OK, time for the caveats. Mean life expectancy at birth is a misleading measure for seventysomething Supreme Court justices. Causes of "young death" like infant mortality and teenage driving accidents and AIDS significantly reduce mean life expectancy at birth, but have no relevance to the life expectancy of people who have passed those filters; so the life expectancy of a 73-year old is considerably higher than the life expectancy of a newborn. Furthermore, Supreme Court justices have an excellent government-funded health care plan, and disagreeing with the politics of the President who would nominate your successor to the Highest Court In The Land is a strong motivator to look after your health. And the probability that Scalia would leave the Court not in a coffin while there's a sitting Democratic President is basically nil. All in all, I suspect that at least one of Scalia or Kennedy will stay on the court past his 80th birthday. So 6-3 is a long shot. 5-4 doesn't seem too unlikely though.

**OK, excluding Thomas, who was an outlier on the Rehnquist Court in more ways than one.

$255,319

2009-06-01T23:36:00.000-07:00

...for every GM employee. R. Reich says we've sunk $60 billion into GM. Divide by 235,000 employees as of 2009q1 (according to Google Finance) and voilà.

Last December, I modestly proposed a $100K per employee bailout, for the more meager cost of $25.2 billion in late 2008*. But this is getting ridiculous. How many new small businesses, back-to-college tuition fees, and cross-country moves to more favorable labor markets could GM employees buy with $255K apiece, tax-free? There are places in the country where you can shelter and feed a family of four for the better part of a decade with that sum. I don't subscribe to the notion that uncoordinated masses of individuals axiomatically spend money more effectively than government all the time, but the orderly winding-down of a failing car company doesn't offer a propitious outlook for government management, not least because management of any kind seems essentially futile when you're dealing with a car sailing off a cliff.

Oh, why do I bother.

The car link was sort of fun to dig up though.

* GM had more employees back then.

On URL redirectors

2009-05-14T00:38:00.000-07:00

Back in the day, the address resolution layer of the Internet was DNS. DNS was an open, unified, distributed, redundant, highly-engineered address resolution mechanism which hid opaque identifiers like 209.131.36.158 under the hood of nice human-readable names like www.yahoo.com. Now we have a fragmented pile of proprietary, centralized, hacked-together address resolution mechanisms which replace a nice URL like freedom-to-tinker.com/blog/felten/modest-proposal-three-strikes-print with the opaque string bit.ly/4p3vL. Progress!

Dynamic dispatch is just higher-order programming

2009-05-12T21:24:00.000-07:00

Wow, this is pretty terrible coming from a Haskell programmer (via HN):

Dynamic dispatch is a scary programming technique. When you call a virtual method, you never know what might happen. This makes is [sic.] difficult to reason about such code, and code that is hard to reason about is hard to maintain.

This is true in the exact same sense that when you implement a higher-order function, "you never know what might happen". Here is a naive implementation of map in Standard ML:

fun map f [] = []
  | map f (x::xs) = (f x) :: (map f xs)

f could be bound to anything! OMG WTF BBQ!

Likewise here is a Java implementation of printList:

void printList(List aList) {
    for (Object o : aList)
        System.out.println(o.toString());
}

o.toString() could dispatch to anything! OMG WTF BBQ!

Well hold up man, let's try that in ML:

fun printList _ [] = ()
  | printList toString (x :: xs) =
    (print ((toString x) ^ "\n");
     printList toString xs);

Oh that's so much better isn't it. Or not. Basically, you will note that we had to type toString three times instead of once to achieve roughly the same effect.* Note also that this function only works over homogeneous lists; if you want a heterogeneous list, you'll have to define a union type and pack/unpack it yourself.

Object-oriented programming is a form of higher-order programming wherein related data and operations are tightly bound to each other, thus freeing you from the pain of having to wrap them up yourself, thus making it exceptionally convenient to pass data and functions together. Or, in other words, OO programming is just a functional programming idiom with lots of very convenient syntactic and semantic sugar on top**.

Now, the OO community has plenty of bad code and bad guruism bouncing around. Maybe there's somewhat less of that in the functional world. But IMO that's largely explainable by the fact that there are vastly more working OO programmers and vastly more OO code modules than functional equivalents. If the Haskell or ML community were as big as the Java community, there would be just as much terrible higher-order function spaghetti as there is terrible inheritance spaghetti today.***

* Q: Wouldn't the version using Haskell and type classes be as compact as the OO version? A: Maybe, but you'll still have to restrict the function's domain to homogeneous sequences of StringConvertable instance values. Once I hand you a heterogeneous list, you're back in the land of passing toString and packing unions. The only way to do this right is to introduce existential pack/unpack into your language, which basically amounts to introducing a slightly crippled object system with additional syntactic overhead.

** In particular, making super sends and self sends work correctly without OO sugar gets messy. OTOH, conversely, the object-oriented programming languages in common use make certain other kinds of functional programming idioms somewhat baroque — for example, Java's lack of a compact syntax for anonymous functions is a huge pile of Lose — but that's a discussion for another day.

*** In fact, I strongly suspect that with a fair number of kids today thinking that Haskell is the new hotness, that community's in for a rude awakening as they realize that Haskell's going to require style guides and "Haskell annoyances" books and assorted baroque frameworks in order to allow programmers of average ability to assemble software of similar complexity to that commonly assembled in Java today.

Robert Martin + Ruby = ughhhh

2009-05-09T18:36:00.000-07:00

A recent talk by Robert Martin on Smalltalk and Ruby is generating a lot of buzz today. Now, I just gave you the link to the talk, but don't take this to mean that I think you should watch it. I had a whole long writeup-in-progress about all the different kinds of Wrong embodied by this talk and its reception. But there was so much to explain, so many kinds of Wrong, that I just gave up.

Instead, I'll give you an abbreviated summary of a few reactions:

Holy shit, Martin is long-winded. A talk of this length with so few ideas* is offensively disrespectful of its audience's time.
Estrogen = weak and insipid? Really? Without a hint of irony? And not a raised eyebrow in the room? WTF?
The fact that many Ruby hackers seem to think that this was a good talk shows how bad that community's taste is.
Twitter's 140 characters are completely inadequate for discussing a subject like this; on the other hand it's a great medium for saying AOL! Which is, I suppose, one reason it has prospered.

I write this post mostly to record how depressed I am that software guruism of such low quality seems to be so popular.

If you want to wash the ugly taste of Martin out of your mouth, I recommend this interview with Leslie Lamport. Ruby hackers have esprit de corps and throw big buzzworthy conferences, whereas Lamport is merely a genius. Guess what the kids end up learning today.

*If you care at all, his entire argument can be summed up in a few sentences as follows:

Smalltalk is much more expressive than C++.
However, Smalltalk does not "punish you for making a mess"; i.e. you can easily write software that is intricate to the point of being incomprehensible. This is why Smalltalk failed.
Test-driven development can provide the discipline which stops the same thing from happening to Ruby.

As I've noted, I think this argument's wrong in so many different ways that it exhausts me mentally even to try to enumerate them all. But just as one angle of attack, note that of the suggested causal factor is false of many failed languages, and true of several popular ones. Forget about correlation not implying causation; I claim that in this case even the correlation is low. Furthermore, long-lasting popularity is so rare for programming languages that it is a fool's game to talk about why unpopular languages never became popular; it's much more fruitful to seek the very rare combination of characteristics common to successful languages.

p.s. UPDATE 8pm: What a relief, lots of other people thought the talk was b.s. too. See J. Robertson of Cincom

Condoleeza Rice: The rationalizing, supercilious face of evil

2009-05-02T10:59:00.000-07:00

When C. Rice retired from her life of crime, she decided to become a professor at Stanford. Unlike, say, Bush's retirement to his ranch, this position entails occasionally being confronted by young and idealistic students about her past:

Evil has many faces and this is one of them. It is a face that blandly smiles upon horrific acts and later spins an endless stream of moral justifications from whatever flimsy fiber is conveniently available. A morally whole human being, upon learning that prisoners had been subjected to controlled drowning hundreds of times due, in part, to actions that she had participated in, would acknowledge the horror, accept some kind of responsibility, express some kind of regret.

Rice does no such thing. She has entirely expunged any feelings of guilt. Any residual sense that she might have been wrong has been entirely conquered by her narcissistic need for self-justification. Observe her demeanor, and her language ("dear", "do your homework"): Not only does she feel that the student's wrong, she feels that it's vaguely unfair that people even think she did something wrong, and especially unfair that young uncredentialed whippersnappers outside the circle of power get to question her conduct.

Forget about the tortured prisoners; what about Condoleeza's suffering? Huh? It puts one in mind of an old Fafblog post.

As to the substance of her argument, it is nonsense, of course. It rests upon the twin assertions that

since the Bush administration orders were phrased so as to authorize only interrogation techniques cleared by the Justice Department, they were ipso facto legal; and
Chewbacca was a wookie — that is, lots of irrelevant misdirection about stuff like the uniqueness of al Qaeda (about which, in particular, A. Serwer has the best answer).

The first defense opens an infinitely large loophole in the rule of law — appoint your pet lawyers to Justice, and then trample on the law all you want! Even if you could, by some reading of the relevant legal texts, construct a legal theory that reached this conclusion, you would not want this reading to be the one that prevails in governance. To establish such a precedent would be disastrously bad as a matter of public policy. It would be much better to construct a different reading, one in which the most powerful people are actually constrained by the law.

If Rice were a dispassionate observer — if the very same quality of mind were transplanted into someone who had no personal or professional ties to the Bush administration or the conservative movement — it is hard to believe that an idea so patently bad and stupid would get past her. But self-interest and tribalist instincts are powerful motivators. And I suppose you go to war with the flimsy legal arguments you have...

...and not the ones you wish for.

Incidentally, judging just from the first video, you might conclude that Rice "wins" the confrontation. And in theatrical, TV terms she does: at the end she fishes out a gotcha question about Guantanamo which the student cannot answer. This was a foregone conclusion. It's not really a fair fight: Rice has had a long time to practice her legalistic justifications for her actions. These students are newbies. Her ability to fend off some callow undergrads demonstrates that an expert in any given subject can extemporaneously ejaculate a convincing wad of noise to defend anything.

But let's examine the two ways she deflects the student's question.

First, she goes on at length about Guantanamo not being a site for interrogations. This is correct; the torture was conducted by the CIA in locations around the world that have not been disclosed, although they're outside the United States. According to ABC News, one likely candidate is abandoned Communist military bases in Eastern Europe. Which is so much better than doing it in Guantanamo! But Rice doesn't mention any of that. The student's confusion regarding interrogation vs. indefinite detention is actually convenient for her, as it allows her to completely change the subject away from torture without being obvious about it.

Second, regarding Rice's gotcha question about putting the Guantanamo prisoners on trial: the answer to Rice's question is that federal courts halted the trials because they concluded that the trials were being conducted illegally. Or, more euphemistically, the tribunals convened by the Bush administration had failed to satisfy the requirements necessary to be called proper ("competent") military tribunals. Or, less euphemistically, Rice's pals setup some kangaroo courts; and she wants this fact to excuse her for holding thousands of mostly innocent people in concrete cells for years on end without providing them timely legal recourse.

Theatrically, Rice won the confrontation. But the cold light of later reflection exposes her rejoinders as total trash. The medium of video is not well-suited to reasoned argument.

C. Shirky on the end of newspapers (again)

2009-03-15T11:42:00.000-07:00

You must Read The Whole Thing, but here is a taste:

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors.

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away.

(Some of these sentences strike me as being more widely applicable than upheavals in industry.)

If newspapers were truly serving a critical social function (keeping the powerful honest by shining light into dark corners of the body politic, etc.) in the 20th century, then it was always somewhat weird that this function was invariably impossible without adverts for department stores and used car salesmen and lonely people looking to get dates with strangers. Perhaps our current configuration of information production is also strange: Google, Yahoo, et al. appear to serve a different critical function in the information economy, and they're also supported by adverts (the equivalence isn't exact; but I won't get into the differences here). Even so, it's really deeply weird that people supposed that the old weird arrangement would last forever — that no possible technological innovation or economic reconfiguration would upset this duct-taped applecart.

I mean, if you explained this to aliens, they would laugh at you. "Ha ha! Your species' highest form of social organization, the 'hybrid capitalist/socialist democracy', cannot function correctly unless local distributors of colored fabric find it economically necessary to piggyback solicitations on the physical distribution network of their local information discovery node!"

As Shirky points out, there's nothing particularly natural about the way things used to be. And the people who will save journalism aren't the ones complaining about technological innovation; they're the ones who are building something new.

Incidentally, lately I'm tempted to get a Kindle and buy some periodical subscriptions. I've realized that the obstreperous physicality of codexes and broadsheets really does affect my reading habits these days. S. B. Johnson has already observed that the Kindle's much better for reading while eating than a codex. And, of course, holding a codex or broadsheet one-handed on a crowded Muni bus, and then turning the page, is a pain in the ass. Meanwhile, although my HTC Dream has a capable web browser that's fine for most RSS and blog reading, it's not the greatest device for reading long stretches of text. Now, all these obstacles can be overcome, but the overall effect of these physical annoyances is to reduce the amount of time I spend reading long-form text or text printed on paper, and increase the amount of time I spend reading short-form text available electronically. But I already read so much short-form electronic text that it would probably be better to shift my habits in the other direction.

On the other hand, for reasons I have difficulty completely articulating, I would feel like a huge tool holding a Kindle on Muni every morning. So there's that to consider.

p.s. Johnson also had a piece yesterday on roughly the same topic as the Shirky essay linked above.

Fix dired under Fink xemacs (Mac annoyance Saturday)

2009-03-14T21:09:00.000-07:00

Tonight I literally spent a couple of hours grappling with an uber-annoying bug under XEmacs in current Fink: dired-mode chokes on directories containing symbolic links. The symptom is that when you try to open a directory in dired, a cryptic "no file on this line" error appears in the minibuffer and dired fails to open.

To make a long story short: this fix from the Apple support forums is (unbelievably) the fastest and most foolproof way to fix this error. Step by step:

sudo xemacs /sw/lib/xemacs/xemacs-packages/lisp/dired/dired.el

Find the line containing the regex

"[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][-+ 0-9+]"

and change the regex to

"[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][@\+]?[-+ 0-9+]"

C-x C-s
M-x byte-compile-file
C-x C-c
Quit and restart XEmacs.

Of course, you will need to do this again whenever Fink decides to overwrite dired.el; unless the Fink maintainers merge in the upstream fix.

Gory details below.

Under Mac OS X, unlike every other Unix in recorded history, ls inserts an @ or + sign after the permissions list on certain types of files (symlinks and files with "extended security information"). The dired.el code is brittle with respect to this oddity. Furthermore, it does not provide any way to customize the regular expression that matches and parses the output of ls (which, incidentally, is my primary evidence that Mac OS X is the first Unix in history to behave this way; it's essentially impossible that a Unix could have existed for any length of time like this without a patch being fed upstream to Emacs).

Now, it turns out that this bug was fixed in Dired 7.15. So the real problem is that Fink XEmacs has not pulled down the latest Dired; it's currently stuck on 7.13.

So, what's a hacker to do? Let me count the frustrations.

1. XEmacs has a built-in elisp package manager called pui. Maybe I can upgrade Dired that way?

I use M-x pui-list-packages; then I set a package mirror from the Tools > Packages menu; finally, I select the Dired package and type I to install:

(error/warning) Error in process filter: (ftp-error FTP Error: CWD 550 /pub/packages/editors/xemacs/packages/.: No such file or directory failed: )

Oh great. I try a half-dozen different package mirrors and every motherfucking one gives me this error (except when it dies even earlier due to some other FTP problem).

2. Aha, MacPorts has a more recent XEmacs! Screw Fink, maybe the MacPorts guys know what they're doing:

$ sudo fink remove xemacs
. . . long console spew . . . 
$ sudo port install xemacs
. . . long console spew . . . 
$ xemacs
Error: attempt to add non-widget child "*scratch*"
to parent "Buffers" which supports only widgets
[xemacs crashes]

The current MacPorts package of xemacs doesn't even start up. Bloody motherfucker.

3. Maybe I should just switch to FSF Emacs, which I already use on all my other boxes. I mean, Fink has emacs22-gtk now. Yeah, right:

$ sudo fink install emacs22-gtk
Information about 8097 packages read in 1 seconds.

. . . [deletia] . . .
The following package will be installed or updated:
 emacs22-gtk
. . . [deletia; making a long story short:] . . .
sed: /usr/X11/lib/libfontconfig.la: No such file or directory
### execution of /var/tmp/tmp.1.UdYe2K failed, exit code 1
Removing runtime build-lock...
Removing build-lock package...
/sw/bin/dpkg-lockwait -r fink-buildlock-fontconfig2-dev-2.4.1-106
(Reading database ... 42622 files and directories currently installed.)
Removing fink-buildlock-fontconfig2-dev-2.4.1-106 ...
Failed: phase installing: fontconfig2-dev-2.4.1-106 failed

4. So, I finally gave Carbon Emacs a try. Carbon Emacs runs, and dired actually works fine. Miracle of miracles.

Unfortunately, fonts are completely busted under Carbon Emacs. You get a slick anti-aliased font out of the box, but you can look forward to a hair-pulling exercise in frustration if you attempt to change that font to anything that's (a) legible and (b) fits more than 50 vertical lines of text on a Macbook Pro screen. Anti-aliased Courier looks even more terrible at 8pt than Courier usually looks. And Carbon Emacs is the first windowed Emacs I've used where no variant of the "Clean" font works out-of-the-box. Congratulations Apple! You've broken a piece of software that's basically worked for twenty years. So Carbon Emacs is out.

5. About twenty minutes of screwing around trying to install dired from CVS into a user-local directory, and have it override the one in the system directory, yields no progress.

6. I could try to install the latest dired package by hand, but I am sort of scared of what else might break if I do this. So, as much as I dislike manually twiddling with system files that should be under my package manager's control, I grit my teeth and did the voodoo noted at the top of this post. Which finally gets me everything I want, but my mouth is left with the distinct bitter residue of time sucked away from my life that I'll never get back.

My conclusions from this whole experience are as follows.

First, package maintainers for Fink and MacPorts aren't as good as the maintainers for Debian or Ubuntu. At any given time, a fair number of important packages will simply be nonfunctional. So the package situation will always be a pain in the ass on Mac compared to Linux.

Second, Mac OS X's random gratuitous differences from other *nix flavors will continue to suck away at least as much time from my life as I spent getting wireless and other random hardware widgets to work on a Linux laptop. Truly, for a programmer, there's no such thing as a laptop where you don't spend a significant amount of time fiddling with random bullshit just so you can get work done.

Option key as Alt in KDE apps on Mac (Mac annoyance Saturday)

2009-02-28T08:20:00.000-08:00

The default keyboard setup of Apple's X11.app for Mac OS X doesn't support Alt- shortcuts in KDE apps (at least not when KDE's installed via current fink, e.g. by sudo /sw/bin/fink install kdebase). This makes konsole just about unusable, as konsole translates Alt events into Meta keypresses on the underlying terminal, and discards true Meta events. To fix this, assign the keycodes as follows in your ~/.Xmodmap:

keycode 66 = Alt_L
keycode 69 = Alt_R

Then, ensure that your X11 "Preferences..." are as follows (dialogue box is from Leopard):

Finally, restart X11.

For completeness, here's the keyboard layout kcontrol module (you shouldn't have to do anything here, as these are the default settings):

Credit to this macosxhints.com post, which took an astonishing amount of searching to sift from the vast oceans of trash and non-working solutions that you get when you search for X11 issues for Mac. I post this in the hope that it will serve as more efficient Google-food.

(Background: I recently killed the hard drive on my ~2.5-year-old work MacBook, and so I've been reconstructing my work environment on the shiny new MacBook they requisitioned for me. Running emacs in a screen session on a remote Linux box is one of my most common activities, so having konsole with no Alt/Meta makes Mac usage just about unbearable. I could not for the life of me remember the magical incantation I did to fix Alt last time, so I literally spent about half a workday intermittently typing random permutations of "kde", "fink", "alt", "meta", "konsole", and "leopard" into my web search box. Blech.)

Centrism begets extremism, in a precise game-theoretic sense

2009-02-26T22:20:00.000-08:00

(Excavated from recent unpublished drafts.)

This is not rocket science.

Suppose a couple* is getting divorced and in joint possession of an indivisible good, such as a house. They have agreed that the wife should keep the house, but pay the husband for 50% of the value of the house. Now they get into an argument about the value of the house: the wife will try to claim that the house is worth less, and the husband will claim that the house is worth more. Who will help them resolve this dilemma?

They first take the claim to Carol the Centrist, whose arbitration strategy is "split the difference" because she believes that "whenever two parties disagree, the truth is always somewhere in the middle". Carol will take the average of the wife's estimate and the husband's estimate. Immediately, the wife will claim that the house has value negative infinity, and the husband will claim that the house has value positive infinity. Since the average of negative infinity and positive infinity is undefined in the general case, no agreement will be reached and Carol's arbitration strategy is fucked. Meanwhile, Carol's silly faith in centrism has driven the husband and wife even further apart than they were to begin with.

The husband and wife realize that Carol is a fool, and they take their dispute to Dave the Decider. Dave understands game theory, and therefore he says: "Each of you write your claimed value down on a piece of paper. I will get an independent assessment of the house's value, and just pick whichever of your claims is closest to the assessment." The husband and wife immediately realize that it is in their interest to bid as close to the true value as possible. Given access to the same information, they will simply bid exactly the same price and Dave will not even have to hire his own assessor. He collects a fat arbitrator's fee and goes home. Meanwhile the husband and wife's set of disagreements has been reduced by one.

This is an extremely simple illustration — in fact, literally a textbook illustration — which I learned from my undergrad class with S. J. Brams (previously mentioned). I say all this by way of commenting on M. Yglesias's and H. Hertzberg's posts about a month ago on the wages of "centrism".

What's amusing is that centrists imagine themselves as essentially a moderating influence, when they are exactly the opposite.

This also puts the lie to the claim, which I sometimes hear journalists making, that if you're getting criticism from both the left and the right, you must be doing something right. Journalism that presents "both sides" without taking sides will, in the long term, lead to an objectively less well-informed public, as it creates bad incentives for the actors. As astonishing as it seems, it would be far preferable to attempt to determine the objective truth, and then print verbatim whichever press release is closer to that truth (so long as you announce this strategy beforehand).

Of course, it's not necessary to go to that extreme. One could instead devise a type of journalism which simply says, "A says X, which is close to the truth. B says Y, which is more of a lie compared to X. Here are the details: ..."

*Given the rest of this post, can you guess the name of the wife and husband?

Free music for the day

2009-02-14T11:29:00.000-08:00

Empire of Love: V-day covers from Mobius Band. (previously; let us hope this becomes a tradition)

C. Shirky vs. W. Isaacson on small payments and newspapers

2009-02-14T10:59:00.000-08:00

Clay Shirky finally has a weblog, and his first post "Why Small Payments Won't Save Publishers" is, like everything else he writes, worthy of your attention (via delicious/cshalizi). Not much new here if you've read his previous critiques of micropayments, but worth reviewing nonetheless in light of recent noise w.r.t. funding news publishers via small payments. I have a couple of (extended clumps of) remarks...

First, Shirky writes:

Faith in salvation from small payments all but requires the adherent to ignore the past, whether existing critiques (e.g. Szabo 1996; Shirky 2000, 2003; Odlyzko 2003) or previous failures. Isaacson’s recent Time magazine cover story on micropayments, How to Save Your Newspaper, a classic of the form, recapitulates the argument put forward by Scott McCloud in his 2003 Misunderstanding Micropayments. That McCloud advanced the same argument that Isaacson does, and that the small payment system McCloud was proselytizing for failed exactly as predicted, seems not to have troubled Isaacson much, even though he offers no argument different from McCloud’s.

It's worth reading the above-linked critiques by Szabo, Shirky, and Odlyzko, and then Isaacson's article, to realize just how terrible the latter is. It goes on a jaunt through the history of paid content, including AOL and CompuServe and (no kidding) Ted Nelson's Xanadu system and a half-dozen failed micropayment startups, and ends up with the conclusion that this time, small payments just might work!⁰ Isaacson breezily dismiss criticism of small-payment systems as follows:

Many tracts and blog entries have been written about how the concept can't work because of bad tech or mental transaction costs.

But things have changed.

This is literally the article's only attempt to engage the logical arguments of small-payment critics. He gives no description of the actual "bad tech" or "mental transaction costs", and he does not even seem aware that critics like Odlyzko have discussed considerably more than the technological shortcomings and transaction costs of particular payment systems.

And what exactly has changed, according to Isaacson?

"With newspapers entering bankruptcy even as their audience grows, the threat is not just to the companies that own them, but also to the news itself," wrote the savvy New York Times columnist David Carr last month in a column endorsing the idea of paid content. This creates a necessity that ought to be the mother of invention. In addition, our two most creative digital innovators have shown that a pay-per-drink model can work when it's made easy enough: Steve Jobs got music consumers (of all people) comfortable with the concept of paying 99 cents for a tune instead of Napsterizing an entire industry, and Jeff Bezos with his Kindle showed that consumers would buy electronic versions of books, magazines and newspapers if purchases could be done simply.

In other words:

"Newspapers are more desperate than anyone's ever been, so it's just gotta work!" (By the same logic, if you've never gotten laid, an Olympian athlete/particle physicist/supermodel with an extraordinarily dextrous tongue will surely show up at your doorstep tonight with a bouquet of flowers and a bottle of wine.)
"By generalizing from services that run on proprietary vertically-integrated hardware-software-service stacks, we can predict the success of small-payment systems for the Internet!" (Isaacson makes this same error when he cites text messaging elsewhere in the article.)

So, Isaacson's article displays objectively shoddy analytical reasoning. Furthermore, by failing even to outline the arguments of small-payment critics, he leaves his readers in the dark about an important aspect of his subject (duty to inform be damned). And to top this stupidity sundae with a cherry of irony, this terrible article about how to save journalism was just published on the cover of one of the nation's leading newsweeklies, in a week when Congress is passing a $800 billion economic stimulus package with huge implications for the national and world economy.

Why did Time's editors choose to run this article, rather than, for example, an article by Shirky or Odlyzko or any number of people who would write something more clueful? I hypothesize two reasons. First, Time's editors themselves do not have a clue, and also do not have any problem publishing articles on a subject they have no clue about. Second, look at the author blurb at the bottom of the article (emphasis mine):

Isaacson, a former managing editor of TIME, is president and CEO of the Aspen Institute and author, most recently, of Einstein: His Life and Universe..

When you're a member of the club, your buddies will publish any old crap you write; better you than some stupid professor nobody knows. We've seen this before.

I mentioned irony earlier. Isaacson has filigreed the irony with extraordinary precision. His article is inferior to material produced for free online by people who draw their paychecks from other sources (Shirky and Odlyzko are both professors who also work(ed) in the private technology sector). Furthermore, it is inferior as a direct consequence of structural weaknesses of traditional magazines. Despite its inferior quality, it presumes its own superior status by ignoring or dismissing contributions to the discussion which occurred outside of traditional "journalistic" media. Finally, taking that superiority as a given, it argues, poorly, that people ought to pay money for products like itself, because (quoting Bill Gates) nobody can "afford to do professional work for nothing".

In short, Isaacson's article not only fails to make its case, it actively undermines its own case while doing so.

Second, Shirky writes:

Another strategy among the faithful [small-payments-advocates] is to extrapolate from systems that do rely on small payments: iTunes, ringtone sales, or sales of digital goods in environments such as Cyworld. (This is the idea explored by David Carr in Let’s Invent an iTunes for News.) The lesson of iTunes et al (indeed, the only real lesson of small payment systems generally) is that if you want something that doesn’t survive contact with the market, you can’t let it have contact with the market.

Apple’s ITMS (iTunes Music Store) is perhaps the most interesting example. People are not paying for music on ITMS because we have decided that fee-per-track is the model we prefer, but because there is no market in which commercial alternatives can be explored. Everything from Napster to online radio has been crippled or killed by fiat; small payments survive in the absence of a market for other legal options.

I think Shirky's overstating the case a bit here. ITMS, Kindle, and cell phone ringtones operate in functioning, albeit flawed, markets. The iPod/ITMS stack has considerable competition: there are numerous music player devices and ways of paying for music online (Amazon, eMusic, Magnatune, etc.). Kindle's not the first or only ebook reading device; it's just the best blend of convenience and device quality currently available. And you can buy unlocked phones and swap in a SIM card from your phone provider (for some providers)¹.

What distinguishes all of these cases is that they are, as I mention above, proprietary vertically-integrated products in which end-user hardware plays an integral role. Unlike software, hardware is currently inconvenient to copy or modify. Therefore, control over the end-user's physical device makes it comparatively easy to control the digital supply chain. Unsurprisingly, this enables restriction of the content supply so that users will tolerate making small payments for access to the available content.

(Naturally, this answer raises the question of why people choose to consume music, books, and mobile conversations on devices which, compared to the highly promiscuous and capable general-purpose computer, amount to Puritanical, crippled shut-ins. I think the answer is fourfold. (1) For obvious reasons, it is easier to build a seamless user experience for a specific application on a vertically-integrated special-purpose platform. (2) A lot of our cultural output is still published² by old-media businesses that feel more comfortable distributing their content on less-capable devices. (3) Users do not commonly process music, books, and conversations through authoring and sharing tools, and without such tools there is only a very weak "demand pull" towards general-purpose device platforms. (4) This will all change; someday your kids or grandkids will have portable devices running an open OS on commoditized general-purpose networked computing hardware, and they will use it to record, copy-and-paste, annotate, remix, and share bits and pieces of music, books, and phone conversations, as we do today with web pages and photographs.)

[0] Lest I be accused of misrepresentation, Isaacson does not cite AOL and CompuServe content as examples of small-payment systems, which they were not. They're just part of his tour of failed user-pays business models of the past, and frankly it's bizarre that he would lament that ISPs don't pay content providers for content as AOL and CompuServe did. The fundamental differences between an open network and a proprietary network are exactly what made AOL and CompuServe fail.

[1] Naturally, unlocked phones are pricier than locked ones, as unlocked phones are not subsidized by the phone plan. That's perfectly fair. (OTOH it grinds my gears that phone companies don't let you opt-out of your plan's phone subsidy if you bring your own phone, and I believe this bundling should be outlawed, but that's a long argument I won't get into here.)

[2] Note "published", not created. Creative work is created by individuals or small groups of individuals: writers, musicians, etc. Creative work is only published — which is to say bundled, converted into distributable form, and then distributed — by these companies. A publishing company's job is to pack things together and move things around, and we should not let them assume for themselves the aggrandizing aura of the creative process.

Rat Park & addiction link roundup on MeFi

2009-02-12T22:07:00.001-08:00

If this is true, it's pretty interesting. It reminds me of the skepticism I have, for similar reasons, about psychology experiments conducted with a couple dozen undergrads in a University lab.

(Apologies for the lazy link propagation.)

Urban pranksterism you can believe in

2009-01-25T23:36:00.000-08:00

By way of explaining my previous post to people not resident in the Bay Area, LaughingSquid brings us a video and press release allegedly from those responsible. If you prefer the work of Real Journalists then you may also peruse this Google News cluster, of which the SFGate blog is probably the best representative.

Incidentally, I'd bet good money that there will someday be an officially recognized Barack Obama St. in San Francisco.

Limits to trust in the executive

2009-01-25T22:31:00.000-08:00

W. Wilkinson points to some mostly good things Obama's done so far* to limit executive power. And, on the one hand, provided he sticks by these commitments, that's great for the next 4 or 8 years.

But on the other hand, the very fact that a President can change these norms simply by issuing an executive order is deeply disturbing. The next President can override these orders with different ones. This can be read as evidence that elections matter — contra the Greens and assorted other cynics who say there's no meaningful difference between the candidates of the two parties — but elections shouldn't matter in this particular way. It just shouldn't be possible for the President to run the executive in a way that's completely unanswerable to the public's questioning, to say nothing of the law.

You can't hand the fox the keys to the henhouse and then simply trust in the fox's integrity. Even if you manage to find the one vegetarian fox in the world, you would not call "finding vegetarian foxes" a long-term strategy for henhouse protection.

Which is why Congress has to play a vigorous role in limiting executive power. Which, in turn, is why it was so shameful for Obama to capitulate on giving Verizon and AT&T immunity for their FISA lawbreaking, and why it's so necessary for Pelosi and Reid to open vigorous investigations into the Bush administration's past behavior.

Sadly, I don't think the Democrats are up to it. It's still early days for the newly elected Congress, but I suspect that ongoing crises of various kinds will serve as ample excuse not to pursue "divisive" investigations.

*Although I don't see the point of freezing the salaries of top White House aides, except as a temporary and symbolic gesture. In a multi-trillion dollar Federal government budget, saving a few millions of dollars in salaries at the top, and thereby risking not attracting the best of the best, is penny wise and pound foolish. Certain super-high-prestige positions will attract top caliber people simply by virtue of the job title. But as you go further down the ladder, you get positions where the title's less impressive, but it's still important to attract people whose other options would include high-paying and high-prestige positions in the private sector. It's all fine and well to say that public servants should be public-spirited, but when choosing their careers, real people balance financial compensation, personal satisfaction, and the public interest. For some skill sets, private-sector work pays more than public-sector work and simultaneously offers less frustration, more autonomy, and still some sense of serving the public interest (albeit via market mechanisms). Do you want to lose people with those skill sets? What marginal reduction in the efficiency of the executive branch will you accept in order to maintain the fiction that White House employees are motivated by public-spiritedness and nothing else? If you'll accept 0.1%, then you've just wasted a couple billion dollars. An expensive fiction.

Corner of Bush & Buchanan, San Francisco, 6:35 a.m.

2009-01-20T07:58:00.001-08:00

Apologies for the low light; it was an early morning today.

Quickie video reviews

2009-01-19T23:26:00.000-08:00

I've been intermittently sick the past few weeks. Partly as a consequence, I've been spending an unusual amount of time sitting around watching videos, including some that I've seen before. I'll try to extract some value from this lost time by passing on some brief remarks:

Casino Royale: Surprisingly good. Tightly constructed and executed with grace and wit. Additionally, the action is shot with uncommon telegraphic clarity for a modern action movie: if you pay close attention, you will understand where everybody's body is at all times. I received other people's positive reviews with skepticism, but that skepticism was overcome. Note that I don't like other James Bond movies.
Tian Mi Mi (a.k.a. Comrades: Almost a Love Story): My quest to watch every Maggie Cheung film continues. This film is, also, surprisingly good, and I plan to write a lengthier post on it Real Soon Now. English subtitled videos of this film are difficult to obtain, but if you can get your hands on it, you should give it a watch.
Duo Luo Tian Shi (a.k.a. Fallen Angels): Still one of my favorite films ever. I actually like it better than Chungking Express or In the Mood For Love, Wong's two best-known-stateside films. The memory of this film's visual potency is pretty much indelible, but I had forgotten how kinky and how silly this movie is. And the final shot still slays me; it may be one of the most miraculous moments in all of cinema.
The Lady Eve: Vastly overrated. It's watchable enough to hold your attention for 90-odd minutes, but it's neither especially funny nor especially romantic. It also has not aged well; to take one random example, Pike's reaction to Eve's "confession" of her non-virginhood just doesn't resonate anymore, and without this tension there is simply no potential energy to be released by laughter. On the other hand, Barbara Stanwyck does have nice legs.
Friday Night Lights (first season & half of second season): Not the sort of show I usually watch, but I put this on my Netflix streaming queue based on recommendations from various quarters (e.g. Yglesias). And it's not bad, at first. Small-town life is sufficiently foreign to me that there's a voyeuristic, almost anthropological charm to watching this carefully textured depiction of it. On reflection, however, every other virtue seems balanced by a significant flaw: the sports scenes are well-executed, but highly predictable; the actors' body language and line delivery are uncommonly naturalistic for television, but the dialogue itself is frequently preachy or contrived; the characters and their interpersonal relationships are layered and well-drawn, but the writers cannot resist the temptation to tie up most conflicts with a neat bow within 1 to 3 episodes. Given this mixed bag of fundamentals, much hangs on the execution of particular episodes or character arcs. On balance, I found the first season enjoyable enough to bear its flaws; by contrast, the second season goes precipitously downhill in many ways, and I've mostly been playing it in the background while I putter around on my laptop.
Batman Begins: Significantly worse than my memory from seeing it in the theater. The movie comes apart badly after Wayne returns to Gotham. The League of Shadows plot feels wildly implausible*, and the stagey "urban chaos" scenes in the Narrows feel too theatrical and insufficiently cinematic. Also, the writing throughout leans too heavily on awkward verbatim callbacks (repetitions of lines said earlier in the movie). On the other hand, Katie Holmes is nowhere near as bad as is popularly remembered; the problem is that she's turning in a merely adequate performance in a movie populated by Christian Bale and Gary Oldman. (*Of course, all comic book movie plots are implausible in some sense, but what I'm saying is that it feels implausible — that is, it strikes a tonally false note, and I'm not sure exactly why. Maybe it's because I'll suspend disbelief for a hokey magical ninja cult boasting (perhaps falsely) about thousand-year-old conspiracies in the alternate-universe Tibetan mountains, but I can't do the same in the context of ~~Manhattan/Chicago~~ Gotham.)
The Dark Knight: Better than my memory from seeing it in the theater. I recall thinking that the Two-Face transformation seemed abrupt and ridiculous, but on rewatching, it holds together somewhat better. Dent's vigilantism begins with straightforward acts of revenge, which seems psychologically plausible, and proceeds to total derangement only by degrees. It's not completely successful, but it doesn't ruin the movie. (However, the coin gimmick remains silly.) In retrospect, Heath Ledger's performance has been overpraised, but it is compelling; listen to how he uses his voice. Finally, the action set pieces remain spectacular.

Power outlet, JFK Terminal 3

2008-12-28T13:23:00.001-08:00

Note to self: Pack a PowerSquid next time I travel.

MoJo on auto credit

2008-12-23T09:46:00.000-08:00

Via cshalizi's delicious, Mother Jones comments on auto industry lending practices. It turns out that sleazy salesmanship w.r.t. the cars themselves isn't the half of how auto dealers screw consumers.

On the bright side, the article says that 900 car dealerships will fail over the next year.