More on taxis

I will fully cop to the fact that my previous bitching about taxis concerned a classic San Francisco yuppie firstworldproblem but here are some more stories about how taxis in many cities serve people of all stripes exceptionally poorly.

Relatedly, I took an UberX with a friend the other night and the experience was awesome.

I might have liked the option of tipping the the driver (via the app, after you get out of the car), but UberX doesn't allow that. On the other hand, maybe tipping would establish a social norm for companies like Uber to underpay their drivers with the expectation of tips? I really like traveling abroad in countries that don't have a tipping culture; it seems more rational and arguably in the long run leaves labor in a better position since their compensation is assured by contractual terms rather than manners. So maybe it's great that Uber doesn't have tipping, only reviews.

What happens when you let people pay to remove ads

Disclaimer: Obviously I worked at Google for 6.5 years so take this with the appropriate grain of salt.[0]

"I wish Google would just let me pay them money to remove ads from this service. WTF GOOGLE IS SO DUMB AND EVIL WHY WON'T THEY DO THIS."

How many times, in various corners of the Internet, have you read some variation of the preceding sentiment?

Now, it turns out, Google is trying to do this with a subset of music videos on YouTube. But when you do this, you need a contract with the rights holder. This is complicated. The rights holder might not agree to your contract terms. For example, they might want more money. So you end up in negotiations. While those negotiations are ongoing, or if they break down, you can't include that music in your paid service.

The Internet's collective reaction has been: "WTF GOOGLE IS SO DUMB AND EVIL WHY ARE THEY DOING THIS?"

Now, YouTube will still host music videos that are not opted into its monetization program, i.e., if you are a band and want to put music videos on the Internet for free, YouTube will host those videos for you. Let's be clear what this means: if you want to distribute a video to a hundred million strangers on the Internet, YouTube will pay for the software and servers and a petabyte of network bandwidth and a small army of SREs holding pagers who will wake up at 3am if too many people in Kuala Lumpur click on your video and get an HTTP 50x error, and it will do so without you paying YouTube a dime. None of that's changing. But if you opt into YouTube's monetization program, you will have to opt into its full, updated monetization program: ads for non-subscribers, and no ads for subscribers.

Note that there is no sensible way to let users pay to remove ads from music videos while also still showing ads to those same users for some music videos. If YouTube did this, you can bet the Internet would collectively scream, once again, "WTF GOOGLE IS SO DUMB AND EVIL WHY DON'T THEY REMOVE ADS ON MUSIC VIDEOS WHEN I PAID TO HAVE ADS REMOVED? I'm unsubscribing from this bullshit service!" The subscription service would fail and YouTube would have to revert to its old model of monetizing via ads only.

In short, "blocking" — i.e., excluding unlicensed music from the monetization program — is an inevitable consequence of having a paid subscription service.

The press has done an abysmal job covering this. It seems that every year my contempt for (most) journalists finds reasons to grow greater and greater. I'm pretty disappointed that the Financial Times, reputedly pretty reliable, appears to be Ground Zero for this particular blast of misinformation.

p.s. As for claims that YouTube has anything approaching a monopoly on online video sharing, I'm honestly puzzled by the claim. Just to take one example competitor, Vimeo is pretty reliable and seems to have rough feature parity, including embedding in external sites. Vimeo's content is included in Google's video search corpus (example) and therefore shows up as a thumbnail image in Google web search, just like YouTube videos. If you need to make money, Vimeo has a few built-in monetization options; if those are insufficient, there's a lot of innovation occurring in the world of funding, and in my opinion the Patreon/Subbable model seems much more promising for creators with a small-to-medium-sized audience than Spotify-style monetization, which currently amounts to "we'll give you 0.0001 pennies per stream play so that both the company and the artists can lose money hand over fist!" Is it really true that Adele fans will face substantial (or even non-substantial) barriers to watching her music videos if they're on Vimeo? And that's just one competitor. The upshot is that indie labels are probably wrangling with YouTube over licensing terms not because it has anything like a monopoly, but because they think wrangling with YouTube will make them more money than all the other alternatives. Which is perfectly fine — more power to them, negotiating the best deal is essential in business, etc. — but we should not misread the situation.

---

[0] On the other hand, one benefit of no longer working there is that I can write stuff like this post, just like I used to before I worked there. Arguably, working at Google made me less predisposed to harshly criticizing misinformed critics of Google.

Uber and taxis in San Francisco

Uber is a national story now, but this post from a former colleague reminded me of a little fact about Uber's origins that many observers outside of San Francisco probably don't know.

Before Uber existed, San Francisco taxicabs were extremely unreliable. On busy nights, you would commonly call a cab dispatching company on your cell phone, talk to a dispatcher who promised that a cab would arrive, wait half an hour or more, and still have no cab show up. This only has to happen a couple of times, on a chilly San Francisco Saturday night while your date stands on the street shivering in her dress and heels, before you start saying to yourself, "Fuck all taxicab companies, forever."

Conversely, I speculate that it was not uncommon for cab drivers to receive a call, only to have the passengers disappear, since they'd grabbed another cab that happened by sooner. In fact, once I concluded that taxi drivers were as likely to flake out on me as not, I started doing this myself. What else could you do?

In other words, game-theoretically, taxicabs and passengers were stuck in a low-trust equilibrium. Passengers could not rely on cabs, so they would grab any cab they could, even if they'd called one. Cabs could not rely on passengers, so they would pick up any fare that looked promising, even if they had been dispatched. A taxi company could have disrupted this equilibrium by offering reliable service and building a brand name as such. None of them did. Much like your local cable company, they were collectively content to sit on their government-protected oligopoly and treat customers like garbage.

(Notice that passengers can't disrupt this equilibrium: there's no equivalent "union of passengers" to which a reputation for reliability could be attached.)

The most important thing about Uber, at least initially, was not that you called it with a handy smartphone app, or that Uber cars were fancy black limos, nor even that they always accepted credit cards.[0] It was simpler than that: Uber cars came when you called. Uber achieves this result through a variety of means, but the most obvious is simply holding individual drivers accountable for every ride.

Therefore, at least in San Francisco, the whining of taxi companies exposed to competition from Uber and Lyft is the whining of any industry that serves its customers badly, and then is exposed to superior competition that serves its customers well.

Uber's initial success in San Francisco was a key stepping stone in their rise to national prominence. One wonders whether Uber would exist today if even a single SF cab company had served customers reliably.

---

[0] p.s. The credit card thing. Every SF taxi rider has had this happen: the driver has the credit card reader in the car, staring you in the face, but the driver says: "Sorry, it's not working." Now, if this happened once in a while, you might believe it. But it happens every single time. Is it really the case that every single credit card reader in the SF taxicab fleet is out of order 100% of the time despite a good-faith effort to keep them operational? Or is it much more likely that the driver is trying to cheat on his taxes, which is made easier if he forces all his customers to tip him in cash? Or else the taxi company is cheating on its obligations to its drivers by keeping tips paid on credit cards for itself? And now this confederacy of cheats wants the government to protect them from competition? Do you feel sympathy for this band of cynical hypocrites?

Why should Mozilla continue developing Rust?

The title of this post is not (entirely) rhetorical. I'm puzzled by Mozilla's decision to burn several of its full-time engineers' cycles on the Rust programming language. Maybe someone can enlighten me. I offer the following observations with all due humility.

As a problem of engineering management, programming language development can be analyzed in terms of cost, risk, and return. I do not see how Rust makes sense for Mozilla along any of these dimensions.

Cost: Getting a new programming language off the ground only requires a handful of people, if you have the right people. For giant technology corporations, which have thousands of engineers on staff, that's a tiny relative overhead, well within the margin of a sensible research budget. By contrast, for Mozilla, a lean, low-headcount organization, it's a major opportunity cost. A handful of Mozilla's top full-time developers represents a big bite out of Mozilla's engineering throughput.

Risk: The baseline probability that a new programming language will become popular is low. It is even lower when you cannot use the popularity of a platform to strong-arm external developers into adopting it (the existence of such a platform is the story behind the success of JavaScript, Objective-C, and C#). Mozilla has no platform that it can use to strong-arm anyone into using Rust, nor does it even wish that it did. And when a programming language does not become popular, its few sponsors end up doing all the heavy lifting for it indefinitely. The language also remains a laggard behind comparable languages in libraries and tools. Mozilla isn't well positioned to mitigate this risk or to deal with its downside.

Return: The returns to adopting a better language are roughly proportional to the developer-hours spent writing code with it. As a organization with both low headcount and few projects, Mozilla will have difficulty recouping its investment:

• Consider Google and Go. Suppose 5 core Go developers work on the language for 2 years; this costs about 5 * 2000 * 2 = 20k developer-hours. This investment can be recouped by saving 1000 Google engineers an average of 2 hours per week for just 10 weeks, or equivalently by providing a quality increase equivalent to 2 more hours per week of design, testing, etc. Viewed this way, Go has a plausibly huge upside for Google. But Mozilla will never have 1000 full-time engineers using Rust; it has a few hundred employees roughly 1k employees total, not all of whom are engineers, and few of whom might be writing Rust full-time in the foreseeable future. Perform a similar calculation using any remotely plausible numbers, and the case for the Rust's return to Mozilla comes out pretty weak. Either Rust will take far too long to pay off, or it must be orders of magnitude better than the next-best alternative (and the latter seems unlikely).
• It is easier to realize the benefits of a better language when starting a new, greenfield project than when rewriting legacy code. In fact, when rewriting legacy code, developers incur more costs than benefits until a tipping point is reached. Google, Microsoft, or Amazon, to take a few examples, each start dozens of substantial projects and hundreds of little projects per year. At such companies, it is conceivable that, by winning over new projects at their inception, a new language could achieve an annual adoption rate of hundreds of developers per year. By contrast, Mozilla can't start more than few projects per year without losing focus. So the rate at which Mozilla can realize the benefit of a new language is constrained even further than their headcount would suggest. And indeed the core developers of Firefox, Mozilla's biggest project by far, will not benefit from Rust anytime soon.

Now, in spite of all the above, one might disregard risk, cost, and return if one had a truly unique opportunity. If you're the only organization in the world that might plausibly deliver on some project of unique importance, then it might be worth rolling the dice: "We have to save the world from X; nobody else will." Perhaps reasonable people can disagree, but I can't see how Rust qualifies. As a "fast, safe, concurrent, static, systems language", Rust is quite comparable to Go or D; even if one grants, for argument's sake, that it's better than those languages in some ways, it occupies a similar space. So Mozilla's opportunity is simply that it has the money and freedom to pay a few software engineers to work on Rust full-time. The "unique opportunity" rationale for overriding cost/risk/reward does not seem to apply.

Alternatively, one might rebut all of the above calculations by saying that Rust is a research project, not strictly an engineering project. But it is a curious research project. Research justifies its existence by hoping to advance computer science in a fundamental way. Rust conversely aims to minimize novelty, as its FAQ states: "Old, established techniques are better [than novelty]". I cannot think of a fundamental computer science question that is investigated by Rust's development. It is research only in the sense that it is a speculative engineering project without an obvious short-term payoff.

Furthermore, all of the above must be considered in light of Mozilla's current position. Mozilla is fighting to maintain relevance in a world where all the major desktop and mobile operating system vendors are shipping high-quality web browsers. And Mozilla is fighting with far less resources than its competitors. Mozilla punches far above its weight, and for that it should be proud, but I doubt that it has the luxury of diffusing its focus at this time. Mozilla is not like the IBM or Microsoft of old; it cannot plow huge surplus profit margins into basic research without subtracting needed resources from product engineering.

Therefore, in my opinion, Mozilla should spin out the Rust project to community maintenance, and try hard to convince its paid engineers to work on something with a cost/risk/return/opportunity profile better suited to Mozilla's place in the world. Rust developers are experts in programming language implementation, so ES6, asmjs, or Firefox and Firefox OS developer tools all seem like promising alternatives.

Or, if Mozilla really believes that reducing dependence on C++ is a major strategic priority, these developers might investigate rewriting Firefox components in another language — one for which they won't have to do most of the heavy lifting in toolchain and library development. Delegating language development to someone else makes far more sense than doing it in-house. By way of analogy, Mozilla does not try to innovate in datacenter design. To do so would be ludicrous, as Google, Amazon, Facebook, etc. are all far better-positioned to do so. Instead, Mozilla is happy to reap the fruits of robust multi-party competition in datacenters. They should take a similar approach to programming language design and implementation (outside of the web client platform, where they are unavoidably a key contributor).

A third alternative might be to find a deep-pocketed external sponsor to take on the heavy lifting for Rust. Samsung made some noise a while ago about sponsoring Rust; but how many active Rust contributors come out of Samsung's headcount rather than Mozilla's? I believe that the key cost of Rust development for Mozilla is the opportunity cost of expert Mozilla developers' labor. Until Samsung or some comparable organization credibly commits not only financial sponsorship but long-term ownership and headcount to Rust development, its costs will not be truly offloaded from Mozilla.

p.s. I say all of the above as someone who likes what I've seen of Rust, purely as a programming language. I would be happy if some organization took up the Rust banner and carried it forward. But I do not think Mozilla is a logical primary sponsor for it.

How much education do computing innovators usually have?

Today brings us this tweet from Neil deGrasse Tyson:

Getting straight "A"s does not guarantee success, but plenty of evidence shows that not getting "A"s doesn't preclude it.

This is fine, as far as it goes. Having an imperfect academic record does not doom you to a life of mediocrity. And it is crucially important for anyone, no matter how you are assessed by the educational system, to continue striving towards a meaningful and successful life, however you choose to define that.

However, I foolishly clicked on the thread to see what people's reactions were, and came upon someone saying this:

@neiltyson you realize Everything you pretty much use on the Internet were created by people who never got grades at all.

Ugh. Sometimes I read something so wrong that I'm pretty much compelled to spend a couple of hours of my life disproving it. (Of course, this particular semi-grammatical tweet is so inconsequential that it's comical that it affects me at all, but it represents a much broader current of popular thought about the world of computing.)

The Internet is an outgrowth of computing and communications technology. Below, I have listed the educational attainment of a few dozen people who, it seems to me, created things that are important to the Internet as it exists today.

The list is, one might say, not drawn to scale. Everyone just gets a bullet, even though some are giants who left behind towering contributions that will be studied centuries from now, and others merely made something that seems to be widely used or influential today. I've also used a fairly subjective criterion for selecting the people: these are simply names that occurred to me in about ten minutes of thinking about the technology that's literally sitting in front of my face as I write this. Nevertheless, I think it is a decent sample, and although it can be improved around the edges, I doubt that you will be able to supply thirty more names that are better candidates and whose educations differ dramatically from these people's.

(I have used "bac." to indicate any 4-year undergraduate degree (B.A., B.S., or equivalent) since I don't find it useful, for this purpose, to distinguish among them.)

Foundations of computing

• Alan Turing: Cambridge bac., Ph.D.
• Alonzo Church: Princeton bac., Ph.D.

Networking and secure communication

• Vint Cerf: UCLA Ph.D.
• Bob Kahn: Princeton Ph.D.
• Tim Berners-Lee: Oxford bac.
• Robert Metcalfe: MIT bac., Harvard Ph.D.
• David Boggs: Princeton bac., Stanford Ph.D.
• Ron Rivest: Yale bac., Stanford Ph.D.
• Adi Shamir: Tel Aviv bac., Weizmann Institute Ph.D.
• Leonard Adleman: UC Berkeley bac., UC Berkeley Ph.D.
• Whitfield Diffie: MIT bac., Stanford Ph.D. dropout.
• Martin Hellman: NYU bac., Stanford Ph.D.
• Ralph Merkle: UC Berkeley bac./M.S., Stanford Ph.D.

Operating systems and programming languages

• Ken Thompson: Berkeley bac., M.S.
• Dennis Ritchie: Harvard bac., Ph.D.
• Rob Pike: unknown, possibly no college. Caltech (according to comment; bac.?)
• Doug Engelbart: UC Berkeley Ph.D.
• John McCarthy: Caltech bac.; Princeton Ph.D.
• Kirsten Nygaard: U. of Oslo M.S.
• Ole-Johan Dahl: U. of Oslo M.S.
• Alan Kay: U. of Utah Ph.D.
• Ivan Sutherland: CMU bac., Caltech M.S., MIT Ph.D.
• Dan Ingalls: Harvard bac., Stanford M.S., Ph.D. dropout.
• James Gosling: U. of Calgary bac., CMU Ph.D.
• Richard Stallman: MIT bac., MIT Ph.D. dropout.
• Bjarne Stroustrup: Aarhus M.S., Cambridge Ph.D.
• Linus Torvalds: U. of Helsinki M.S.
• Alan Cox: Swansea University dropout.
• Theodore Ts'o: MIT bac.
• Brendan Eich: Santa Clara bac., UIUC M.S.
• Guido Van Rossum: U. of Amsterdam bac./M.S.
• Yukihiro Matsumoto: U. of Tsukuba bac.

Founders of important technology companies

• Gordon Moore: UC Berkeley bac., Caltech Ph.D.
• Steve Wozniak: UC Berkeley dropout.
• Steve Jobs: Reed College dropout.
• Bill Gates: Harvard dropout.
• John Warnock (Adobe): U. of Utah Ph.D.
• Charles Gesche (Adobe): CMU Ph.D.
• Marc Andreessen: UIUC bac.
• Larry Page: Michigan bac.; Stanford Ph.D. dropout.
• Sergey Brin: U. of Maryland bac.; Stanford Ph.D. dropout.
• Jeff Bezos: Princeton bac.
• Jerry Yang: Stanford bac./M.S.
• David Filo: Tulane bac., Stanford M.S.
• Diane Greene (VMWare): MIT M.S., Berkeley M.S.
• Mendel Rosenblum (VMWare): Berkeley Ph.D.
• Ed Bugnion (VMWare): Stanford M.S., Stanford Ph.D. dropout.
• Evan Williams: U. of Nebraska dropout.
• Mark Zuckerberg: Harvard dropout.

Assorted hackers

• Jamie Zawinski: college dropout, institution unknown.
• John Carmack: U. of Missouri dropout.
• Michael Abrash: U. of Pennsylvania Ph.D. dropout.
• Jeff Dean: U. of Washington Ph.D.
• Sanjay Ghemawat: Cornell bac, MIT Ph.D.
• Russ Cox: Harvard bac., MIT Ph.D.
• Paul Buchheit: Case Western Reserve bac.
• Lars Bak: Aarhus bac./M.A.
• Paul Graham: Cornell bac.; Harvard Ph.D.

One thing you will notice right away is that there are lots of Ph.D.s and Master's degrees, generally from highly-ranked universities. A lot. Obviously far more than the general population, and probably more than leaders in most other industries. The notion that invention and innovation in computing are done mostly by scruffy, uncredentialed twenty-year-olds is a myth. Yes, you can see a substantial minority of college dropouts. But it should not surprise anyone that the major achievements in a field based on technological advancement are often created by people who studied technology at an advanced level.

Among those who do not have Ph.D.s or Master's degrees, many attended some grad school (usually meaning they finished an undergrad degree), and almost all attended some college. Even John Carmack attended some college, and he is maybe the best example alive of a hacker who created an entire industry with his mind simply by sitting in front of a personal computer and writing code nobody else could write.

That said, there is some truth to the notion that (college) dropouts succeed in the computing industry at a higher rate than in the general population.

As far as I can tell, most of the successful college dropouts quickly found something else to do that was unique and important at that moment. Based on this list, one sign that dropping out might be a good idea, rather than a copout, is if the thing you do instead requires much harder work than what your college classes require.

Also, I think it is worth observing that the people behind major technological advances tend to have advanced educations. By contrast, successful dropouts tend to be people who made their mark by designing and marketing commercial products using widely available technology, or with modest and incremental technological innovation.

There are a few anomalies: Carmack, Wozniak, and perhaps Pike (UPDATE: see comments) were supremely technical and don't have college degrees.[0] On the other hand, they are among the most brilliant hackers who ever lived. I think that, for a young hacker, betting that you are the next Carmack is an extreme long shot (although if you really believe that this possibility is substantiated by the evidence of your work quality, then go for it).

Why does this matter? Why did I write this? Mostly because when someone's wrong, it annoys me. But here are two other reasons.

First, people should be giving credit where credit is due. The vast majority of technological innovation in our field comes from people who participated, in some fashion, in the higher education system — in most cases, the elite part (broadly construed [1]) of the North American university system. The stuff you use on the Internet was emphatically not mostly created by people who "never got grades at all". Don't propagate this myth.

Second, it would be a mistake to generalize from the small minority of successful dropouts to the conclusion that a college education is not meaningful, and that skipping it would therefore be a good idea for you. Dropping out of college is a terrible idea for most people. I think it is probably a bad idea all the way up to, maybe, the top 2-3 percentiles of ability or motivation, and even for those people it is only a good idea in particular circumstances.

The past decade has seen outsized financial success accrue to a few consumer Internet businesses, like Facebook and Twitter, which were founded by dropouts. This has led to a certain current in our cultural imagination that substitutes a remote lottery-ticket shot at getting rich quick on a startup for the dream of broad-based prosperity based on equitable access to high-quality education, health care, and housing. Actually, both components are necessary. Big prizes do seem to drive risk-taking entrepreneurship. But the foundations of an innovative technological economy are built by people who mostly study hard, get good grades, and pursue advanced study in their field. The exceptions are just that — exceptions — and the sound way to get more innovation is to instill in more people the habit of studying hard, and to give more people the opportunity to pursue advanced education.

---

[0] Actually, it's not clear to me whether Rob Pike attended no college, or whether he did, and for characteristically eccentric reasons refuses to reveal it to the public. But until I find more information, I'm assuming he didn't. (UPDATE: If the anonymous comment below is to be believed, Pike went to Caltech, degree unspecified.)

[1] By "elite" here I mean roughly top-100-ranked four-year institutions. This may strike some readers as an overly broad class to label "elite", but the universe of higher education includes a much broader class still, including community colleges, professional and vocational schools, etc., and if you count heads in higher education then it is still a minority of students who go to top-100 4-year schools.