tag:blogger.com,1999:blog-56210242024-03-07T11:05:00.552-08:00The Abstract Factoryin a city of the future
it is difficult to concentrate<br>
(<a href="https://keunwoo.com/">keunwoo</a>'s blog)Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.comBlogger599125tag:blogger.com,1999:blog-5621024.post-31765157981838374412022-05-03T06:00:00.025-07:002022-05-03T06:00:00.190-07:00Redirect<p>Not sure if anybody reads stuff here anymore, but I'm posting newer writing on <a href="https://keunwoo.com/">my homepage</a> now; here's a <a href="https://keunwoo.com/feed.atom">direct link to the site feed</a>.</p>
<p>I guess that means this blog is now <em>officially</em> retired? Thanks to everyone who visited over the past, uh, <a href="https://abstractfactory.blogspot.com/2003/07/my-flimsy-rationale.html">almost 19 years</a>? It's hard to believe I've wrtten almost 600 posts (598, including this one). It's been a fun hobby.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-42086911513415809062021-02-06T12:00:00.002-08:002021-02-06T12:06:17.142-08:00Freedom of speech, and how to reason productively about it<p>Most people develop opinions about how speech should be restricted via <i>a priori</i> reasoning from abstract principles, often absorbed from primary school civics lessons or other tribal recitations (for example, these days, social media memes).</p>
<p>But in a consequentialist, utilitarian framework, how discourse should be regulated depends on several empirical questions:</p>
<ul>
<li>Speech is a physical reality; the speech that occurs in a given medium can be measured.</li>
<li>The beliefs, behaviors, and harms that a system of speech engenders are also physical realities, and also to some extent measurable.</li>
<li>The effects of a given mode of speech <em>regulation</em> are also measurable realities.</li>
</ul>
<p>To put it another way, media are systems with particular mechanics, like games; Twitter is a different game, for example, than Reddit, and both are different from digital journalism or cable news. The mechanics of a medium drive higher-level emergent dynamics, which (for nontrivial systems) can only be studied empirically. The proper instruments of study cannot be drawn from the toolkit of <i>a priori</i> philosophy alone; they require the methods of science and engineering: experimentation, simulation, modeling, comparison of modeling predictions against empirical data.</p>
<p>I posit that any practicing computer game designer could easily design a "speech game" where bad speech totally drives out good. In fact, this would be so easy that the only design challenge would be making it fun enough that anyone would want to play. As a player of that game, you would be a fool to embrace the strategy that "the remedy to be applied is more speech" (quoting Brandeis); you would simply be crushed; your adversaries would laugh at your naivete. I hope this idea seems obvious to you. But are you certain that we aren't all playing such a game in one or more spheres of real life?</p>
<p>It is sad to me that so much effort is spent (wasted!) arguing in totally unproductive circles about some aspect of free speech ("censorship", "cancel culture", "deplatforming", etc.), and so little effort is spent understanding empirically the connection between mechanics and dynamics.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-8337460127714120232018-05-15T07:00:00.000-07:002018-05-15T11:08:33.513-07:00On the Duplex demo<p><i class="disclaimer">Disclaimer: I worked for Google long ago, and I may work there again someday. I had nothing to do with Duplex.</i></p>
<p>The Google Duplex demo has caused some fairly heated & widespread reactions; to paraphrase, "The person on the other end doesn't know or consent to talking to a robot! This is a deceptive violation of their rights!" I think that I understand, a little, why people react this way, but on balance I find it logically ridiculous. I am also convinced that it will seem both logically <em>and emotionally</em> ridiculous to most people within a generation.</p>
<p>As someone in my 40s, I understand the relationship that 20th century people once had with their phones. I am old enough to remember "reach out and touch someone" being a thing that real people felt was the primary purpose of telephonic communication. However, over the course of my adult life, this use case has been utterly swamped by the rise of automated or semi-automated telephonic processes, constructed by <a href="http://bactra.org/weblog/699.html">immense and remorseless engines of bureaucratic modernity</a>, that use the telephone as an electronic siphon to suck value out of my time: <a href="http://thehill.com/policy/technology/286628-entire-federal-government-exempt-from-robocall-laws-fcc-rules">legally protected political robo-calling</a>, <a href="https://www.washingtonpost.com/lifestyle/magazine/how-robo-call-moguls-outwitted-the-government-and-completely-wrecked-the-do-not-call-list/2018/01/09/52c769b6-df7a-11e7-bbd0-9dfb2e37492a_story.html">flagrantly illegal commercial robo-calling</a>, telemarketing driven by script-reading call center employees (who, in this role, are functionally biological components of a machine, not autonomous individuals), and a constant nagging flood of scam hangup calls.</p>
<p>Conversely, nearly all outbound phone calls that I make, except to close friends and family, now involve navigation through a robotic phone tree. In other words, I am interacting with a succession of artificial voices for many minutes before a usually-brief chat with a human being.</p>
<p>In other words, for any human being less than about 45 years of age, nearly all telephonic interactions in their adult life have been to a large degree robotic. It is hard to get up in arms because the robots are going to be slightly more fluent in the future. They have been getting more fluent all my life (for example, many phone systems can now recognize numbers that are spoken rather than dialed on a touch-tone pad). I don't care. Deep down, you probably don't either.</p>
<p>Search your feelings; you know it to be true. How many times in your life have you picked up the phone to a telemarketing call and thought to yourself, "Oh thank goodness, I am super glad to have been interrupted in this fashion because it's a <em>human being</em> talking to me rather than a robot? My heart brims over with joy!" None. Zero times. You have never thought this.</p>
<p>On the other hand, suppose you received a call that said: "Hey, just wanted to let you know that a recently deceased distant relative left you a one million dollar inheritance; a check and a letter with details will be arriving in your mailbox today. You don't have to do anything else but cash the check, thanks, take it easy!" You would not care whether the voice was a robot or a human being. You would be skeptical, but when you got the check and the letter, which said, "By the way, a robotic call was placed to your phone number earlier today to inform you that this letter would arrive so you wouldn't miss it," you would not think to yourself, "OMG, I feel so unbelievably violated because that voice was a robot! Fuck this stupid million dollar check and fuck the horse that it rode in on!"</p>
<p>The moral valence of a phone call is determined by the value that the participating parties get out of the call, not by whether one or both parties on the call are mediated by machines that are slightly more adept than the machines which existed in 1992.</p>
<p>Lastly, if the above doesn't convince you, here are two more quick reasons that the future will welcome talking bots:</p>
<ul>
<li>People under 30 today ("millennials" or whatever you want to call them) <em>hate</em> making voice calls. They will probably welcome any opportunity to delegate this stupid chore.</li>
<li>People under 15 today will grow up taking bots for granted; for example, textual chatbots, or other types of bots in online games that they play. They will feel no horror at the idea that spoken-word bots can have warm, engaging voices.</li>
</ul>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com1tag:blogger.com,1999:blog-5621024.post-53833989657151145472018-03-20T14:37:00.000-07:002018-03-20T14:39:31.503-07:00How resistant is Bitcoin to government regulation?<p><i class="disclaimer">Excavated from the drafts folder. I composed this in December 2017 (although I can't prove it) and I am publishing it now because <a href="https://www.theguardian.com/technology/2018/mar/20/child-abuse-imagery-bitcoin-blockchain-illegal-content">recent news</a> has rendered the postscript timely.</i></p>
<p>Bitcoin depends on the following three types of infrastructure:</p>
<ul>
<li>Semiconductor fabrication plants.</li>
<li>Electric power plants.</li>
<li>Transoceanic fiber-optic Internet cables.</li>
</ul>
<p>It depends on the first two of these to an unusual extent; the third simply comes along for the ride because Bitcoin is a global Internet-connected system.</p>
<p>You can't access these things, or the stuff that comes out of them, when governments really don't want you to, because they are big, expensive works of physical infrastructure. A government, or coalition of governments, with sufficient motivation and resources could introduce regulation upstream or downstream of any of these choke points that simply makes further Bitcoin mining non-viable.</p>
<p>"The Internet interprets censorship as damage, and routes around it" is true only up to a point. It is instructive to compare Bitcoin with child porn, a category of online activity that most governments are already highly motivated to stop. Governments have mostly succeeded in eliminating it from the non-darknet parts of the Internet; even sites like 4chan, which revel in their own transgressiveness, aggressively police child porn, because these sites run on servers that sit in datacenters in the physical world that are governed by laws just like everything else. Those servers are hooked up to the power grid and connected to the Internet by physical wires that belong to some utility. To maintain these connections, money changes hands, and a paper trail is generated that ultimately leads to the server operators. If you decide to be a badass rebel and distribute child porn under these conditions, you are looking forward to jail time.</p>
<p>It is true that child porn exchange still occurs online, but it occurs mostly on obscure systems that are specifically designed to be censorship-resistant at the expense of widespread availability. Bitcoin could be shut down or marginalized just as easily (where "marginalized" simply means that it is used as a transaction processing system only in highly unusual circumstances, rather than as a pillar of the economy).</p>
<p>In fact, it could be shut down much more easily. The resource cost to produce and distribute child pornography is nearly fixed: the size of a collection of digital imagery is essentially constant, and can be produced and distributed with extremely modest equipment. Basically, individual criminals can sustain a cottage industry indefinitely. By contrast, the exponentially increasing computational power demands of Bitcoin make it particularly vulnerable to regulation. If you cannot get your hands on an ever-increasing supply of semiconductor chips and electricity, you cannot mine Bitcoin, at least not at the scale that today's largest miners operate. Bitcoin mining is not a cottage industry; it is a large-scale industrial process, with datacenters as the factories. Large capital equipment is inherently easy to regulate. And replacing all of today's gigantic mining operators with small-scale, individual miners suffers from economic and engineering problems similar to those that you'd confront if you tried to replace a Google datacenter with ten million mobile phones (let alone trying to do that while dodging mining regulations).</p>
<p>Bitcoin has not been regulated because governments mostly do not care enough to regulate it (yet). The starry-eyed anarchist fever dreams of the most anti-statist libertarian early Bitcoin proponents were always total fantasies. If Bitcoin does not fail completely, then either it will be tamed into just another boring part of the existing world financial system, or else governments will wake up and cripple it.</p>
<hr/>
<p>p.s. Incidentally, since <a href="https://bitcoin.stackexchange.com/questions/39347/how-to-store-data-on-the-blockchain">it is possible to encode data on the Bitcoin blockchain</a>, an attacker with sufficient motivation and resources could save an instance of child pornography on the blockchain. Since every node maintains a copy of the entire blockchain, and Bitcoin lacks the ability to erase transactions, the entire Bitcoin network would be transformed at a single stroke into a child porn distribution system, rendering all Bitcoin node operators criminals. This might cost an incredible amount of money — millions of dollars to stash a single image of a few KB — but once done, it would irrevocably taint Bitcoin forever. This is an aspect of Bitcoin that is obvious on inspection, yet almost never discussed.</p>
<p>p.p.s. <a href="https://www.theguardian.com/technology/2018/mar/20/child-abuse-imagery-bitcoin-blockchain-illegal-content">It happened.</a></p>
<hr/>
<p><i>This post closed to comments because Bitcoin attracts an unusually high ratio of vocal kooks.</i></p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.comtag:blogger.com,1999:blog-5621024.post-9933757365600603552018-01-19T12:48:00.001-08:002018-01-19T15:03:09.753-08:00The evolution of players in the American constitutional game<p>Another looming threat of Federal government shutdown prompts <a href="http://timothyblee.com/2018/01/19/our-constitutional-system-is-broken-and-we-should-fix-it/">Timothy B. Lee to concur</a> with the <a href="https://www.vox.com/2015/3/2/8120063/american-democracy-doomed">Yglesian view</a> that American constitutional democracy is doomed — or, at least, in need of major structural changes:</p>
<blockquote><p>The hour-by-hour style of conventional news coverage tends to obscure the big picture: the perpetual crises the US government has suffered over the last decade are a symptom of America’s deeply flawed constitutional system. This isn’t a new insight on my part. You can read Matt Yglesias’s <a href="https://www.vox.com/2015/3/2/8120063/american-democracy-doomed">classic 2015 write-up</a> of the argument, which in turn draws on a large body of political science literature.</p>
<p>The basic issue is that the American system of checks and balances was designed for a nation without ideologically polarized parties. . . . The problem is compounded by the fact that it’s so hard to remove a bad president from office.</p>
</blockquote>
<p>One seemingly-strong rebuttal to these arguments is that American democracy has survived for a long time, so probably the system is fine. The last Civil War veterans <a href="https://en.wikipedia.org/wiki/Last_surviving_United_States_war_veterans">died in the 1950s</a>; unlike most nations on Earth, America has no living citizens with a firsthand memory of existential risk to its constitution. This history of recent stability is probably the main reason that most Americans instinctively reject arguments, no matter how logically sound, that America's constitutional system is fundamentally flawed.</p>
<p>But this rebuttal has less force than it seems. The American constitution is a game, and political actors are players. When people play a game, it takes time to explore the available strategies. Partly this is because, even for relatively simple games, the space of strategies can be immense, requiring time to explore; partly this is because external forces, such as social norms, may prevent players from using optimal strategies initially. However, once a stronger strategy is discovered, it is difficult to stuff the genie back into the bottle; rewards accumulate for those who ruthlessly exploit the most lucrative methods of play, and those who use less optimal strategies are driven out.</o>
<p>A nice illustration of this dynamic can be seen in Google's training of its chess-playing program <a href="https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess">AlphaZero chess</a> — openings such as the French Defense and Caro-Kann Defense appeared strong to the program while it was training itself, but eventually it abandons these almost entirely in favor of strategies that are inherently stronger, such as the English Opening.</p>
<p class="illustration"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghdWFtH0dlcwMH5aauoh9EwWXiVxnqkNwY0qq4VUyDtD-K_kjFE0Mc6KEW_UWwQF3P2xxP7gAdxj1pculadFPqEru5XuSQl51JfByUgxZtX71ZXkkes67mUMzYDy91f4N7H1DjXg/s1600/Screen+Shot+2018-01-19+at+11.42.05+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="976" data-original-width="756" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghdWFtH0dlcwMH5aauoh9EwWXiVxnqkNwY0qq4VUyDtD-K_kjFE0Mc6KEW_UWwQF3P2xxP7gAdxj1pculadFPqEru5XuSQl51JfByUgxZtX71ZXkkes67mUMzYDy91f4N7H1DjXg/s640/Screen+Shot+2018-01-19+at+11.42.05+AM.png" width="496" /></a></p>
<p>In other words, every nontrivial game is also an optimization process, where the set of players explores the landscape of available strategies over time. In such a process, it is entirely possible for the most prevalent strategies to shift dramatically and even discontinuously; the past is not necessarily a guide to the future.</p>
<p>If you had been taught to play chess by the version of AlphaZero that existed two hours into its training, you might have learned that the French Defense was the "normal" way to win at chess; if I showed you a single later game using the Queens Gambit, you might view that as a temporary aberration. You would be wrong. Likewise, if you came of age in the 20th century, you might view electoral politics as practiced back then as "normal", and the current era, where the government walks up to the brink of shutdown or debt ceiling default every year or two, as a temporary aberration. I suppose it's possible, but the persistence of this observed behavior suggests that it is simply a stronger way for political parties to play the game.</p>
<p>Lee and Yglesias, and the political scientists whose work they draw upon, point to increasing partisan polarization as the cause of the shift in American electoral politics in recent decades. This is fine as far as it goes, but it is important also to realize that the constitutional game itself has always had, encoded within its rules, the possibility of the current configuration of power. The numerous veto points of the American system have always had the potential to be used to hold one policy objective or another hostage. The division of electoral authority between the President and Congress has always had the potential to allow both to dodge accountability for outcomes. It has always been possible for ideologically united parties to gridlock against each other. Accidental features of the American political landscape prevented these strategies from being exploited, but now the players are playing at this level, and thereby evidently beating those who would play differently. There are now only two possible avenues to change: either some novel strategy emerges to beat these strategies, or the rules must be revised.</p>
<hr/>
<p>Incidentally, this general idea — that a ruleset and the actors who interact with it coevolve — recurs across many fields:</p>
<ul>
<li>It is at the heart of the concept of regulatory capture in public choice theory.</li>
<li>Designers of multiplayer computer games understand that a game's rules must be periodically patched for "balance" as players discover dominant strategies which render the game trivial or un-fun.</li>
<li>Security researchers are, of course, dreadfully familiar with the fact that every nontrivial system has unknown exploitable vulnerabilities. Once a vulnerability has been found, they would laugh at the notion that you could simply convince attackers not to behave that way, which is the analogue of hoping that American political parties won't use the toxic tactics currently available to them.</li>
</ul>
<p>You'll find many other places to apply this concept once you have it in your toolkit.</p>
<p>(I've actually been meaning to write a longer essay on this idea and its implications for a while now, but haven't done so for the usual reasons, so this post will have to do for now. Well, this, and <a href="https://pinboard.in/search/u:absfac?query=coevolution-of-rulesets-and-their-adversaries">my Pinboard tag</a> on the subject)</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com2tag:blogger.com,1999:blog-5621024.post-43589491152695975762017-12-04T21:53:00.001-08:002017-12-05T15:14:51.095-08:00Two ways ISPs can do content-based filtering of encrypted traffic<p>Vaguely a propos of the revived net neutrality debate, a while back I saw someone on Twitter claim that it is technically not possible for ISPs to do content-based (as opposed to destination-based) filtering of SSL traffic. This statement seems initially plausible, but is false. I can think of two technical mechanisms to do content-based filtering.</p>
<p>First, <a href="https://beautyburst.github.io/">it is possible to identify encrypted content via traffic analysis</a>. ISPs could compile a database of traffic signatures which they wish to throttle (e.g., for videos that are available from their own video streaming services) and throttle any traffic matching that signature.</p>
<p>Second, ISPs can require that users add a trusted SSL root cert owned by the ISP, thus allowing the ISP to man-in-the-middle all SSL traffic. Obviously, content-based filtering then becomes trivial.</p>
<p>You might object that this second measure would be unacceptably onerous, and would be rejected by the market. In the near future, a middle-class American family of four may own ten or twenty Internet-connected devices, running a half-dozen operating systems, and demanding that users install a root cert on all of them would cause unbelievable inconvenience and outcry. This might be true, but without even trying very hard I can think of numerous ways that ISPs could try to acclimate users to this bitter pill:</p>
<ul>
<li>Of course, the software package would be named something relatively innocuous, like "Comcast Internet Security Accelerator" or some such nonsense.</li>
<li>The MITM cert might only be required for devices that wish to access the "fast lane" — in other words, the ISP would simply throttle any SSL connection that it does not MITM. All the household's devices would be functional even if you didn't jump through this hoop, but the ones that need the fastest connections — say, the PC that streams HD VR video — would require the MITM cert installation.</li>
<li>The ISP could distribute web browsers and other apps that embed the trusted cert — for example, Comcast could provide a custom build of Chromium — and require their use for the "fast lane". Again, you wouldn't need this app for casual web browsing, only for sites that are sensitive to speed.</li>
<li>ISPs could strike distribution deals with mobile carriers to install root certs on phones. The most obnoxious way to do this would be to ship the phone's ROM with the MITM cert baked in; this would probably cause massive outcry, akin to the <a href="https://www.us-cert.gov/ncas/current-activity/2015/11/24/Dell-Computers-Contain-CA-Root-Certificate-Vulnerability">eDellRoot debacle</a>. A sneakier way to do it would be to ship a carrier-branded app that has the ability to update the trusted cert store (by itself this is arguably innocuous), along with an ISP-branded app that (a) nags the user for consent when it detects that the phone is on the ISP's network, something like "Welcome to Comcast! Do you want to enable Comcast Fast Lane[TM]?", and (b) when the user "consents", installs the MITM root cert by delegating to the carrier's app.</li>
<li>ISPs could embed a web browser connected to a virtualized display in the set-top box. The set-top box, of course, would already trust the MITM cert. Then, instead of browsing directly to https://www.youtube.com/ or whatever, you would first browse to http://xfinity.local/, which would present you with a web app that is itself a browser running via remote desktop protocol. Then you would type https://www.youtube.com/ into the address bar of this web browser. The ISP could even "helpfully" set up its DNS to perform this redirection automatically (if you type youtube.com without the https).</li>
</ul>
<p>These are just the ideas that occur to me in about twenty minutes of thinking. If these seem farfetched to you, there may be other ways to boil this frog. Companies can be rather creative when there are billions of dollars of rents to be extracted. The result does not have to be low-cost or seamless for the user; local broadband ISPs in the United States are subject to practically no competition and whatever they implement just has to be marginally less painful than waiting for your content to download over the cellular network.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com1tag:blogger.com,1999:blog-5621024.post-72760475886757506962017-09-29T07:00:00.000-07:002017-10-05T10:09:21.654-07:00Tentpole sponsors: an idea for improving paid service virality<p>Ad-supported communication platforms like Facebook have many structural advantages over hypothetical competitors that charge users money directly. One advantage is that a purely ad-supported service can spread virally, from user to user, at a vastly greater rate than a service that demands direct monetary payment.</p>
<p>For most users, the unpredictable, frequently unmeasurable harms of losing privacy and control over their social identity are less tangible than the direct time and money cost of signing up for a paid software subscription <a href="#20170929_footnote0">[0]</a>. Thus free services which strip-mine your privacy and lock you into their prison spread like wildfire, while paid services that respect their users barely get off the ground. It seems that every large social networking service on the Internet has been hammered on the anvil of this seemingly inescapable logic and beaten into a Facebook-like shape.</p>
<p>However, user preferences vary. One can conjecture that within any social network subgraph of size N (for some N), there exists at least one user who cares an unusual amount about privacy and control. This user might be willing to subsidize a large subset of their local subgraph. Let R be the ratio of the local neighborhood of size N that such a user is willing to subsidize.</p>
<p>If N and R have the right values, a possible hack for the virality problem is to charge money to these special users — call them "tentpole users" — and allow them to sponsor the addition and ongoing use of the users around them. Most users will not be tentpoles; but given enough poles, positioned appropriately, the tent may be lifted over the entire addressable user population.</p>
<p>In the most basic form, you can imagine that a paid subscription gives every user a certain number of tokens, which they can use to sponsor accounts for their friends and family. When a new user is invited, some tokens would be allocated to them — one to support that user, and optionally some extra tokens gifted so that they could invite more users in turn. A non-sponsor user who wants additional invitations beyond their starter set would purchase more, thereby becoming a sponsor, or ask their network for some spare tokens. Sponsorship would be fungible — that is, users would be able to change their sponsor at any time — but every user would be either a sponsor or a beneficiary or both.</p>
<p>In principle, with proper tuning, most users could be beneficiaries, and pay nothing. A service engineered this way would be closer in virality to an ad-supported one. (It's still not quite as viral; for one thing, there is still some real friction at the edge of the "sponsorship radius", the distance from a sponsor at which users run out of tokens for further invitations. This needs further thought.)</p>
<p>Another model would allow all users to join free of charge, but grant additional privileges to sponsored users. This works, economically, as long as the aggregate cost of free-riding users is less than the total revenue from sponsors. This "tentpole freemium" model resembles an <a href="https://en.wikipedia.org/wiki/Freemium">ordinary freemium</a> model (where only the sponsors themselves pay <a href="#20170929_footnote1">[1]</a>); arguably it is simply a freemium model where one of the premium benefits is improved amenities for one's contacts.</p>
<p>When I mentioned these ideas to a colleague a few months ago, he immediately pointed out that tentpoling leads to a situation where sponsored users are socially indebted to their sponsors. This has at least two effects. First, debt potentially causes social awkwardness, and this risk must be navigated (c.f. <a href="https://press.princeton.edu/titles/8023.html">V. A. Zelizer</a>). Second, users may feel a sense of precarity because sponsorship could end (for example, if their sponsor cancels their subscription), and thus would be reluctant to adopt the platform. These are definitely challenges, but it may be possible to overcome them.</p>
<p>Social awkwardness may be amenable to psychological hacks which obfuscate the transactionality of the interaction. To invent a silly example, one can imagine a social network where your profile picture can be decorated with a virtual hat, which degrades over time. You can only remain on the service if your profile has a hat; sponsors receive a certain number of hat credits, which they can use to purchase various hats and gift them to their peers. Lastly, any user can trade or gift a hat that they possess. The combination of these mechanics makes the act of "wearing" a hat expressive, not merely pecuniary; wearing a hat that one of your friends obtained and gave to you can be construed as a fun social act which strengthens your friendship, rather than a purely financial necessity. By adjusting the number of hat credits that sponsors get, you can create enough liquidity in the system that most active users have multiple hats. Therefore, it is possible to beg your friends for a particular hat without disclosing that you just don't feel like buying any hats — for example, a user who doesn't want to pay for the service might ask "Hey, anybody got a spare blue knit cap? My last one is expiring next week." A certain degree of strategic ambiguity is preserved.</p>
<p>This example is crude and probably too nakedly gamified to work, but I hope it illustrates that there is a gigantic space of possibilities for designing the social character of sponsorship. Somewhere in that space, I conjecture that there is a point where people are comfortable with sponsor-beneficiary relations in a social network.</p>
<p>Precarity may also be amenable to engineering solutions. For example, one could allow and encourage users to be sponsored by multiple people, and then grant enough tokens to sponsors that their "radius of influence" would, in practice, always overlap with other sponsors'. Then, in steady state, most users would feel secure, because they would be sponsored by more than one person. And in a tentpole freemium model, users would always continue to have access to their identity even when sponsorships expire, reducing the downside even if one were to lose all of one's sponsors.</p>
<p>Have there been examples of tentpole sponsorship as a business model in the wild? I have trouble thinking of them.</p>
<p>Anecdotally, one sometimes hears of people buying paid Slack workspaces to socialize or organize activities that are not part of their day job. I assume that there are usually free riders in this arrangement. So, Slack may have stumbled on this model without intending to (obviously, Slack's primary revenue stream is charging businesses for employee accounts, which is socially a very different scenario, although arguably isomorphic to tentpole sponsorship in some ways).</p>
<p>Alternatively, one could argue that whenever a highly technical user sets up a custom email domain for their family, rather than just signing everyone up for Gmail, they are tentpoling the base protocols of the Internet. The difference, I guess, is that sponsorship is not fungible: if you set up a domain for your family, your child cannot change their sponsor later in life without migrating to another domain, which incurs various transition costs.</p>
<p>The last example I can think of is in gaming. In some multiplayer games like <i>Lineage</i>, players can organize into clans, and clans can purchase in-game <a href="https://l2revolution.gamepedia.com/Clan_System#Purchase_Shop">collective goods</a>. I've never played <i>Lineage</i>, but I assume that players within a clan differ in their level of contribution, and thus the most committed players are effectively sponsoring the rest.</p>
<p>Overall, however, I think the idea of tentpole sponsorship has seen little use, and this seems like a space that is ripe for experimentation.</p>
<hr/>
<p>Having read this, your reaction might be (probably should be!), "Talk is cheap. Ideas are cheap. What are you gonna do about it?"</p>
<p>Alas, I have to admit that the answer is very little.</p>
<p>To really pursue this idea would be multi-year effort, and there are all kinds of reasons that this does not seem like the thing that I want to spend the next few years building. (For one thing, a half-hermit misanthrope like me is probably one of the worst people in the world to try building a social network.) So, instead, I'm throwing this post out there in a sort of cry to the universe, both to get it out of my head, and also in the vague hope that it infinitesimally increases the probability that somebody will figure out how to make it work.</p>
<p>This may be the dumbest <a href="http://www.theoryofchange.org/what-is-theory-of-change/toc-background/toc-origins/">theory of change</a> that's ever been written down, but it's about what I can muster at this point in my life. On the other hand, if you back up and squint, <a href="https://abstractfactory.blogspot.com/2009/07/business-model-for-reporting.html">in 2009 I predicted</a> (sort of) both <a href="https://en.wikipedia.org/wiki/Patreon">the business model of Patreon</a> and <a href="https://en.wikipedia.org/wiki/Jeff_Bezos#The_Washington_Post">Jeff Bezos's purchase of the Washington Post</a>, so maybe the universe will again cough up something resembling my half-baked ideas.</p>
<hr/>
<p>Bonus thought: once you have the idea of tentpoling in your mental toolkit, you will begin to see echoes of it in many places. For example, nearly every software package is sometimes hard to use. But some users have the inclination and capability to become expert in that software, and then spend effort helping others cope with it. These helpful experts are technical (rather than financial) tentpoles, paying the cost of onboarding and support for users in some radius around them. Every geek who serves as tech support for their parents' devices is holding up the tent of Microsoft or Apple or Google or whatever over their family.</p>
<p>In fact, many instances of free riding can be thought of as tentpoling on some level. I suppose the difference between the concept of tentpoling and free-riding in general is that tentpoling is voluntary and has a significant dimension of locality in the social graph.</p>
<hr/>
<p class="footnote"><a name="20170929_footnote0">[0]</a> Arguably, there is also a market in lemons for software services that offer users privacy and control. This is a separate issue and much too big to tackle in this post.</p>
<p class="footnote"><a name="20170929_footnote1">[1]</a> On a vaguely related note, observe that <a href="https://twitter.com/Pinboard/status/913500089137557504">Maciej</a> <a href="https://twitter.com/Pinboard/status/900519092955873285">Ceglowski</a> <a href="https://twitter.com/Pinboard/status/900781090721476608">has repeatedly</a> <a href="https://twitter.com/Pinboard/status/887164089050574848">suggested</a> that Twitter should adopt an ordinary-freemium model where users just pay money for additional features. It is an interesting thought puzzle to contemplate why Twitter has never even experimented with doing this. There seems to be a real organizational dynamic in business that once a company settles on an advertising-supported revenue model, this sucks up all the oxygen necessary for alternate revenue models to breathe, and I do not entirely understand why. Consider how long it took for YouTube to offer YouTube Red; although this is also a case which proves that it is not impossible for the alternative model to break through.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-87578190893992796292017-09-25T08:00:00.000-07:002017-10-05T10:09:30.159-07:00What's the point of Facebook alternatives?<p>It is clear at this point that Facebook has a monopoly on online human-to-human interaction that no private forces, market or otherwise, will break in the foreseeable future. The network effects from a billion users are unsurmountably large. If we take Metcalfe's Law literally, even a social network that accumulates a hundred million users will be a hundred times less powerful than Facebook.<a href="#20170925_footnote0">[0]</a> In fact, you're probably confused by the title of this post: <i>What Facebook alternatives?</i></p>
<p>Facebook is furthermore unlike the other American technology giants in that it alone locks up <em>all</em> its users' interactions inside its walled garden. Apple, Alphabet, Amazon, and Microsoft are, to greater or lesser degree, porous at the edges — you can use an iPhone to chat with people who don't have iPhones; you can use Gmail to email people who don't have Gmail; buying things from Amazon doesn't prevent you from buying other stuff elsewhere; even Microsoft has realized belatedly that it is not the center of the universe & its products have started playing nice with others. But Facebook locks up your posts, locks up your photos, locks up your entire social identity inside its prison. There simply is no way to interact with Facebook users except by creating a Facebook account yourself and creating content that further entrenches Facebook's monopoly.</p>
<blockquote class="twitter-tweet" data-conversation="none" data-lang="en"><p lang="en" dir="ltr">what'd help community groups get off of facebook? I see a lot of small community groups trying to plan events via facebook.</p>— alienghic (@alienghic) <a href="https://twitter.com/alienghic/status/910560086115807232">September 20, 2017</a></blockquote>
<blockquote class="twitter-tweet" data-conversation="none" data-lang="en"><p lang="en" dir="ltr">In my experience, there is no plausible alternative to Facebook if you're working with ordinary people. <a href="https://t.co/B5gLjuERzc">https://t.co/B5gLjuERzc</a></p>— Zeynep Tufekci (@zeynep) <a href="https://twitter.com/zeynep/status/910560474013593605">September 20, 2017</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>The gradual decay of open Internet protocols as human interaction disappears down the black hole of Facebook's ever-expanding digestive tract has been one of the great disappointments of my lifetime. In the end, AOL seems to have beaten the Internet after all.</p>
<p>I have opted for only <i>de minimis</i> engagement with Facebook, and more or less refuse to communicate via its platform. This has probably attenuated some of my relationships with people in a regrettable way (if you're somehow reading this and you wish this hadn't happened between us, send me email! it still works!) but the actions of conscientious objectors like me have not made the tiniest scratch on Facebook's dominance.</p>
<p>It is only a matter of time before governments realize that this entity must be regulated, whether under antitrust law or otherwise. The question is what will happen then.</p>
<p>In my opinion, it is clear what the ideal outcome would be: forcing Facebook to adopt open APIs that give users transparency, portability, and interoperability. Users should be able to see the data that Facebook has stored about them. Users should be able to export that data <i>in toto</i> to competing platforms. And users should be able to interoperate between Facebook and other social networks — a future version of <a href="https://diasporafoundation.org/">Diaspora*</a>, for example, should be able to see and interact with Facebook content generated by that Diaspora* user's social network, and vice versa; interactions between users across platforms should be reflected accurately on both sides. A user would thus be able to leave Facebook without severing their ties to the users they have left behind.</p>
<p>In a world where these APIs existed, users would have a way to reject Facebook's toxic business model and questionable privacy practices without exiling themselves from their social life. In <a href="https://en.wikipedia.org/wiki/Exit,_Voice,_and_Loyalty">Hirschmanian terms</a>, users would have the option of exit, not just voice, as a way of signaling dissatisfaction. Facebook would probably even get healthier, as a product, as a result of the opportunity for meaningful competition.</p>
<p>This outcome is exceptionally unlikely. Government regulation of Facebook, although likely inevitable, is also likely to be ham-fisted and ineffective, simply because governments are terrible at understanding technology and rarely have the political will to impose effective solutions even if they knew of them. The last time the U.S. government, for example, <a href="https://en.wikipedia.org/wiki/United_States_v._Microsoft_Corp.">used antitrust law against a technology monopolist</a>, it basically bogged down the company in red tape for a decade but did little to meaningfully give its competitors an opening in the allegedly monopolized market. Windows is still by far the most widely used desktop operating system and the <a href="https://www.google.com/chrome">web browser</a> that finally dethroned Internet Explorer on Windows did so through incredibly aggressive Internet marketing, not by using the remedies forced on Microsoft by antitrust law.</p>
<p>However, there is one thing that technologists might be able to do to make the desirable outcome marginally more likely, and that is to develop the protocols, and plausible implementations thereof, that would allow effective federated social networking to be mandated by government decree. Diaspora* may have made a significant dent in a subset of the technical problems, but <a href="http://randomwalker.info/publications/critical-look-at-decentralization-v1.pdf">there are significant open challenges</a> inherent to federated social networking that I suspect have not been solved.</p>
<p>Critics of Diaspora*, Mastodon, etc. thus misstep when they observe that organic growth of these platforms is limited. The ultimate destiny of a successful federated social networking protocol, if one ever arises, will be to stock the toolkit of a future regulator, not to overtake Facebook via organic growth.</p>
<blockquote>
<p>There is enormous inertia—a tyranny of the status quo—in private and especially governmental arrangements. Only a crisis—actual or perceived—produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes politically inevitable.</p>
<p>— <a href="https://en.wikiquote.org/wiki/Milton_Friedman">Milton Friedman, <i class="title">Capitalism and Freedom</i></a></p>
</blockquote>
<hr/>
<p class="footnote"><a name="20170925_footnote0">[0]</a> 100M is, to an order-of-magnitude approximation, the size of Snapchat's active user population. Observers who think Snapchat is a credible challenger to Facebook are off by a factor of a hundred, not a factor of ten.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-35514302829596578422017-08-02T13:51:00.001-07:002017-08-02T14:12:24.119-07:00How important is decidability in programming tools?<i>Hoisted from the drafts folder because <a href="https://github.com/Microsoft/TypeScript/issues/14833">the discovery that TypeScript's system is Turing-complete</a> is making the rounds.</i>
<p>It's a common assumption in the programming language community that practical type systems should be decidable. Or, at least, it used to be: in the early 2000s, I had a grad school colleague who spent a lot of time trying to invent restrictions that made the type system of <a href="https://en.wikipedia.org/wiki/Cecil_(programming_language)">Cecil</a> decidable, and people seemed to feel that his formalization of Cecil would be unreasonable if he could not solve this problem.</p>
<p>Over a decade later, it's <a href="https://arxiv.org/abs/1605.05274">well-known</a> that Java with generics and wildcards was undecidable for years before people even realized it. Furthermore, devising restrictions to the type system that achieve decidability seems to be an <a href="https://dl.acm.org/citation.cfm?id=1993570">open research problem</a>.</p>
<p>Did hundreds of thousands of Java programmers experience an existential crisis when Java with generics was found to be undecidable? Are they waiting with bated breath for researchers to solve this urgent problem? I'm pretty sure 99.99% of them don't know that the problem exists and won't notice if it's ever fixed.</p>
<p>Meanwhile, the PL research community has mostly gotten bored of object-oriented languages and type systems thereof, and moved on to other things. And the functional programming community (or some fraction thereof) has embraced Scala, whose type system is also undecidable.</p>
<p>Conversely, it is well-known that in ML, it is possible to write a program by hand on a single sheet of paper that will take longer than the lifetime of the universe to typecheck, because <a href="https://cs.stackexchange.com/questions/6617/concise-example-of-exponential-cost-of-ml-type-inference">Hindley-Milner is superexponential in the worst case</a>.</p>
<p>What is the practical difference between a typechecker that can hang until the universe is a cold, featureless void of uniformly distributed entropy and a typechecker that can hang forever? In both cases, you would hit Ctrl-C after a decent interval and rewrite your program. Would it make much difference if ML's type system were undecidable? Probably not.</p>
<p>I remember standing up at an ECOOP talk one year and asking a question along those lines. The researchers had made a big deal about the importance of decidability for the problem they were solving, and I asked why it was so important. The presenters seemed to think the question was ridiculous and I'm pretty sure everyone else in the room thought so too. I probably didn't word my question well — I wasn't a good researcher and I'm not proud of my time in academia — but I still think I was basically right that one time.</p>
<p>The important thing is that tools have tractable runtimes for the programs that people want to write. Unfortunately, "tractable runtime for reasonable programs" is much harder to demonstrate using the formal and empirical tools that exist today than "decidable for all syntactically valid programs". And the gap in available intellectual tools has led a research community to handcuff itself needlessly in hunting for useful results.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-75924498372376760452017-05-07T23:14:00.001-07:002017-06-23T20:24:38.391-07:00On the efficacy of online flamewars<p><i>Excavated from the drafts folder for no particular reason.</i></p>
<p>Isn't it great how, since the Brendan Eich affair, all his online defenders have become active labor rights organizers, fighting for workplace political freedom for all? Galvanized by the realization that not only CEOs but all workers deserve robust protections for their political beliefs, Eich's erstwhile defenders channeled all their passion into effective political action. Which is why Congress will vote this week on a bill with three key provisions: first, it outlaws any form of workplace discrimination based on political speech or activity conducted outside the office; second, it handsomely funds an investigative division of the FBI tasked specifically with working with the NLRB to track down and prosecute violators; third, by analogy with the Foreign Corrupt Practices Act and the 2003 Protect Act (which grants American prosecutors broad latitude to charge Americans who molest children while abroad), it makes it illegal for companies operating on American soil to subcontract work to overseas employers which restrict their workers' political rights.</p>
<p>There was never any danger that Eich's defenders would just basically forget about the whole affair and get on with their lives. Ha ha! Yeah that definitely couldn't have happened, given how deeply committed these people were to the principle of workplace political freedom. It's not like they only care about workers' rights when it's an incredibly wealthy white male celebrity who is being criticized.</p>
<p>Likewise, when the dead bloated corpse of patriarchy is laid to rest this fall, everyone will have to recognize that the great Twitter Flamewar of March 2014 was really the spike through its heart. No, not that one, I mean the other one, the one where we all wrote angry all-caps tweets at that one dude, he was totally mansplaining and stuff, you remember the one I'm thinking of.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-91963901922628784062016-06-22T22:58:00.001-07:002017-08-13T08:30:16.227-07:00We must reject watchlists if we are a nation of laws<p>I favor strict national gun regulation. There are bad ways to do it, and good ways, but almost any consistent regime would be an improvement over the status quo. If you had asked me the day before yesterday, I would have said that I'd support nearly any gun control measure that was brought before the U.S. Congress.</p>
<p>Well, congratulations, Democrats! You've discovered <a href="https://www.aclu.org/blog/washington-markup/use-error-prone-and-unfair-watchlists-not-way-regulate-guns-america">a way to do it, maybe the only way,</a> that I find stupid and unconscionable: extending the power of "terrorist watch lists" into the realm of gun control. That you should stage a theatrical stunt on the House floor over this measure, of all measures, after you've spent my entire adult life caving cravenly to one right-wing authoritarian demand after another, from illegal war to the normalization of torture, beggars the imagination.</p>
<p>Right off the bat, let's be clear that this measure will <a href="https://nplusonemag.com/online-only/online-only/the-gun-control-we-deserve/">have almost no effect on gun violence</a>. But unlike most gun control measures, this one is not merely feeble; it is actively malign, because it further empowers an evil institution.</p>
<p>The so-called <a href="https://en.wikipedia.org/wiki/Terrorist_Screening_Database">"terrorist watch list"</a> is a fundamentally broken idea that is both impossible to implement well, and a moral catastrophe for the rule of law and human equality and liberty.</p>
<p>There are over a million identities on the watchlist, mashed together from many sources of unknown provenance and overseen by nobody with any accountability to disinterested review. Anybody who has worked with database integrations and human organizations of any size knows with total certainty that this watchlist is full of nonsense. Ted Kennedy was on the watchlist. <a href="https://en.wikipedia.org/wiki/Shah_Rukh_Khan">Bollywood movie star Shah Rukh Khan</a> was on the watchlist. The "terrorist watchlist" is a pile of garbage wrapped in a tire fire.</p>
<p>It can't help but be so: the count of people worldwide who engage in non-state terrorism against civilians is miniscule. It is a numerical certainty that nearly all of the people who are on a million-member watchlist are entirely innocent and have no potential to commit a terrorist act.</p>
<p>And how did the data come to be such a fetid swamp of nonsense? Nobody will tell you. The contents of the watchlist are secret; the processes for getting an identity onto the watchlist are secret; the criteria for getting your identity <em>off</em> the watchlist are secret. Good luck if you end up on it, and you're somebody with less clout than a sitting United States Senator or an international celebrity.</p>
<p>Maybe it's time for a brief refresher on how a nation of laws is supposed to work. Laws must be known to the people who govern them. There must be an agreed-upon process — a <em>due process</em>, you might say — by which people are deprived of their rights under those laws. Once you have been convicted of a crime, the state's carceral machinery may act upon you, but until then, <a href="http://abstractfactory.blogspot.com/2005/05/suspected-terrorists-have.html">merely <em>suspected</em> persons retain their rights</a>. And you don't lose your rights in secret, whereupon you may sue to get them back; the burden of proof is on the state to exercise this due process to take your rights away.</p>
<p>Making legal processes open to public inspection is the most powerful way that we know to ensure that their operation is just; that, for example, we are not merely turning people into second-class citizens for being Muslim or black or whatever the current least-favored category of citizens is. One would think that this argument, at least — that, as Black Lives Matter and related movements have made undeniably clear in recent years, the state operates in flagrantly discriminatory ways when it operates without scrutiny — would carry some currency, even in a left-of-center discourse that values civil liberties less and less as the Bush years recede into the distance.</p>
<p>Yet through the depressing machinery of tribalism, it has suddenly become conventional wisdom in progressive circles that all right-thinking people shall support a bill that further entrenches the influence of the racist, unaccountable, unconstitutional no-fly-list. Otherwise-intelligent people in my Twitter feed have even argued that passing this legislation is the first step towards "fixing" the watchlist:</p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">5/ both so we can *start* regulating guns, *and* so we can start fixing the watchlist, we should absolutely pass the Democratic proposal.</p>— Ben Adida (@benadida) <a href="https://twitter.com/benadida/status/745784336192921600">June 23, 2016</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Astonishing. Astonishing. I have no way of even processing the level of wishful thinking and partisan groupthink necessary for a reasonable person to make this argument in good faith. Can you think of a single example in history where giving a Kafkaesque bureaucratic apparatus additional unaccountable power led to its reform, let alone any such apparatus connected with the security state? I doubt it; the ratchet turns in the other direction.</p>
<p>I have an alternate prediction. Should this legislation be passed by Democrats, we can look forward to the positions around the watchlist becoming crystallized along partisan lines. Since empowering the watchlist will count as a signature political achievement for Democrats, attained through a highly memorable media stunt, henceforth Democrats will fight any efforts that even remotely smell like dismantling the watchlist; this will, of course, include any efforts at meaningful reform. As reverence for the immaculate watchlist turns into a shibboleth of partisan identification, enterprising Democrats will eventually start to propose even more unconstitutional measures to extend its influence into other areas of public life. The extensions will come with, at best, token reforms; perhaps these reforms will moderate its effects on upper-class and upper-middle-class people with surplus time and financial resources, but leave the overall system fundamentally unaccountable and outside any recognizable form of due process for nearly everybody on it. Millions more human beings will suddenly become second-class in the eyes of state; their circle of rights will gradually shrink. The normalization of secret law in the United States will accelerate. Neither Democrats nor Republicans will have the stomach to turn this ratchet backwards.</p>
<p>Furthermore, having discovered again that "terrorism" is the magic word which can rally even the most spineless legislators into action and cow even the most intransigent opponents, Democrats will use this handy rhetorical cudgel to beat anybody who disagrees with them, just as Republicans did during the Bush years, until it is nearly meaningless with overuse. This will be used to pass all manner of additional legislation, equally stupid. Meanwhile, progressives like myself will be excoriated for allegedly being on the side of terrorists and right-wing loons, simply for opposing stupid and malign laws.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-49027554797382281202014-11-14T07:00:00.000-08:002014-11-14T11:46:43.662-08:00The goTo object pattern in Node.js code<p>I've been messing around with Node.js lately. Like everyone using Node.js, I've been wrestling with the fact that it forces programmers to use hand-rolled <a href="http://matt.might.net/articles/by-example-continuation-passing-style/">continuation passing style</a> for all I/O. Of course, I could use something like <code><a href="https://github.com/caolan/async">async</a>.waterfall()</code> to eliminate boilerplate and deeply indented nested callbacks. However, since I am crazy, I am using <a href="https://developers.google.com/closure/compiler/">Closure Compiler</a> to statically typecheck my server code, and I don't like the way async.waterfall() defeats the typechecker.</p>
<p>You can stop reading here if you're perfectly happy with using <code>async</code> for everything, or perhaps if you're one of those people convinced that static typing is a boondoggle.</p>
<p>Anyway, I've been using a "goTo object" for my callbacks instead. The essence of the pattern is as follows:</p>
<ul>
<li>Define a local variable named <code>goTo</code> whose members are your callbacks.</li>
<li>Asynchronous calls use <code>goTo.functionName</code> as the callback expression (usually the final argument to the asynchronous function).</li>
<li>At the end of the function, outside the <code>goTo</code> object, start the callback chain by calling <code>goTo.start()</code>.</li>
</ul>
<p>Here is an example, loosely patterned after some database code I recently wrote against the <a href="https://github.com/brianc/node-postgres"><code>pg</code></a> npm module. In raw nested-callback style, you would write the following:</p>
<pre>
var upsert = function(client, ..., cb) {
client.connect(function(err, conn) {
if (err) {
cb(err);
return;
}
conn.insert(..., function(err, result) {
if (err) {
if (isKeyConflict(err)) {
conn.update(..., function(err, result) {
conn.done();
if (err) {
cb(err);
return;
}
cb(null, 'updated');
});
}
conn.done();
cb(err);
return;
}
conn.done();
cb(null, 'inserted');
});
});
};
</pre>
<p>With a goTo object, you write this:</p>
<pre>
var upsert = function(client, ..., cb) {
var conn = null;
var goTo = {
start: function() {
client.connect(..., <b>goTo.onConnect</b>);
},
onConnect: function(err, conn_) {
if (err) {
goTo.finish(err);
return;
}
conn = conn_; // Stash for later callbacks.
conn.insert(..., <b>goTo.onInsert</b>);
},
onInsert: function(err, result) {
if (err) {
if (isKeyConflict(err)) {
conn.update(..., <b>goTo.onUpdate</b>);
return;
}
goTo.finish(err);
return;
}
goTo.finish(null, 'inserted');
},
onUpdate: function(err, result) {
if (err) {
goTo.finish(err);
return;
}
goTo.finish(null, 'updated');
},
finish: function(err, result) {
if (conn) {
conn.done();
}
cb(err, result);
}
};
goTo.start();
};
</pre>
<p>This pattern is easy to <a href="https://developers.google.com/closure/compiler/docs/js-for-compiler">annotate</a> with accurate static types:</p>
<pre>
var upsert = function(...) {
/** @type {?db.Connection} */
var conn = null;
var goTo = {
...
/**
* @param {db.Error} err
* @param {db.Connection} conn_
*/
onConnect: function(err, conn_) { ... },
/**
* @param {db.Error} err
* @param {db.ResultSet} result
*/
onInsert: function(err, result) { ... },
/**
* @param {db.Error} err
* @param {db.ResultSet} result
*/
onUpdate: function(err, result) { ... },
/**
* @param {?db.Error} err
* @param {string=} result
*/
finish: function(err, result) { ... }
};
goTo.start();
};
</pre>
<p>Some notes on this pattern:</p>
<ul>
<li>It is slightly more verbose than the naive nested-callback version. However, the indentation level does not grow linearly in the length of the call chain, so it scales better with the complexity of the operation. If you add a couple more levels to the nested-callback version, you have <a href="https://twitter.com/SlexAxton/status/455568049181958144">"Tea-Party Code"</a>, whereas the goTo object version stays the same nesting depth.</li>
<li>As in the nested-callback style, error handlers must be written by hand in each callback, which is still verbose and repetitive: the phrase <code>if (err) { goTo.finish(err); return; }</code> occurs repeatedly. On the other hand, you retain the ability to handle errors differently in one callback, as we do here with key conflicts on insertion.</li>
<li>Callbacks in the <code>goTo</code> object can have different types, and they will still be precisely typechecked.</li>
<li>The pattern generalizes easily to branches and loops (not surprising: it's just an encoding of old-school <code>goto</code>).</li>
<li>Data that is initialized during one callback and then used in later callbacks must be declared at top-level as a nullable variable. The top-level variable list can therefore get cluttered. More annoyingly, if your code base heavily uses non-nullable type annotations (Closure's <code>!T</code>), you will have to insert casts or checks when you use the variable, even if you can reason from the control flow that it will be non-null by the time you use it.</li>
<li>Sometimes I omit <code>goTo.start()</code>, and just write its contents at top-level, using the <code>goTo</code> object for callbacks only. This makes the code slightly more compact, but has the downside that the code no longer reads top-down.</li>
<li>There is no hidden control flow. The whole thing is just a coding idiom, not a library, and you don't have to reason about any complex combinator application going on behind the scenes. Therefore, for example, exceptions propagate exactly as you'd expect just from reading the code.</li>
<li>The camelCase identifier <code>goTo</code> is used because <code>goto</code> is a reserved word in JavaScript (reserved for future use; it currently has no semantics).</li>
</ul>
<p>For comparison, here is the example rewritten with <code>async.waterfall()</code>:</p>
<pre>
var async = require('async');
var upsert = function(client, ..., finalCb) {
var conn = null;
<b>async.waterfall</b>([
function(cb) {
client.connect(..., cb);
},
function(conn_, cb) {
conn = conn_;
client.insert(..., cb);
}
], function(err, result) {
if (isKeyConflict(err)) {
client.update(..., function(err, result) {
conn.done();
if (err) {
finalCb(err);
return;
}
finalCb(null, 'updated');
});
return;
}
conn.done();
if (err) {
finalCb(err);
return;
}
finalCb(null, 'inserted');
});
};
</pre>
<p>In some ways, this is better and terser than the <code>goTo</code> object. Most importantly, error handling and operation completion are isolated in one location. Also, blocks in the waterfall are anonymous, so you're not cluttering up your code with extra identifiers.</p>
<p>On the other hand, this has some downsides, which are mostly the flip sides of some properties of <code>goTo</code> objects:</p>
<ul>
<li>Closure, like most generic type systems, only supports arrays of homogeneous element type (<s>AFAIK <a href="http://www.typescriptlang.org/Handbook#basic-types-array">Typescript shares this limitation</a></s> update: fixed in 1.3; see comments). Therefore, the callbacks in <code>async.waterfall()</code>'s first argument must be typed with their least upper bound, <code>function(...[*])</code>, thus losing any useful static typing for the callbacks and their arguments.</li>
<li>Any custom error handling for a particular callback must be performed in the shared "done" callback. Note that for the above example to work, the error object must carry enough information so that <code>isKeyConflict()</code> (whose implementation is not shown) can return true for insertion conflicts only. Otherwise, we have introduced a defect.</li>
<li>Only a linear chain of calls is supported. Branches and loops must be hand-rolled, or you have to use additional nested combinators. This doesn't matter for this example, but branches and loops aren't uncommon in interesting application code.</li>
</ul>
<p>Now, goTo objects are not strictly superior to the alternatives in all situations. The pattern still has some overhead and boilerplate. For one or two levels of callbacks, you should probably just write in the naive nested callback style. If you have a linear callback chain of homogeneous type, or if you just don't care about statically typing the code, <code>async.waterfall()</code> has some advantages.</p>
<p>Plus, popping up a level, if you are writing lots of complex logic in your server, I'm not sure Node.js is even the right technology base. Languages where you don't have to write in continuation-passing style in the first place may be more pleasurable, terse, and straightforward. I mean, look: I've been reduced to programming with goto, the original <a href="http://en.wikipedia.org/wiki/Considered_harmful">harmful</a> technology. By writing up this post, I'm trying to make the best of a bad situation, not leaping out of my bathtub crying eureka.</p>
<p>Anyway, caveats aside, I just thought I'd share this pattern in case anyone finds it useful. Yesterday I was chatting about Node.js with a friend and when I mentioned how I was handling callback hell, he seemed mildly surprised. I thought everybody was using some variant of this already, at least wherever they weren't using <code>async</code>. Apparently not.</p>
<hr/>
<p>p.s. The above pattern is, of course, not confined to Node.js. It could be used in any codebase written in CPS, in a language that has <code>letrec</code> or an equivalent construct. It's hard to think of another context where people intentionally write CPS by hand though.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com6tag:blogger.com,1999:blog-5621024.post-72668926928276283282014-11-10T21:13:00.001-08:002014-11-10T21:20:11.150-08:00A lottery is a tax on... people who are good at reasoning about risk-adjusted returns?<p><i class="aside">Rescued from the drafts folder because <a href="https://www.youtube.com/watch?v=9PK-netuhHA">John Oliver has rendered it timely</a>.</i></p>
<p>People who consider themselves smart sometimes joke that a lottery is a tax on people who are bad at math.</p>
<blockquote class="twitter-tweet" lang="en"><p>A smug of satisfied pundits. <a href="http://t.co/em5ZUfcp5f">pic.twitter.com/em5ZUfcp5f</a></p>— Noah McCormack (@noahmccormack) <a href="https://twitter.com/noahmccormack/status/531955240787259393">November 10, 2014</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>The root of this reasoning is that the expected return on investment for a lottery ticket is negative. It can't help but be otherwise: the lottery turns a profit, hence the payout multiplied by the probability of winning must be less than the price of the ticket. Therefore, the reasoning goes, the people who buy lottery tickets must be incapable of figuring this out. Ha ha, let us laugh at the rubes and congratulate ourselves on our superiority.</p>
<p>One rejoinder is that the true value of a lottery ticket, to the buyer, is the entertainment value of the fantasy of winning. One obtains the fantasy with probability 1 and thus as long as the entertainment value of this fantasy exceeds the price of the ticket, the buyer comes out ahead.</p>
<p>There is something to this. But I think a stronger claim can be made.</p>
<p>A lottery is the mirror image of catastrophe insurance. Note that buying insurance <em>also</em> has a negative expected return. Provided their actuarial tables are accurate, insurance companies turn a profit. Therefore the probability of being compensated for your loss, multiplied by the compensation, must be less than the cost of the insurance premiums. But nobody says that insurance is a tax on people who are bad at math. Quite the opposite: buying insurance is viewed as a sign of prudence.</p>
<p>The issue, of course, is that the naive expected return calculation fails to adequately consider the nature of risk and catastrophe. At certain thresholds, financial consequences as experienced by human beings become strongly nonlinear, probably due to the declining marginal utility of money. Suffering complete ruin due to, say, a car accident entails such severe consequences that we are willing to accept a modestly negative expected return in exchange for the assurance that it will simply never occur.</p>
<p>A lottery is simply the flip side of this coin. The extremely rare event is a hugely positive one, instead of a hugely negative one, huge enough to produce qualitative rather than merely quantitative changes to your lifestyle. And one accepts a modestly negative expected return, not so that one can <em>avoid the risk</em>, but that one can be <em>exposed to the risk</em>.</p>
<p>If you are a member of the educated, affluent middle class, there is an excellent chance that your instinct rebels, and you're already hunting for the flaw in this reasoning. Surely there's something sad, and not prudent, about those largely working-class souls who buy a lottery ticket every week, rather than rolling that $52 per year into a safe index fund at a post-tax rate of return of roughly 2.5% per annum, at which rate their investment, if it somehow survived unperturbed the ups and downs of a life that is considerably more exposed to financial risk than a middle-class person's, might compound to the princely sum of a couple months' rent by the time they die, or alternatively enough to pay for a slightly nicer coffin.</p>
<p>And maybe there is a flaw in this reasoning. I'm not completely convinced myself and I'm not going to start buying lottery tickets. But I'm honestly having trouble finding the flaw. Any indictment of spending (small amounts of) money on lottery tickets must surely also apply to buying insurance. If it is worth overpaying a little bit to eliminate the possibility of a hugely negative outcome, surely it is worth overpaying a little bit to create the possibility of a hugely positive outcome. The situations are symmetric, and I think one can only break the symmetry by admitting the validity of loss-aversion (usually viewed as irrational by economists) or something similar.</p>
<p>Alternatively, if you have access to the actuarial tables of your insurance company, then perhaps you can argue that a lottery is, quantitatively, simply a worse deal than your insurance typically is... but you probably don't have access to those actuarial tables. And I sincerely doubt that the widespread middle-class snobbery towards lottery players is based on quantitative calculations of this sort. (Actually, I strongly suspect that it is based on a fallacious, gut-instinct "folk probability" feeling that gambling on any extremely remote event, like winning the lottery, is somehow inherently foolish.)</p>
<p>So what's the flaw? I ask this question non-rhetorically; that is, I am genuinely curious about the answer.</p>
<hr/>
<p>p.s. None of the above, of course, is to say that I think the taxation effect of lotteries, which is staggeringly regressive, is a good thing. I would strongly support the replacement of lotteries with progressive tax increases plus transfer payments! Gambling addiction is bad too. But these things seem distinct from the notion that playing the lottery in small amounts is irrational in some game-theoretic sense.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-87035122080807801902014-10-01T18:44:00.000-07:002014-10-02T10:00:35.369-07:00On recent events in Hong Kong, and related lessons from American history<p><a href="http://www.bbc.co.uk/news/world-asia-china-29453490">The BBC reports:</a></p>
<blockquote>In China, an editorial in People's Daily warned of "unimaginable consequences" if the protests continued...</blockquote>
<p>Unimaginable consequences! I literally can't imagine, actually. They can't roll out the tanks this time. In the age of ubiquitous digital photography and the Internet, it won't be a few grainy shots of one guy with a shopping bag.</p>
<p>I've been telling people for a while that I think that Chinese leaders are way too fearful of political protests. Look at the United States. Street protests may have affected the course of history a half century ago, but in my lifetime, street protests in America have proven to be ephemeral outbursts of emotion, more a substitute for change than a precursor of it. As I've <a href="http://abstractfactory.blogspot.com/2011/10/occupy-and-iraq-war.html">said before</a>, the system of government by elected representatives has evolved a nearly 100% effective immune system against alteration via protest.</p>
<p>For example, a hundred thousand people <a href="http://en.wikipedia.org/wiki/February_15,_2003_anti-war_protest">took to the streets of New York</a> protesting the Iraq war. A decade later, we appear to be <em>mired in eternal war</em> in the Middle East. At this stage, I have serious doubts that the United States military will leave the Middle East in my lifetime. But those protestors, oh, I bet they felt heroic, like they were doing something world-changing when they were marching down that street in such massive numbers. Even that one inevitable guy holding the "Free Mumia" sign at the anti-war rally.</p>
<p>China's leaders need to learn to relax. Rather than clamping down on protests, they should learn from the West: create free speech zones, let the protestors expend their energy, let them lose focus and direction as the tide of life's million distractions gradually erodes their morale, and meanwhile make all the important decisions in incredibly boring committee meetings and reports, which are freely available (and sometimes even broadcast on television channels that nobody watches) but too numerous and too dull for any normal person to keep up with.</p>
<p>It turns out that if you construct your society just right, the drama and glamour of protests can be used, in a judo move, to undermine their effectiveness. The protest becomes a big identity-affirmation party, an end-in-itself. The shorthand "demo" (for "demonstration"), which is what many activist groups in the U.S. call a street protest, is apropos: the point becomes to show off. After the demo winds down, a bunch of energy has been expended, yet the same people are in charge and are even quite likely to be <a href="http://en.wikipedia.org/wiki/United_States_presidential_election,_2004">re-elected</a>.</p>
<p>I don't know quite how we got here, and I'm not saying that I have a better answer, but I will note that if you told aliens that a widely held theory of change among many political activists on Earth was</p>
<ol>
<li>throw a parade;</li>
<li>the government changes its behavior.</li>
</ol>
<p>then the aliens might be confused. They might ask: what is the mechanism that you propose for this causal effect, given that the formal mechanisms for altering laws generally make no reference to parades, but rather to activities that take place mostly indoors, such as voting, drafting legislation, bureaucratic rule-making, and court decisions?</p>
<p>Ultimately, the Chinese government's attitude towards protest displays a deep, perhaps even touchingly naive faith in the raw power of people to change the government. It is heartwarming, in an odd way. And, all cynicism aside, I sincerely wish the Chinese people the best with the whole protest thing. Maybe they can keep it working better than we did.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-3236080409519630992014-09-18T14:37:00.000-07:002014-09-18T15:13:35.349-07:00US ISPs do not deserve much credit for increasing broadband speeds<p><i class="disclaimer">Disclaimer: I worked for Google from 2006-2013, although not on Fiber.</i></p>
<p><a href="https://twitter.com/binarybits/status/512672847374737408">Towards the end of this Twitter thread sparked by Timothy B. Lee</a>, a commenter writes (by way of defending US ISPs):</p>
<blockquote>Internet speeds have increased 1500% in ten years. 250k Wi-Fi hotspots are now available. That's progress.</blockquote>
<p>When I read this, I just thought, this is such bullshit. Taking it apart with sufficiently satisfying thoroughness requires more than 140 characters, so I'm going to say it here.</p>
<p>US ISPs are riding the wave of technological progress. Giving them credit for the wave is evidence of cluelessness so extreme as to strongly suggest intellectual dishonesty.</p>
<p>First, crediting ISPs for the spread of wireless hotspots is especially egregious: it's a bit like giving them credit for Moore's Law. Even if ISPs had completely failed to increase speeds beyond dial-up, people would still want local networks without the hassle of Ethernet cables. The development and spread of wireless technology was not driven by ISPs. In fact, in some ways the opposite is true, as many ISPs retained official policies against running an open wireless hotspot (or even connecting multiple devices via NAT!) long after broadband became widely available.</p>
<p>Second, as for a 15x improvement in ten years[0]: that might sound impressive to some people, but all I can think is that ten years equals nearly seven doublings of Moore's Law and 2<sup>7</sup> = 128. Network speed doesn't track transistor density exactly, but computing technology is full of exponential curves like this (storage density, for example, doubles even faster than Moore's Law). To anyone with a clue about computing technology, 15x in 10 years obviously sounds somewhere between mediocre and lousy. In fact, <a href="http://www.nngroup.com/articles/law-of-bandwidth/">Nielsen's Law</a> predicts compound improvement of 57x over 10 years, or nearly 4x the observed improvement claimed by Dietz.[1] When Dietz calls out 15x improvement as a talking point in ISPs' favor, he is trying to rhetorically exploit an information asymmetry, namely his audience's presumed ignorance of exponential curves in computing technology.</p>
<p>Therefore, the reality is that US ISPs are badly managed technological laggards, just like everyone thinks.</p>
<p>In fact, the pace at which ISPs have taken up technological innovation is so bad that <a href="https://fiber.google.com/">an entrant from another industry</a> was able to enter the market and, without even trying very hard, spank the incumbents so thoroughly that it became a national embarrassment.</p>
<p>Google does so many things so well that you may not even be properly surprised by this fact. Let me try to give you a visceral feel for how anomalous Google Fiber is. Consider what happened when Google entered a market where its major competitor really was pushing innovation to the limit: consider Android. Google had to dedicate hundreds of its best and hardest-working engineers to the project and enlist the support of essentially every other company in the industry, and after eight hard-fought years, the best it can show for its efforts is rough technological parity with Apple.</p>
<p>Or, to take an example where Google was the defender, consider what happened when Microsoft decided to enter the search engine market. Again, this is a market where the strongest competitor really was pushing innovation to the limit. Billions of dollars and millions of engineer-hours later, Bing's search quality is still slightly worse than Google's.</p>
<p>I don't know the head count for Google Fiber (and even if I did, it would be covered by my NDA) but I will venture a <em>strong guess</em> that its engineering head count is far less than Android's, like at least an order of magnitude. In Google's product portfolio, judging by the scale of investment and strategic value, Google Fiber is basically a hobby, one that Google would never even have tried if US ISPs were remotely as good at their jobs as, say, ISPs in South Korea. And yet, technologically, Google Fiber simply outclassed these incumbents, who are supposedly competing and innovating furiously to earn your dollar.[2]</p>
<p>Imagine you had a friend who claimed to be training super-hard at tennis. But whenever you're at the courts, you see them go out to the court, set up the tennis ball machine, and swat lethargically at balls for about ten minutes, after which they sit down to sip an energy drink and diddle around on their phone. Then, one day, your niece, who's in reasonably good shape but has never picked up a tennis racket in her life, drops by, and she walks onto the court and crushes your friend in straight sets without breaking a sweat. You might reasonably question whether your friend was really training as hard as they claim.</p>
<p>In short, lauding the big US ISPs for their piddling achievements is the soft bigotry of low expectations.</p>
<p>But don't worry, ISPs, America isn't giving up on you. We believe that if you're subjected to the right mixture of tough love, including the right kinds of competition and regulation, you're capable of achieving just as much as Google Fiber did. Or, at a minimum, we believe that the ISP <em>market</em> is capable of achieving that. In the meantime, we'll try hard to ignore the risible bullshit of telecom hacks who claim that you're doing well enough.</p>
<hr/>
<p class="footnote">[0] Engineers generally use multiples rather than percentages to describe improvements of this magnitude. Dietz's use of "increased 1500%" rather than 15x is a classic PR hack's way of making modestly sized numbers seem gigantic.</p>
<p class="footnote">[1] Nielsen's predictions are fitted to observations, but those observations are for "high-end users". Dietz's number, which presumably describes mean or median users, reveals how poorly ISPs have done at making high-speed internet affordable despite the march of technology. Note that we do not see this failure in other, more competitive areas of computing technology: Intel's mid-range notebook-class processors follow Moore's Law just as well as its high-end server chips.</p>
<p class="footnote">[2] This paragraph contains a logical honeypot for telecom hacks, who are likely to see an argument that <em>they think they can debunk</em>, but which they can only debunk by resorting to utter bullshit, which they will promptly be called upon. See if you can spot it.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com1tag:blogger.com,1999:blog-5621024.post-22942258704570026532014-09-17T15:24:00.000-07:002014-09-17T18:35:51.533-07:00An anecdote from the UK, autumn 2013<p>With <a href="http://www.theguardian.com/politics/2014/sep/10/scottish-independence-guide-referendum-uk-yes-no">polls opening tomorrow for Scottish independence</a>, I feel like sharing a little story. Last fall, A. and I were in Scotland and England for a couple of weeks*. In Scotland, there was more or less incessant discussion of independence: it was discussed constantly on BBC Scotland; there were leaflets, posters, and newspaper headlines; one cab driver gave us a long disquisition, during a short ride to the train station, on the evils of Blair's capitulation in Iraq and how an independent Scotland was <i>nae goan tae be dragged intae wars over Israel and oil any longer</i> by the U.S.</p>
<p>Now, if you travel through Scotland and then England as a tourist, there is a good chance that you will pull some money out of an ATM in Scotland and want to spend it in England. And although the UK shares a single currency, bills printed in Scotland look different from bills printed in England, and the Scottish variants are rare enough in England that paying with a <a href="http://www.scotbanks.org.uk/banknotes_current_bank_of_scotland.php">Scottish £10</a> note in London elicits a moment of surprise. (Apparently it is not even technically legal tender in England, although everyone accepts it anyway.)</p>
<p>So, I was buying a sandwich at a Pret a Manger in London, and as the cashier was eyeballing both sides of the note, I remarked, "Got it in Scotland. Still a part of the UK, for now."</p>
<p>The cashier looked at me and laughed, saying instantly, "Oh, that's not going to change anytime soon."</p>
<p>"Really? It's all they can talk about up there. It's on the radio all the time..."</p>
<p>"Nah, it's not going to happen."</p>
<p>Her tone was casually amused, not so much <i>Scotland shouldn't secede, that would be terrible</i> and more <i>Ha ha, this Yank, thinks that silliness up north is going to amount to something.</i></p>
<p>Her accent was English, of course; being American, I don't have the ear to pare it down more finely than that, but she was dark-skinned, of African or South Asian descent, and she sounded to me like any other working class London girl. I would venture to guess that this wasn't really your classic upper-class English snobbery about Northerners at work, at least not directly. I think, rather, that the English and Scottish, despite sharing a government, a currency, a language, and a relatively small island, amply interconnected by transit, media, and commerce, had developed completely different collective understandings of the state of the Scottish independence movement. There were, for example, no front-page headlines in London papers (that I can recall) about Scottish independence.</p>
<p>As it turns out, the Scots were right. Even if the current referendum fails, independence has clearly become a real possibility. And it's interesting to me how unseriously the English seemed to take the whole thing, even as late as last fall. Why didn't they see that how real this was? Did most English have no Scottish friends, did they not travel to Scotland? Or maybe the thought of the UK breaking apart was just too massive an event to ponder, sort of like how San Franciscans live in a state of day-to-day collective denial about The Big One?</p>
<hr/>
<p>*Photos (more than any normal person would want to see): <a href="https://plus.google.com/+KeunwooLee/posts/H85XX3gPYPe">Edinburgh</a>, <a href="https://plus.google.com/+KeunwooLee/posts/UTWA52sNwuJ">Holyrood Park</a>, <a href="https://plus.google.com/+KeunwooLee/posts/11MvKMnSwxn">Edradour distillery</a>, <a href="https://plus.google.com/+KeunwooLee/posts/hp44YfnivHc">Quiraing on the Isle of Skye</a>, <a href="https://plus.google.com/+KeunwooLee/posts/1NpPU77FKd5">miscellaneous Scotland</a>; <a href="https://plus.google.com/+KeunwooLee/posts/bSmxrSq6F78">London</a>, <a href="https://plus.google.com/+KeunwooLee/posts/8UfNi4Jj4mc">York</a>.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com1tag:blogger.com,1999:blog-5621024.post-88103452439677784582014-09-16T23:27:00.002-07:002014-09-16T23:28:07.010-07:00The world's most popular functional language, and what it teaches us<p>I realized today, when I read <a href="http://lambda-the-ultimate.org/node/5048">the phrasing of this LtU post</a>, that my <a href="http://abstractfactory.blogspot.com/2014/09/is-education-to-blame-for-functional.html">last</a> <a href="http://abstractfactory.blogspot.com/2014/09/more-on-programming-language-adoption.html">two</a> posts were too pessimistic about functional programming languages. There is, of course, at least one very popular functional programming language, and that is Emacs Lisp. Emacs Lisp is even widely used, at least a little bit, by countless programmers who never use any other functional programming languages at all. But this just confirms my original hypothesis that <a href="http://abstractfactory.blogspot.com/2010/05/how-to-design-popular-programming.html">language popularity is driven almost entirely by platform</a>, not by characteristics of the language itself.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-24644011209726623032014-09-08T13:33:00.002-07:002014-09-08T13:40:55.762-07:00More on programming language adoption, from Meyerovich and Rabkin<p><a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.295.5862">A little bit of vindication</a> from Meyerovich and Rabkin; a quote I found particularly interesting (emphasis added):</p>
<blockquote><p>A given prior language only occasionally correlated with the choice of a specific different language for the next project. Most notably, developers have high propensities to switch between Windows scripting and application languages, such as VBScript and C#. These languages also correlate with Microsoft web-development languages such as ASP. Such correlations are also visible in the results of Karus and Gall [12], who found groupings such as WSDL and XML being used in conjunction with Java.</p>
<p>Notably, <strong>we do <i>not</i> see significant exploration within linguistic families. There is a relatively low probability of switching between Scheme and LISP, or between Ruby, Python, and Perl.</strong> We conclude that developer movement between languages is driven more by external factors such as the developer’s background or technical ecosystem than by similarity of the underlying languages. This implies that language advocates should focus on a domain and try to convince programmers in that domain, instead of trying to convince programmers who use languages with semantic similarities to the new language.</p></blockquote>
<p>Note that this clearly weighs against the <a href="http://abstractfactory.blogspot.com/2014/09/is-education-to-blame-for-functional.html">Chaudhuri/Hicks hypothesis</a> that education or unfamiliarity with functional programming is the "real problem". If developers tended to choose languages based primarily on comfort and familiarity, then we would expect them to switch more frequently among languages within a family than across families. Instead we observe the converse pattern: developers switch quite freely between programming language families whenever they need to do so in order to get work done in their domain.</p>
<p>In fact I think Meyerovich and Rabkin are too tentative in their formulation (maybe appropriate in an academic paper, but here we don't need to be so tentative). I think it is quite unlikely that developer background is a major deterrent to new language adoption. To repeat something I said the other day, developers routinely learn all kinds of weird, complicated, and frequently frustrating technologies in the course of their work. New programming languages are not fundamentally harder than all these other technologies, and programmers will learn them when they need to in order to get work done. The problem most unpopular programming languages face is simply that nobody needs them to get work done.</p>
<p>Overall, people who wish to change the mix of programming languages currently in use should spend less time extolling the virtues of their language (and criticizing competing languages!), and much, much more time developing platforms and libraries to make their language of choice a stellar tool in some concrete domain.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-26830082077554320272014-09-02T19:50:00.000-07:002017-07-29T11:04:12.333-07:00Is education to blame for functional programming's minority status?<p>Rice University's Swarat Chaudhuri <a href="http://www.pl-enthusiast.net/2014/09/02/who-teaches-functional-programming/">asks (and attempts to answer) the perennial question:</a> why isn't functional programming more popular? I have my own long-running theory about why programming languages become popular (or don't), but first let me dispute a couple of specific things in the linked post. Chaudhuri writes:</p>
<blockquote>The same survey also showed that the factor that correlates the most with preferring and using a language is the availability of libraries. This is certainly behind the meteoric rise of, say, Python. However, it seems implausible that this factor is the primary reason why functional programming is unpopular. If it were so, the F# language, which allows functional programmers to utilize the entire .NET infrastructure, would be a mainstream language by now. There’s no indication it will be any time soon.</blockquote>
<p>I think Chaudhuri dismisses the hypothesis far too lightly. Here are three obvious reasons why F# is not a counterexample:</p>
<ul>
<li>I have never programmed in F# (although I've done a little OCaml, and I gather they're almost identical), but my long experience with cross-language interoperability makes me suspect strongly that accessing nontrivial C# libraries from F# is nothing like using libraries written idiomatically in F#. It probably feels much more like calling through a foreign function interface — for example, like accessing Java classes from Jython, except possibly worse, because C# is not only a different language but a different programming paradigm.</li>
<li>There are a large variety of inevitable network effects that come from using a single language within a project. If you are going to use mostly C# libraries in the .NET ecosystem, then a sensible project manager is probably going to choose to implement the project itself in C# rather than F#. This is especially true if libraries would force you to write a lot of your code in a semi-OO style within F# anyway.</li>
<li>The .NET ecosystem has never had great mindshare in the communities where most of the "hot" industrial software development is happening: open source, backend software running in the datacenter, web development, and mobile development. Spend a little time walking around Silicon Valley and San Francisco, and see how many hackers are using Windows. If, somewhere in the sea of Macbooks, you even glimpse someone using a Thinkpad, there's an excellent chance that it's running Ubuntu. Conversely, if you see someone using Windows, there's an excellent chance they're a business suit from a large corporation (at startups, even the businesspeople use Macbooks).</p>
<p>In fact, this was almost as true, last I checked (years ago, admittedly), even <em>within the programming language research community</em>. It is startling to me that a programming language researcher would look around, observe almost nobody they know hacking on Windows, and still ask why F# on .NET has not taken off.</p>
<p>(<a href="http://www.mono-project.com/">Mono</a> notwithstanding, my understanding is that Microsoft has never made it a priority to make .NET development a really great experience on non-Windows platforms. C# may have a lot of libraries, but <a href="http://stackoverflow.com/questions/22462829/library-compatibility-between-c-sharp-net-vs-c-sharp-mono">Mono has always been a second-class citizen</a> and there is an excellent chance that large swaths of the C# ecosystem <a href="http://www.mono-project.com/docs/tools+libraries/tools/moma/">depend</a> on APIs (or, worse, subtle implementation quirks) specific to Microsoft .NET. I suspect any prudent project manager looking at the .NET ecosystem is unlikely to bet the farm on Mono.)</p></li>
</ul>
<p>Next, Chaudhuri goes on to argue that the lack of university education in functional programming is to blame. Well, I won't deny that this is a contributing factor, but: few CS schools these days teach Ruby or Perl or Objective-C, yet those languages seem reasonably popular; few CS programs teach more than rudimentary use of version control, but git (i.e. the most complex version control system known to humankind) seems popular; few CS programs teach web frontend development frameworks or MVC or template metaprogramming or MapReduce (at least, not until recently, and certainly not in intro level classes), yet all those things and many more have managed to achieve significant adoption in industry.</p>
<p>In short, professional developers routinely adopt all sorts of complex technologies that are not taught academically. As cool as functional programming is, I just don't believe it's fundamentally that much weirder or harder than all the things modern developers use every day. If I had told you a decade ago that in 2014, a nontrivial number of professional programmers would be writing server applications and developer tools in <em>hand-rolled <a href="http://c2.com/cgi/wiki?ContinuationPassingStyle">continuation-passing style</a></em>, you would have looked at me funny; yet <a href="http://nodejs.org/">here we are</a>!</p>
<p>So, then, how do I explain the relative unpopularity of functional programming languages?</p>
<p>First, I would observe that most programming languages are not popular, period. People have invented tens of thousands of programming languages, and nearly all of them languish in obscurity. Only a very select few manage to achieve popularity. Given that functional programming languages are a minority of all languages, we should naively expect a minority of popular languages to be functional, just from random selection. The null hypothesis does a lot of work here.</p>
<p>Second, I would observe that nearly all popular programming languages seem to be hybrids. Consider a different programming paradigm: Smalltalk-76 was purely "object-oriented" (everything is an object, every object has a class, every class has a superclass, objects communicate strictly by sending messages), but its most popular descendants seem to be hybrids. For example, C++, Java, and Python are not purely OO.</p>
<p>Therefore, we should expect that a <em>popular</em> functional programming language would also be a hybrid. And indeed when you view things in this light, many popular languages today have adopted bits and pieces that were once viewed as features of functional programming, such as automatic memory management, first-class lexical closures, and parametric polymorphism. Functional programming purists no doubt view this ad hoc borrowing as hopelessly inadequate cargo-cultism that misses the fundamental point of functional programming, but it is nevertheless exactly what we should expect from the gradual popularization of functional programming. In the essay <i class="title">Real Programming in Functional Languages</i> (<a href="http://www.amazon.com/gp/product/0521245036/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0521245036&linkCode=as2&tag=theabstfact-20&linkId=HW7VQ4VP3C733STS">1982</a>), <a href="http://books.google.com/books?id=O_M8AAAAIAAJ&lpg=PA175&ots=l5dausbA04&dq=j%20h%20morris%20jr%20real%20programming%20in%20functional%20languages&pg=PA173#v=onepage&q=functional%20languages%20as%20a%20minority%20doctrine&f=true">J. H. Morris Jr. memorably wrote</a>:</p>
<blockquote>Functional languages as a minority doctrine in the field of programming languages bear a certain resemblance to socialism in its relation to conventional, capitalist economic doctrine. Their proponents are often brilliant intellectuals perceived to be radical and rather unrealistic by the mainstream, but little-by-little changes are made in conventional languages and economies to incorporate features of the radical proposals. Of course this never satisfies the radicals, but it represents progress of a sort.</blockquote>
<p>I therefore claim that some small part of FP's "unpopularity" is apparent rather than real.</p>
<p>However, I admit that even the combination of the previous two explanations does not seem sufficient to explain why no <em>primarily functional</em> programming language has become the default way to program in a popular domain. But I don't think education is enough explanation either.</p>
<p>So I have to fall back on my primary theory: <a href="http://abstractfactory.blogspot.com/2010/05/how-to-design-popular-programming.html">I maintain once again</a> that languages reach popularity via platforms.</p>
<p>Thus, for example, Swift will probably be a big deal, independent of almost any qualities it has as a language. Apple is the dictator for the iOs platform. It seems likely that Apple will eventually make Swift the default way to program on iOs. Therefore, Swift will become popular, despite the fact that <em>zero</em> people graduating from university computer science programs in 2014 were taught Swift in school.</p>
<p>If functional programmers want FP to be a bigger deal, then my personal recommendation is:</p>
<ul>
<li>develop an industrial-quality platform for doing something that large numbers of developers really want to do, and</li>
<li>evangelize the hell out of it, with a seriousness matching that of professional DevRel teams: videos, tutorials, books, portfolio-quality demo sites in GitHub, reliable turnkey commercial hosting infrastructure if need be, etc.</li>
</ul>
<p>Web development is one good candidate domain, since (a) web development is a clusterfuck and thus ripe for improvement; (b) web developers are culturally eager to try the new hotness (in fact, arguably a little <em>too</em> eager); (c) you can reach a large audience without requiring any hard technology transitions of users since everyone has a web browser.</p>
<p>Look, for example, at how Rails lifted Ruby from relative obscurity to the default way (at least for a little while) that startups built websites in the Valley. The web framework space is more crowded today, but the field for new ones still seems fairly open, as long as you bring something new to the table. For example, focusing on realtime interaction seems to have bought <a href="https://www.meteor.com/">Meteor</a> a lot of buzz, despite the fact that its backend is currently built on a <a href="http://hackingdistributed.com/2013/01/29/mongo-ft/">broken database</a>.</p>
<p>Personally I think there is an opening for a "better PHP" — for all PHP's <a href="http://www.phpwtf.org/">WTF</a>/<a href="http://www.reddit.com/r/lolphp">lol</a>, if you study <a href="http://www.infoq.com/presentations/php-history">Keith Adams's talk "Taking PHP Seriously"</a> (<a href="http://www.slideshare.net/zerutreck/taking-php-seriously-keith-adams">slides</a>) it is clear that the PHP runtime does a few things right that no other platform currently does. Of course, at this point, you're probably laughing at the notion that a bunch of functional programming mandarins is going to successfully devise something for the median PHP programmer to pick up and use. But that is the type of work that might make functional programming the default way to do stuff.</p>
<hr/>
<p>EDIT: For more evidence that Chaudhuri and Hicks are wrong, see <a href="https://abstractfactory.blogspot.com/2014/09/more-on-programming-language-adoption.html">Meyerovich and Rabkin's study on language adoption</a>.</p>
<hr/>
<div class="postscript">
<p>p.s. Bizarrely, in a comment on Chaudhuri's post, Bob Harper (whom I have tremendous respect for) claims that Java doesn't have a conditional expression. What? Am I missing something?</p>
<pre>
Object x =
boolExpr1 ? valExpr1 :
boolExpr2 ? valExpr2 :
boolExpr3 ? valExpr3 :
defaultexpr;
</pre>
<p>Is this not just <code>cond</code> with somewhat uglier syntax?</p>
</div><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-48430063766060490022014-08-29T10:40:00.001-07:002014-08-29T10:40:53.157-07:00Uber vs taxi wait times: non-anecdotal evidence<p><a href="http://www.citylab.com/tech/2014/08/uber-has-an-enormous-wait-time-advantage-over-regular-taxis/379358/">More on the taxis vs. Uber in San Francisco beat.</a></p>
<p>I repeat <a href="http://abstractfactory.blogspot.com/2014/06/uber-and-taxis-in-san-francisco.html">my previous assertion</a> that, anecdotally, ">20 min" is an understatement on weekend nights, and that the decision to use only 3 buckets ("<10 min", "10-20 min", ">20 min") conceals the long tail. As in so many other applications, tail latency has a disproportionate negative effect on on user experience, and looking at even the median or mean latency is insufficient: you should measure out to 95th percentile latency at least (n.b. this is standard procedure at Google).</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-90527041087832022572014-08-23T16:59:00.002-07:002014-09-02T20:36:16.369-07:00Making a package.json from exact installed npm package versions<p><i class="disclaimer">Attention conservation notice: Google-food for nodejs users.</i></p>
<p>It's easy to get a list of just the packages you directly depend on:</p>
<pre>npm list --depth=0</pre>
<p>However, there's no built-in way to get <tt>npm list</tt> to output results in package.json format. Here's a little shell recipe:</p>
<pre>npm list --depth=0 \
|tail +2 \
|sed '/^\s*$/d' \
|gawk -F ' ' '{print $2}' \
|gawk -F '@' '\
NR > 1 { printf ",\n" } \
{ printf "\"%s\": \"%s\"", $1, $2 } \
ENDFILE { printf "\n" }'
</pre>
<p>This isn't a complete <tt>package.json</tt>, but it's a format that can easily be copied into one, via a second shell recipe or whatever else you like. (If your pipe-fu is strong you can probably figure out how to extend this to do the whole <tt>package.json</tt> in a one-liner.)</p>
<p>Motivation: When you <code>npm install --save</code> or <code>npm install --save-dev</code>, npm inserts packages with "the default semver operator" by default. It's easy to forget to pass <code>--save-exact</code>; or, if you're just doing exploratory hacking, you might not even want <code>--save-exact</code>. But when you're ready to cut a build for deployment, you need to capture exact package versions, because semver <s>is basically bullshit</s> can't be relied upon. Hence the recipe above, which can be used to generate or update the dependencies section of the package.json in a deploy directory.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-13729286024649839282014-08-09T12:59:00.000-07:002014-09-18T14:55:54.399-07:00A few problems with Firefox<p class="disclaimer"><i>Disclaimer: I worked at Google for 6.5 years, although not on the Chrome team. I am currently independent. I also worked a little bit with Rob at IBM in 2001.</i></p>
<p><a href="http://robert.ocallahan.org/2014/08/choose-firefox-now-or-later-you-wont.html">Rob O'Callahan has a good post</a> explaining why you should use Firefox. I am sympathetic to this argument, but I can't bring myself to switch yet. I try periodically, and every time I end up bouncing off again. Alas, Firefox is currently inferior in specific ways that are cripplingly bad for my needs.</p>
<p>First, as a web developer:</p>
<ol type="a">
<li>The developer tools are <i>really</i> janky.</li>
<li>The profiles functionality is buried and has no in-browser UI. I need quick, simple profile management and switching when testing my apps. (Don't refer me to the add-on market; add-ons are a cesspool.)</li>
</ol>
<p>Second, as someone who recommends tech to my family:</p>
<ol type="a" start="3">
<li>Multi-process isolation and sandboxing have <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=925570">still not shipped</a>.</li>
<li>Firefox's updates are still not as timely as Chrome's. On my personal machines, I often find that I'm running an older version of Firefox <i>weeks</i> after a new version is released. In fact, just as I was writing this, I found that the machine I'm typing it on was still running 30.0, when 31.0 was released on <a href="https://www.mozilla.org/en-US/mobile/31.0/releasenotes/">July 22</a>.</li>
</ol>
<p>As a result of (c) and (d), Firefox is plausibly the least secure major web browser shipping today. I can't recommend Firefox to my family until these things are fixed. I won't expose them to a high risk of exploitation, here and now, solely to protect them from a theoretical risk that they'll be harmed by the Chrome team's product choices in the future.</p>
<p>In addition to all of the above, I think Rob overstates the extent to which Google is (1) winning and (2) likely to use that position to harm users in the foreseeable future.</p>
<p>Obviously, I am biased w.r.t. (2), so I don't think it's productive for me to try to convince you of my point of view in this post. At any rate, Rob is better-informed than me about browser politics, not to mention much smarter than me, so I am willing to believe that he has good reasons for believing what he does.</p>
<p>However, w.r.t. (1):</p>
<ul>
<li>On desktop, Chrome doesn't even have majority market share. It is trending upwards, but it's a long, long climb from its present ~40% position to the 80-90% that Microsoft once had in desktop operating systems. And Google's competitors today are a lot more impressive than Microsoft's were in the 1990s. I don't think it's plausible that a large fraction of the web will build strictly for Chrome, or even Blink-based web browsers, anytime in the foreseeable future, unless all of Google's competitors fumble the ball mightily.</li>
<li>On mobile, iOs isn't going away any time soon. And I would bet (a small amount of) money that forks of Android, from China and elsewhere, will reach rough market share parity with Google Android in the long run.<a href="#20140809_footnote0">[0]</a>.</li>
</ul>
<p>Today's giants always look more invulnerable than they really are. Apple looked unstoppable just a few years ago, Microsoft not that long before that. Facebook looked like it would become the identity layer for all human interaction; now it's just a boring and somewhat declassé social media site for middle aged people (plus a server farm for a few flashy acquisitions). Google may seem like a juggernaut, and to be fair I think it is much more competent on average than any of the three companies I just mentioned, but it's vulnerable in ways that aren't even obvious to us today and I'll be very surprised if we look back in 2024 and find that Google is dictating terms to the rest of the technology industry, in the old Microsoft (or new Apple) mold, rather than being merely one influential player among many.</p>
<hr/>
<p class="footnote"><a name="20140809_footnote0">[0]</a> Incidentally, since Google has let the AOSP web browser languish, and restricted Google Chrome(TM) to its increasingly-tightly-constrained partners, making Firefox run amazingly well on non-Google Android — well enough that non-Google Android users almost universally either get Firefox shipped with their device or install it themselves — might be a more plausible path to getting large mobile market share globally than Firefox OS. (I do think Firefox OS is an important and worthwhile project as well.)</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-72971962082077122522014-07-19T20:28:00.001-07:002014-07-19T20:28:46.643-07:00Fafblog returns, again<p><a href="https://twitter.com/Fafblog">Sort of.</a></p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-27464589108255127982014-06-23T11:28:00.000-07:002014-08-23T17:03:57.511-07:00More on taxis<p>I will fully cop to the fact that <a href="http://abstractfactory.blogspot.com/2014/06/uber-and-taxis-in-san-francisco.html">my previous bitching about taxis</a> concerned a classic San Francisco yuppie firstworldproblem but <a href="http://www.washingtonpost.com/blogs/wonkblog/wp/2014/06/23/very-short-true-stories-about-trying-to-hail-a-cab/">here are some more stories</a> about how taxis in many cities serve people of all stripes exceptionally poorly.</p>
<p>Relatedly, I took an UberX with a friend the other night and the experience was awesome.</p>
<p>I might have liked the option of tipping the the driver (via the app, after you get out of the car), but UberX doesn't allow that. On the other hand, maybe tipping would establish a social norm for companies like Uber to underpay their drivers with the expectation of tips? I really like traveling abroad in countries that don't have a tipping culture; it seems more rational and arguably in the long run leaves labor in a better position since their compensation is assured by contractual terms rather than manners. So maybe it's great that Uber doesn't have tipping, only reviews.</p><div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0tag:blogger.com,1999:blog-5621024.post-16834727925795666982014-06-18T10:02:00.001-07:002014-06-18T11:41:07.236-07:00What happens when you let people pay to remove ads<p><i class="disclaimer">Disclaimer: Obviously I worked at Google for 6.5 years so take this with the appropriate grain of salt.[0]</i></p>
<p>"I wish Google would just let me pay them money to remove ads from this service. WTF GOOGLE IS SO DUMB AND EVIL WHY WON'T THEY DO THIS."</p>
<p>How many times, in various corners of the Internet, have you read some variation of the preceding sentiment?</p>
<p>Now, it turns out, <a href="https://plus.google.com/u/0/111111598146968769323/posts/KMWDJYF5eNJ">Google is trying to do this</a> with a subset of music videos on YouTube. But when you do this, you need a contract with the rights holder. This is complicated. The rights holder might not agree to your contract terms. For example, they might want more money. So you end up in negotiations. While those negotiations are ongoing, or if they break down, you can't include that music in your paid service.</p>
<p>The Internet's collective reaction has been: "WTF GOOGLE IS SO DUMB AND EVIL WHY ARE THEY DOING THIS?"</p>
<p>Now, YouTube will still host music videos that are not opted into its monetization program, i.e., if you are a band and want to put music videos on the Internet for free, YouTube will host those videos for you. Let's be clear what this means: if you want to distribute a video to a hundred million strangers on the Internet, YouTube will pay for the software and servers and a petabyte of network bandwidth and a small army of SREs holding pagers who will wake up at 3am if too many people in Kuala Lumpur click on your video and get an HTTP 50x error, and it will do so without you paying YouTube a dime. None of that's changing. But if you opt into YouTube's monetization program, you will have to opt into its full, updated monetization program: ads for non-subscribers, and no ads for subscribers.</p>
<p>Note that there is no sensible way to let users pay to remove ads from music videos while also still showing ads to those same users for some music videos. If YouTube did this, you can bet the Internet would collectively scream, once again, "WTF GOOGLE IS SO DUMB AND EVIL WHY DON'T THEY REMOVE ADS ON MUSIC VIDEOS WHEN I PAID TO HAVE ADS REMOVED? I'm unsubscribing from this bullshit service!" The subscription service would fail and YouTube would have to revert to its old model of monetizing via ads only.</p>
<p>In short, "blocking" — i.e., excluding unlicensed music from the monetization program — is an <em>inevitable consequence</em> of having a paid subscription service.</p>
<p>The press has done an abysmal job covering this. It seems that every year my contempt for (most) journalists finds reasons to grow greater and greater. I'm pretty disappointed that the Financial Times, reputedly pretty reliable, appears to be Ground Zero for this particular blast of misinformation.</p>
<p>p.s. As for claims that YouTube has anything approaching a monopoly on online video sharing, I'm honestly <a href="http://en.wikipedia.org/wiki/List_of_video_hosting_services">puzzled by the claim</a>. Just to take one example competitor, <a href="http://vimeo.com/">Vimeo</a> is pretty reliable and seems to have rough feature parity, including embedding in external sites. Vimeo's content is included in Google's video search corpus (<a href="https://www.google.com/search?q=fingleton+magpie&tbm=vid">example</a>) and therefore shows up as a thumbnail image in Google web search, just like YouTube videos. If you need to make money, Vimeo has <a href="http://vimeo.com/creatorservices">a few built-in monetization options</a>; if those are insufficient, there's a lot of innovation occurring in the world of funding, and in my opinion the <a href="http://www.patreon.com/">Patreon</a>/<a href="https://subbable.com/">Subbable</a> model seems much more promising for creators with a small-to-medium-sized audience than Spotify-style monetization, which currently amounts to "we'll give you 0.0001 pennies per stream play so that both the company and the artists can lose money hand over fist!" Is it really true that Adele fans will face substantial (or even non-substantial) barriers to watching her music videos if they're on Vimeo? And that's just one competitor. The upshot is that indie labels are probably wrangling with YouTube over licensing terms not because it has anything like a monopoly, but because they think wrangling with YouTube will make them more money than all the other alternatives. Which is perfectly fine — more power to them, negotiating the best deal is essential in business, etc. — but we should not misread the situation.</p>
<hr/>
<p class="footnote">[0] On the other hand, one benefit of no longer working there is that I can write stuff like this post, just like <a href="http://abstractfactory.blogspot.com/2004/05/this-gmail-bullshit-has-gone-too-far.html">I used to</a> before I worked there. Arguably, working at Google made me <em>less</em> predisposed to harshly criticizing misinformed critics of Google.</p>
<div class="blogger-post-footer"><script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
<script type="text/javascript">_uacct = "UA-729338-1";urchinTracker();</script></div>Keunwoo Lee (Cog)http://www.blogger.com/profile/05577836853536292311noreply@blogger.com0