Tab completion for meta-bang shell commands (Wednesday Emacs blogging)

You probably use M-! to run a quick shell command now and then, when you don't want to be bothered with a full M-x shell. But if, like me, you're a former XEmacs user, then you probably find FSF Emacs' default lack of tab-completion for files in the minibuffer rather annoying. Well, your pain ends here:

(if (not (string-match "XEmacs" emacs-version))
    (progn
      (defadvice read-from-minibuffer
        (around tab-is-pcomplete-in-minibuffer activate)
        "Bind TAB to pcomplete in minibuffer reads."
        (let ((keymap minibuffer-local-map))
          (define-key keymap "\t" 'pcomplete)
          (ad-set-arg 2 keymap)
          ad-do-it))))

Ta-da. Now when you M-! mv LongAnnoyingFileName.java LongAnnoyingFileNameFactory.java, you'll be able to tab-complete the filename.

Incidentally, who knew that elisp supported aspect-oriented programming? Apparently it does. Astonishing. I owe this tip to a co-worker, who I'd name except that I doubt he'd want to be associated with the other content on this blog. (p.s. AF, if you ever run across this post and don't mind being credited, I'll happily add your name.)

Meta-slash performs dabbrev-expand, and you require this knowledge (Wednesday Emacs blogging)

If you don't know this one already, then go to an emacs window right now, open up any source file (~/.emacs works fine), navigate to any function, and type the first couple of characters of a nearby identifier. Then type M-/. Ta-da!

Details: By default M-/ is bound to dabbrev-expand, which triggers the dynamic abbreviations facility. This dynamically compiles a dictionary from nearby identifiers in the source file, and offers matching identifiers as completions for the current token, preferring identifiers closer to the cursor over more distant ones.

For bonus points, type M-/ multiple times to cycle among recent matches, or use C-M-/ to pop up a list of matching completions in another buffer.

dabbrev-expand isn't as sophisticated as the semantically aware tab-completion available in many IDEs. Conversely, however, it works with no modification in almost every buffer type under the sun, so you can use it when editing code in elisp, Java, or the language you invented this morning. It even completes reasonably well when editing English prose (although since token prefixes are much less unique within an English document, it's only worthwhile for longer words).

Meanwhile, because dabbrev-expand's algorithm is so simple, it doesn't require a heavyweight background process to scan all your project files and keep an in-memory database up-to-date. This is, of course, a typical IDE pitfall. You'll never be waiting for emacs to repopulate the dabbrev-expand database after you refresh all the files in your project checkout.

I'm a little embarrassed to admit that I only learned this keyboard shortcut a couple of months ago. Yes, that's right, I've been typing all my identifiers manually (or using M-w/C-y to copy-and-paste) for my entire freaking career. I estimate that my long-term danger of RSI declined dramatically the day one of my teammates mentioned this feature.

(On the other hand, my incentive to keep names short has been reduced slightly, and I wonder what effect this will have on the code that I write. It seems to me that although names that are too short can be cryptic, it's good to keep code as concise as it can be, consistent with maintaining clarity. Along similar lines, I suspect, for example, that IDEs which make it too easy to extrude large volumes of boilerplate code, or to import functions from many different modules, result in looser, less organized code.)

ibuffer: If you do not use it, you are insane (Wednesday emacs blogging)

In short, use ibuffer, now; the relevant .emacs magic:

(global-set-key (kbd "C-x C-b") 'ibuffer)
(autoload 'ibuffer "ibuffer" "List buffers." t)

This is so much better than the regular buffer list it's not even funny. The first thing you'll notice is font-lock colorization (welcome to the 21st century!); but the killer feature is the wealth of buffer management keyboard shortcuts. On your first trip to ibuffer, you'll want to spend a little time reading through C-h b to learn the keyboard bindings; a small sample of just the marking functions:

% f     ibuffer-mark-by-file-name-regexp
% m     ibuffer-mark-by-mode-regexp
% n     ibuffer-mark-by-name-regexp

* *     ibuffer-unmark-all
* /     ibuffer-mark-dired-buffers
* M     ibuffer-mark-by-mode
* e     ibuffer-mark-dissociated-buffers
* h     ibuffer-mark-help-buffers
* m     ibuffer-mark-modified-buffers
* r     ibuffer-mark-read-only-buffers
* s     ibuffer-mark-special-buffers
* u     ibuffer-mark-unsaved-buffers
* z     ibuffer-mark-compressed-file-buffers

Once you start chaining these together, you'll wonder how you ever got along without them. For example, I commonly do * s * r t D y: mark all "special buffers" (*shell*, *scratch*, etc.); mark all read-only buffers; toggle marks (marking ordinary read-write buffers); delete marked buffers; confirm. This is handy since I often work on projects via multiple emacs instances, switching between an X11 emacs instance and a tty instance running under screen. When I switch, I want to close all the files I have open for editing (even if there are no unsaved changes), but leave *shell* and dired buffers alone.

Buffer management has historically been a real bottleneck in emacs productivity. Unlike most IDEs, emacs makes it trivial to have dozens or hundreds of buffers open simultaneously. This works great, except that working with all these buffers can become troublesome. For example, to switch to a buffer, you typically C-x b and type-complete the name; but when you have a half-dozen dired buffers all named client, it gets hard to remember whether the one you wanted was client<2> or client<5>. And, of course, the C-[left click] buffer list gets ridiculous with many buffers — you might as well be using Eclipse or something. ibuffer doesn't completely solve all these problems, but it certainly mitigates them.

Fix dired under Fink xemacs (Mac annoyance Saturday)

Tonight I literally spent a couple of hours grappling with an uber-annoying bug under XEmacs in current Fink: dired-mode chokes on directories containing symbolic links. The symptom is that when you try to open a directory in dired, a cryptic "no file on this line" error appears in the minibuffer and dired fails to open.

To make a long story short: this fix from the Apple support forums is (unbelievably) the fastest and most foolproof way to fix this error. Step by step:

1. sudo xemacs /sw/lib/xemacs/xemacs-packages/lisp/dired/dired.el
2. Find the line containing the regex

   "[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][-+ 0-9+]"

   and change the regex to

   "[^ ][-r][-w][^ ][-r][-w][^ ][-r][-w][^ ][@\+]?[-+ 0-9+]"

3. C-x C-s
4. M-x byte-compile-file
5. C-x C-c
6. Quit and restart XEmacs.

Of course, you will need to do this again whenever Fink decides to overwrite dired.el; unless the Fink maintainers merge in the upstream fix.

Gory details below.

---

Under Mac OS X, unlike every other Unix in recorded history, ls inserts an @ or + sign after the permissions list on certain types of files (symlinks and files with "extended security information"). The dired.el code is brittle with respect to this oddity. Furthermore, it does not provide any way to customize the regular expression that matches and parses the output of ls (which, incidentally, is my primary evidence that Mac OS X is the first Unix in history to behave this way; it's essentially impossible that a Unix could have existed for any length of time like this without a patch being fed upstream to Emacs).

Now, it turns out that this bug was fixed in Dired 7.15. So the real problem is that Fink XEmacs has not pulled down the latest Dired; it's currently stuck on 7.13.

So, what's a hacker to do? Let me count the frustrations.

1. XEmacs has a built-in elisp package manager called pui. Maybe I can upgrade Dired that way?

I use M-x pui-list-packages; then I set a package mirror from the Tools > Packages menu; finally, I select the Dired package and type I to install:

(error/warning) Error in process filter: (ftp-error FTP Error: CWD 550 /pub/packages/editors/xemacs/packages/.: No such file or directory failed: )

Oh great. I try a half-dozen different package mirrors and every motherfucking one gives me this error (except when it dies even earlier due to some other FTP problem).

2. Aha, MacPorts has a more recent XEmacs! Screw Fink, maybe the MacPorts guys know what they're doing:

$ sudo fink remove xemacs
. . . long console spew . . . 
$ sudo port install xemacs
. . . long console spew . . . 
$ xemacs
Error: attempt to add non-widget child "*scratch*"
to parent "Buffers" which supports only widgets
[xemacs crashes]

The current MacPorts package of xemacs doesn't even start up. Bloody motherfucker.

3. Maybe I should just switch to FSF Emacs, which I already use on all my other boxes. I mean, Fink has emacs22-gtk now. Yeah, right:

$ sudo fink install emacs22-gtk
Information about 8097 packages read in 1 seconds.

. . . [deletia] . . .
The following package will be installed or updated:
 emacs22-gtk
. . . [deletia; making a long story short:] . . .
sed: /usr/X11/lib/libfontconfig.la: No such file or directory
### execution of /var/tmp/tmp.1.UdYe2K failed, exit code 1
Removing runtime build-lock...
Removing build-lock package...
/sw/bin/dpkg-lockwait -r fink-buildlock-fontconfig2-dev-2.4.1-106
(Reading database ... 42622 files and directories currently installed.)
Removing fink-buildlock-fontconfig2-dev-2.4.1-106 ...
Failed: phase installing: fontconfig2-dev-2.4.1-106 failed

4. So, I finally gave Carbon Emacs a try. Carbon Emacs runs, and dired actually works fine. Miracle of miracles.

Unfortunately, fonts are completely busted under Carbon Emacs. You get a slick anti-aliased font out of the box, but you can look forward to a hair-pulling exercise in frustration if you attempt to change that font to anything that's (a) legible and (b) fits more than 50 vertical lines of text on a Macbook Pro screen. Anti-aliased Courier looks even more terrible at 8pt than Courier usually looks. And Carbon Emacs is the first windowed Emacs I've used where no variant of the "Clean" font works out-of-the-box. Congratulations Apple! You've broken a piece of software that's basically worked for twenty years. So Carbon Emacs is out.

5. About twenty minutes of screwing around trying to install dired from CVS into a user-local directory, and have it override the one in the system directory, yields no progress.

6. I could try to install the latest dired package by hand, but I am sort of scared of what else might break if I do this. So, as much as I dislike manually twiddling with system files that should be under my package manager's control, I grit my teeth and did the voodoo noted at the top of this post. Which finally gets me everything I want, but my mouth is left with the distinct bitter residue of time sucked away from my life that I'll never get back.

---

My conclusions from this whole experience are as follows.

First, package maintainers for Fink and MacPorts aren't as good as the maintainers for Debian or Ubuntu. At any given time, a fair number of important packages will simply be nonfunctional. So the package situation will always be a pain in the ass on Mac compared to Linux.

Second, Mac OS X's random gratuitous differences from other *nix flavors will continue to suck away at least as much time from my life as I spent getting wireless and other random hardware widgets to work on a Linux laptop. Truly, for a programmer, there's no such thing as a laptop where you don't spend a significant amount of time fiddling with random bullshit just so you can get work done.

Soft word wrap for long lines (Wednesday Emacs blogging)

The short story: Use M-x longlines-mode.

The long story: longlines-mode is a minor mode that enables "soft wrap" for Emacs text. In longlines, lines will be displayed wrapped on word boundaries, as under most word processors (but not most text editors). This is useful for editing certain textual formats like HTML or wiki markup.

longlines comes bundled in FSF Emacs 22, but for earlier versions (or XEmacs) do the following:

1. Get longlines.el, from the longlines Emacs Wiki page.
2. Copy longlines.el somewhere on your load path. (To add a directory to your load path, add a line (add-to-list 'load-path "/path/to/directory/") to your .emacs)
3. Add (require 'longlines) to your .emacs.

longlines-mode is a minor mode, and does not automatically hook itself into any major modes. You can hook longlines automatically in the standard fashion:

(add-hook 'html-mode-hook
          '(lambda () (longlines-mode)))

Alternatively, you can simply do M-x longlines-mode to toggle the minor mode manually, as I suggest above.

Once you get started with longlines, you'll probably want to customize its settings to taste. For example, by default the word wrap width won't automatically adjust to the window size. To see all relevant customizable settings, use M-x customize-group [enter] longlines [enter].

Incidentally, this feature is astonishingly obscure and late to arrive to Emacs, given that it's been a standard menu item in crippled brain-dead text editors like Windows Notepad for over a decade. Yet another example of how open source software development is not rationally optimized to serve the needs of the user.

Unix man page lookup (Wednesday Emacs blogging)

Today's a quick one:

; man page lookup (by default, f1 is help, but I
; already know how to bring that up using C-h)
(define-key global-map [f1]
  (lambda () (interactive)
    (manual-entry (current-word))))

As the comment implies, by default f1's bound to Emacs help. But of course, longtime Emacs users know how to acces the rich interactive help system using C-h; C-h ? ? gives you a list of the main C-h functions, but here are some favorites that I hit reflexively almost once a day:

C-h a

"Apropos" help: looks up anything (including both functions and variables) matching a substring

C-h b

List all key bindings in the current mode

C-h k keystroke

Look up the function bound to this keystroke

So, what do you really need that f1 help for? Looking up a Unix manpage on the current word is much more useful, especially in M-x shell mode or other modes where you're editing shell commands.

Fast access to dired file manager (retroactive Wednesday Emacs blogging)

Despite the existence of modern graphical file managers, I still find myself dropping into Emacs dired-mode ("the directory editor") for file management surprisingly often.

I haven't really figured out why. Certain operations just seem easier or faster in dired. Maybe it's because I can use Emacs idioms like incremental search (C-s) for moving to files and directories, and dired-advertised-find-file (f) to open things, which makes navigation speedier than scrolling and mousing inside a folder window. Maybe it's because (unlike graphical file managers) you can mark files (m) for an operation without fear of losing that selection on an errant mouse-click or keystroke. Or maybe it's because of little things like tilde (~), which marks all files in the current dired buffer that match the glob *~ for deletion.

Anyway, calling dired on the current directory when you're editing a file is extremely useful. Accordingly I've bound a keystroke to this function:

; F4 for dired buffer of the current directory in the other window
(define-key global-map [f4] (lambda () (interactive)
    (dired-other-window default-directory)))

The above Elisp binds F4 to open dired on the current directory. If you're a Unix user, then some directories have dotfiles (files whose names begin with .) that you'd rather ignore sometimes. Accordingly, I've also got a binding that uses a regexp to filter out all dotfiles from the view:

; F8 to open dired buffer of the current directory without dotfiles
; in other window
(define-key global-map [f8] (lambda () (interactive)
    (dired-other-window (concat default-directory "[^.]*"))))

Scroll bindings (Wednesday Emacs blogging)

Part of the Zen of Emacs is the fact that --- in contrast to merely mortal text editors or word processors --- you need not move your hands from the default typing position for common navigation tasks, like moving to the beginning of a line (C-a) or word (M-b).

Clearly, therefore, it is uncivilized to use arrow keys, or to move your hand to the mouse and hunt for that little arrow on your scroll bar, when you merely want to scroll up or down a couple of lines. Accordingly, the following elisp binds downwards and upwards scrolling to the M-n and M-p keystrokes respectively.

; Handy incremental scrolling keys
(define-key global-map "\M-n" (lambda () (interactive) (scroll-up 1)))
(define-key global-map "\M-p" (lambda () (interactive) (scroll-down 1)))

I chose M-n and M-p to be mnemonic "cognates" with C-n and C-p. Note that scroll-up is so named because it moves the document up, which makes the viewport appear to scroll down, and vice versa for scroll-down.

Nevertheless, it is sometimes convenient to do scrolling with your mouse --- e.g., if your hand's already there because you're switching focus among open windows. Recent Emacsen understand the scroll wheel, but some older versions do not. Fortunately, they can be taught, using the following recipe suitable for either (FSF) .emacs or (XEmacs) .xemacs/init.el:

; Mouse wheel: scroll up/down; control-wheel for pgup/pgdn.
(defun wheel-scroll-up   ()   (lambda () (interactive) (scroll-up 2)))
(defun wheel-scroll-down ()   (lambda () (interactive) (scroll-down 2)))
(defun wheel-scroll-pgup ()   (lambda () (interactive) (scroll-up 20)))
(defun wheel-scroll-pgdown () (lambda () (interactive) (scroll-down 20)))
(cond
 ((string-match "XEmacs" emacs-version)
  (progn
    (define-key global-map 'button5 (wheel-scroll-up))
    (define-key global-map 'button4 (wheel-scroll-down))
    (define-key global-map '(control button5) (wheel-scroll-pgup))
    (define-key global-map '(control button4) (wheel-scroll-pgdown))))
 (t ; FSF Emacs uses weird [bracket] keymap specifiers.
  (progn
    (define-key global-map [mouse-5] (wheel-scroll-up))
    (define-key global-map [mouse-4] (wheel-scroll-down))
    (define-key global-map [C-mouse-5] (wheel-scroll-pgup))
    (define-key global-map [C-mouse-4] (wheel-scroll-pgdown)))))

This recipe also binds C-mouse-4 and C-mouse-5 to page-down and page-up, for handy fast scrolling, which does not come standard even on recent Emacsen. It should be pretty obvious how to customize it to scroll more or fewer lines at a time.

Rotating background colors (Wednesday Emacs blogging)

Like every veteran Emacs user, I've long used multiple buffers, multiple windows in a buffer (C-x 2) and multiple frames (C-x 5 2) to edit multiple files simultaneously during heavy sessions of programming or writing. Lately I've been working on a monitor so vast, and reading so many different files simultaneously, that I've found it useful to run multiple instances of Emacs as well. (Among other things, this speeds up tab-completion when switching buffers with C-x b, because each instance has shorter buffer list.).

When running multiple Emacs instances, it gets hard to tell which frame corresponds to which running instance of Emacs. The following Elisp code enables rotation among a set of background colors on a keystroke:

(defvar background-color-rotation
  '("aliceblue" "thistle" "lemonchiffon" "khaki" "papayawhip"
    "honeydew" "mistyrose" "paleturquoise")
  "List of background color names to rotate")
(defun next-background-color ()
  "Rotates among colors in background-color-rotation."
  (set-variable 'background-color-rotation
                (append (cdr background-color-rotation)
                        (list (car background-color-rotation))))
  (car background-color-rotation))
(define-key global-map [f10]
  (lambda ()
    (interactive)
    (set-face-background 'default (next-background-color))))

Paste this into your startup file for recent FSF Emacs (.emacs) or XEmacs (.xemacs/init.el). As always you can also try it out quickly by pasting into your *scratch* buffer and using C-j to evaluate the defvar, the defun, and define-key expressions in turn. Once the above has been evaluated, hit F10 to rotate among the named background colors.

The code's pretty trivial, so if you know Elisp --- or even if you don't --- it should be relatively straightforward to customize it for different colors, or a different keystroke. To list all the color names your display supports, use M-x list-colors-display.

I considered making the macro save the background color rotation between Emacs invocations, so that each new instance would automatically come up with a different background color. It wouldn't be too hard to do this, but I decided it didn't fit the way I work. I like to have a particular color correspond to a particular task --- "aliceblue" for general hacking (the first Emacs I open when I login), "thistle" for writing notes to myself, etc. --- so I prefer to do the rotation manually.

Incidentally, note the use of string arguments to defvar and defun to document the function and variable. Some Python advocates tout Python's "innovative" and "unique" use of string literals in class and function bodies for documentation comments. I agree that it's a clever idea, but it predates Python by several decades. There's a reason that Emacs is called the "extensible, self-documenting display editor".

P.S. The title of this post declares my intention to post a random section from my Emacs init.el files every week until I run out of things to describe. My startup Elisp has been on the web for ages, but I think that having a well-titled blog post for individual tweaks will make better Google-food, and will therefore be more useful to the world.