Different Strokes For Different Folks

So, Emacs and the mouse. This is an unexpectedly contentious topic, with discussions that end, at best, with careless dismissal. More often they turn into arguments with folks talking past one another.

The advantages of using the mouse for common actions in Emacs are immediate and obvious. Window selection is a natural extension of basic mouse usage. Resizing windows is a snap. Context (right-click) menus See context-menu-mode. and drag and drop support, which improve with each new Emacs release, are very intuitive. The mouse provides access to basic editor functions with no learning curve, and no exhortations to work through the Emacs tutorial are necessary. Also, feature discoverability via Emacs’ menu-bar is surprisingly good.

Unfortunately, I have to address the rodent in the room carefully before we can talk about mitigating the many disadvantages of mouse-first interaction, since Emacs users tend to be very opinionated about this topic. Ergonomics, speed, and efficiency are three common reasons people advocate for the keyboard against the mouse for non-graphical work. I’m not convinced any of these arguments hold water. The first two are research-worthy topics on their own, with many quotable precedents, and beyond my ability to argue for or against convincingly just from personal experience Besides, anyone who’s had bouts of RSI has made up their mind already on this topic. . On the matter of speed, for instance, I am obliged to include this quote because y’all are going to send me emails linking to it otherwise:

We’ve done a cool $50 million of R & D on the Apple Human Interface. We discovered, among other things, two pertinent facts:

Test subjects consistently report that keyboarding is faster than mousing.

The stopwatch consistently proves mousing is faster than keyboarding.

– Bruce Tognazzini

So let’s pretend for the purpose of today’s discussion that we’re not concerned about the difference in editing speed one way or another.

In the context of Emacs specifically, there are further disadvantages. For example, mouse actions compose in limited ways (such as when dragging to form selections) and cannot build on each other like key sequences can with prefix keys and transient maps. You can’t automate mouse usage with keyboard macros easily.

But today we’re primarily going to address the third reason: efficiency.

I find the arguments about efficiency, in the sense of poor expressivity or economy of motion, shortsighted at best. Here are two facets of this argument, where “mouse” includes any kind of pointing device.

Economy:

Frequent context switching between the keyboard, mouse and keyboard+mouse is wasteful and causes low amounts of constant friction.

Expressivity:

Mouse usage has a low ceiling: it makes easy tasks trivial but involved actions impossible. Adding buttons or widgets to the screen doesn’t scale.

Economy and interprogram contexts

Economy first. I actually agree with the context switching argument, to the extent that the friction of repeated switching feels progressively more annoying. But the conclusion – it’s best to avoid the mouse to avoid context switching – is shortsighted because it only applies if you’re only ever in Emacs and never have to use the mouse, or conversely if you’re never in a text editor and always in a mouse-driven (or keyboard+mouse driven) application like a 3D modeling tool or a big commercial IDE. Most of us are somewhere in the middle.

I’m as keyboard-centric as you can reasonably get: besides Emacs, I use a tiling window manager, a keyboard-driven web browser and mostly TUI applications. Keybindings are the topic I’ve written the most about on this website – an embarrassing amount of yakshaving devoted to an inane subject. Even so, I use the mouse frequently because it’s often the best way to select text, images and links, do graphical work, and to interact with (both simple and complex) GUI software.

When working on something that involves Emacs and one of these mouse driven applications, having to move over to the keyboard for the Emacs bit is exactly the friction we’d like to avoid. The other app may not be flexible, but Emacs is! So we don’t need to switch to the Emacs-typical both-hands-at-keyboard context. Here are a few examples of quick interprogram interaction involving Emacs that you can easily do without moving your hand off the mouse.

Pasting text from other applications into a new buffer

Switch to a scratch buffer+window in Emacs to paste some text:

I use a mouse gesture in Emacs to create a scratch buffer in Org mode – courtesy of the scratch package – then drag some text into it Incidentally, the text is a LaTeX environment that is rendered automatically via Org’s new LaTeX preview system. . I then switch back to the previous buffer using another mouse gesture.

git-clone from a repository URL

Middle click to paste a link from the clipboard. (This is only for clarity, it’s unrelated to the actual action.) Then use a mouse gesture to git-clone from that link to a prespecified directory.

open a project directory in dired so we can drag and drop a file

Use a mouse gesture to bring up a list of directories in the active project. Pick one – also using the mouse – then drag an image file into dired before switching back.

mpv + live video transcript in Emacs

Control mpv (a video player) by clicking around in a live transcript file in Emacs:

No mouse gestures here – the clickable transcripts and mpv connection are provided by elfeed-tube, which was designed to be usable with the mouse since it involves interacting with a video player.

My counter-argument can be summarized: Use the lowest effort means of interaction in each context you work in. Often this means driving Emacs with the mouse.

Expressivity

The expressivity counterargument: Buttons and widgets indeed don’t scale, but they bear the additional burdern of discoverability. Mouse interactions in Emacs don’t have to be prescribed or discoverable – when they are actions we set up, typically after repeatedly observing a cumbersome task that could be mouse-driven. Point-and-click in an editor is not as expressive as full-on keyboard usage, but that’s only true in the generic sense. The relevant question is if we can express the specific verbs that we care about, and from the above demos it’s clear we can.

An example: considering that using the mouse to select and resize windows is already a more natural experience than spamming C-x o and friends (or your evil-mode equivalents), it is possible to extend it to be a more comprehensive approach to window and buffer management:

This demo showcases the use of mouse gestures to do the following:

• Split the frame vertically and horizontally
• Delete windows
• Cycle through buffers in windows by left and right clicking the buffer name in the mode line.
• Delete other windows
• Swap windows to the right and left
• Toggle between the last two buffers shown in a window
• Go back to a previous window configuration (via tab-bar-history-back or winner-undo)

How to mouse around

This is not, strictly speaking, an article on how to use the mouse in Emacs – it’s more about why and when. For the how I suggest reading Philip Kaludercic’s write-up. You could also try just clicking around, especially in the mode line! However the use of gestures in Emacs rarely gets much attention, so here we go.

For once, this is going to be a short description, since this is a batteries included situation. Emacs ships with everything you need.

Adding mouse gesture support to Emacs is easy with the included Strokes library Many thanks to David Bakhash and the Emacs maintainers for strokes.el. . For each action you want to map to a gesture,

1. Find or (optional) write a command that does the thing.
2. Set it to a mouse stroke (gesture) with strokes-global-set-stroke.

That’s it M-x strokes-help has more information. .

Optionally, you could change the mouse button you hold down to perform a gesture. The default is S-mouse-2, which is rather awkward. I use a side button on my mouse (typically the “forward” button):

(keymap-global-set "<down-mouse-9>" 'strokes-do-stroke)

Also optionally, you can hide the live-display of the stroke as you draw (strokes-use-strokes-buffer) – I left it in for clarity in the demos.

I’ve not had to customize anything else about the strokes library, but feel free to explore – it has support for multi-glyph (complex) strokes, for instance.

There are a few general mouse settings you might be interested in:

• mouse-autoselect-window: Set to t for focus-follows-mouse behavior inside Emacs. Typically what you’d want if you use mouse gestures.
• mouse-drag-and-drop-region: To enable text drag-and-drop within Emacs,
• mouse-drag-and-drop-region-cross-program: (Emacs 29+) and across programs from Emacs.
• dired-mouse-drag-files: To drag files from dired to other programs.

Improving Strokes

Mouse gestures are a decades-old concept, as many features included with Emacs are. And the Strokes library has a list of notes and todos, as many features included with Emacs do.

1. No contextual actions: The use of strokes is limited, as of Emacs 29, by the lack of support for mode-specific or buffer-local strokes. This means the same gesture cannot do different things in different contexts without writing cumbersome, bespoke dispatch commands to handle this.
2. Somewhat unreliable gesture matching: Drawing with the mouse or trackpad is fuzzy and imprecise. The matching algorithm is quite rudimentary and there are mismatches when you have many strokes defined (> 25, let’s say), even with an increased “grid” resolution when drawing. Fixing 1 can take the pressure off the matching algorithm, but we could also use a more sophisticated approach to matching.
3. No multi-touch gestures: This can increases the dimensionality of the gesture space, allowing for a greater number of “basic strokes” – although the OS usually captures and maps most of these for its window management.

Despite these limitations, this is a very handy tool. I’m not sure why gestures aren’t more common – we only see them on touch interfaces these days. You can draw them anywhere on the screen, and they tolerate a fair bit of fuzz. They avoid the problem of having to move the mouse over to a small target on screen. Not having to fill the screen up with buttons is a win by itself. The only significant problem appears to be that they’re not (re)discoverable, but this isn’t a problem inside Emacs: we have strokes-describe-stroke (like describe-key) and strokes-list-strokes.

It’s not all or nothing. It’s not even something.

Some of the “talking past each other” I mentioned in the beginning is because of assumptions we make about using the mouse in text editors. We gravitate to one of two specious requirements:

• The mouse should substitute for the keyboard, such as when using menu or toolbar buttons instead of keybindings.
• The mouse should complement the keyboard, so we work best with a hand each on the keyboard and mouse. (This is the Acme editor approach.)

Actual usage patterns are messier than either of these. Sometimes we’re working in a mouse-heavy context, sometimes it’s two handed keyboarding, sometimes its a mix. Often the editor is at the periphery of the task at hand, sometimes it’s dead center This is true despite Emacs swallowing up my desktop applications. Calendar, email, reading, some web browsing… the list keeps growing. . Any degree of flexibility on the part of the editor is welcome. Instead of dismissing the mouse on the basis of the available design affordances, it can help to think about what we actually care to avoid – frequent context switching for me, your answer will be different – and how driving Emacs with the mouse helps us do that.