For Your Reference: RefTeX in Org Mode

Cross-references in Org mode are not as well developed as its (relatively) new citation system. Org’s built-in linking system is fairly comprehensive and exports to all formats well enough. But if you’re coming to Org from LaTeX, you might prefer something more familiar and oriented towards the label/references mental mapping than the anchor/link system. Over the decades, I drifted into Org mode from writing LaTeX, and brought along RefTeX support. Sort of.

In the past few months I received a number of questions about my usage of LaTeX-style cross-references in Org, so here’s a short explanation and, uh, reference.

Just give me the code

One half of my long-winded Emacs articles begin or end with a call to action:

…and it’s a package, and here’s a link, it’s on MELPA, try it out.

This is not one of those. There is a package, and more code besides. But this is stuff extracted from my Emacs init file, which is a cross between a palimpsest, a crime scene and a dry-erase board that don’t erase no more. Use at your own peril, or better yet, reuse these ideas to make something coherent and plug-and-play:

1. consult-reftex: A better interface to RefTeX, with Consult
2. reftex-xref: xref and eldoc support for RefTeX
3. Some configuration to turn on these enhancements.

Demos and explanations below.

Background: org-link, org-ref and oxr

There are other solutions – actual solutions – for cross-references in Org mode. Org’s built-in linking system can be repurposed for this, and it will export to all document formats. Bruce D’Arcus’ oxr is a proof of concept of a cross-referencing system built on top of Org’s links. For LaTeX exports, there’s John Kitchin’s frighteningly full-featured org-ref, with simple and elegant cross-referencing syntax.

One of these is probably what you want.

One of these is probably what I would have used if I had given my slow transition from LaTeX to Org any active thought. Instead I just continued to use RefTeX.

What’s a RefTeX?

RefTeX is an Emacs minor mode with distinct support for \ref, \label, \cite, and \index commands in (multi-file) LaTeX documents. It is included with Emacs.

The official description undersells it. RefTeX is extensive: it supports cross-references and citations, semi-automatic label creation, reference tracking and updates, indexes, multi-file documents, references to external documents, a dynamic table of contents and more, and provides the means to do everything that we’re about to do more clumsily below.

In a happy coincidence, RefTeX works without a hitch in Org documents… to a point It gets dicey with multi-file Org documents. . This is an oft-ignored advantage of regexp-based, structure-agnostic parsing – once you iron out the many, many edge cases over a few years, it tends to work pretty much everywhere. RefTeX’s Org mode “support” is good enough for my purposes, and having it work the same in Org and LaTeX documents is a bonus.

It even integrates with cdlatex and org-cdlatex, with automatic label insertion when inserting environments. (This is not a coincidence.)

There are two limitations.

1. Obviously, LaTeX style cross-references only work with LaTeX exports.

2. The other issue with RefTeX, and the main focus of this write-up, is the UI. It’s very… 90s, let’s say. Better previewing, faster navigation, integration into the modern Emacs API, some DWIM behavior – these are all achievable with a few band-aids. The result is a patchwork, but that’s most things Emacs.

Finally, there’s one other deep connection between RefTeX, CDLaTeX and Org mode. It’s two words. Take your guesses, or open up reftex.el and read the header!

Sprucing up the RefTeX experience

A reassertion: RefTeX already works reasonably well in Org mode, and you can work around edge cases if you encounter them. The write-up until now was thus mainly a PSA. What follows is minor improvements to the UI and the general experience of Org when using RefTeX.

Insert references with consult-reftex

As mentioned above, RefTeX is a kitchen sink library – it provides everything you need to manage references, including a multi-step reference insertion menu. But this process leaves a bit to be desired:

I wrote consult-reftex a while ago to provide a better interface for reference creation: The LaTeX previews in the buffer, in consult-reftex’s previews and elsewhere are from Org’s upcoming LaTeX preview system overhaul.

I wrote it for LaTeX documents, but thanks to RefTeX’s flexibility it works all the same in Org mode.

Looking up references: consult-reftex (again)

The preview interface for consult-reftex serves a second purpose – you can use it as a listing of the equations in your document (by label), and act on labels via Embark.

I don’t do this when writing, but it’s occasionally useful when reviewing work.

Fontify…

RefTeX labels in Org are plain by default,

so a little syntax highlighting goes a long way:

(font-lock-add-keywords
   'org-mode
   '(("\\(\\(?:\\\\\\(?:label\\|ref\\|eqref\\)\\)\\){\\(.+?\\)}"
      (1 font-lock-keyword-face)
      (2 font-lock-constant-face))))

I didn’t bother making a comprehensive regexp for font-locking here, you might want to thrown in more keywords.

This is more than eye-candy: a little further below, we use the faces we specify here as anchors for jumping between references. If you use different faces for the keywords, you’ll need to update the navigation code below.

…and Prettify: TeX-fold-mode

TeX-fold-mode, provided with AucTeX, is useful for shortening long references by placing overlays over them.

Before:

After (with the mouse over one of the references):

The effect is purely visual, moving the cursor into the overlay reveals the full reference:

This requires turning on TeX-fold-mode.

It can fold most LaTeX constructs, including environments, sections, comments and even individual math symbols and operators. We’re only interested in folding references and labels here, though, so we can specify this:

(setq TeX-fold-type-list '(macro))

You can make this Org mode-specific with a hook if you want to fold more constructs in LaTeX documents.

Optional: we can tweak the overlay styling for labels and refs as well:

Reference/Label folding style: code

;; Faces
(set-face-attribute 'TeX-fold-folded-face nil :foreground nil :inherit 'shadow)
;; Custom folded display for labels and refs
(defun my/TeX-fold-ref (text)
  (let* ((m (string-match "^\\([^:]+:\\)\\(.*\\)" text))
         (cat (or (match-string 1 text) ""))
         (ref (or (match-string 2 text) text)))
    (when (> (length ref) 13)
        (setq ref (concat (substring ref 0 6) "..." (substring ref -6))))
    (concat "[" (propertize cat 'face 'shadow) ref "]")))
(defun my/TeX-fold-label (&rest texts)
  (cl-loop for text in texts
           for m = (string-match "^\\([^:]+:\\)\\(.*\\)" text)
           for cat = (or (match-string 1 text) "")
           for ref = (or (match-string 2 text) text)
           collect (concat "[" (propertize cat 'face 'shadow) ref "]") into labels
           finally return (mapconcat #'identity labels ",")))
(setq-default TeX-fold-macro-spec-list
              '((my/TeX-fold-label ("cite"))
                (my/TeX-fold-label ("label"))
                (my/TeX-fold-ref ("ref" "pageref" "eqref" "footref"))))

Lastly, you might want to turn folding on in Org buffers. TeX-fold uses the C-c C-o key as a prefix, watch out for key conflicts with Org!

(add-hook 'org-mode-hook 'TeX-fold-mode)

Moving around references: big ol’ pile o’hacks

This is mostly to avoid having to use the mouse – Isearch or Avy aren’t very useful if the reference is folded. That said, its main advantage is not that it moves the cursor (for editing), but that it enables other actions we might want to take immediately after, such as jumping to the definition of a label with xref.

We want several things to happen when we jump to a reference:

• Org should open up folded/invisible text if the references is in a hidden region,
• Any TeX-fold overlay at the reference should be cleared,
• and if we jumped from a reference it should be folded again by TeX-fold.
• If a preview is available (via Eldoc, see below), it should be activated.

The code for this is below. It’s not pretty.

Jumping by reference: code

(defun my/next-reference-or-label (_arg)
    (interactive "p")
    (let* ((prop))
      (pcase-let
          ((`(,_ . ,ov)
           (get-char-property-and-overlay (point) 'TeX-fold-type)))
        (when ov (TeX-fold-hide-item ov)))
      (save-excursion
        (and (setq prop (text-property-search-forward
                         'face nil
                         (lambda (_ val)
                           (memq val '(font-lock-constant-face org-cite)))
                         t))))
      (if prop
          (progn (goto-char (prop-match-beginning prop))
                 (when (and (derived-mode-p 'org-mode) (org-invisible-p))
                   (org-fold-show-context 'link-search))
                 (when eldoc-mode (eldoc--invoke-strategy t))
                 (pcase-let
                     ((`(,_ . ,ov)
                      (get-char-property-and-overlay (point) 'TeX-fold-type)))
                   (when ov (TeX-fold-show-item ov))))
        (message "No more references/labels."))))

(defun my/previous-reference-or-label (_arg)
   (interactive "p")
   (let ((p))
     (save-excursion
       (and (text-property-search-backward
             'face nil
             (lambda (_ val)
               (memq val '(font-lock-constant-face org-cite
                           TeX-fold-folded-face)))
             t)
            (setq p (point))))
     (pcase-let
         ((`(,_ . ,ov)
           (get-char-property-and-overlay (point) 'TeX-fold-type)))
       (when ov (TeX-fold-hide-item ov)))
     (when p
       (goto-char p)
       (when (and (derived-mode-p 'org-mode) (org-invisible-p))
         (org-fold-show-context 'link-search))
       (when eldoc-mode (eldoc--invoke-strategy t)))
     (pcase-let
         ((`(,_ . ,ov)
           (get-char-property-and-overlay (point) 'TeX-fold-type)))
       (when ov (TeX-fold-show-item ov)))))

While we’re at it, we might as well make this command repeatable, so you can navigate with just r and R after the first jump:

(keymap-set org-mode-map "M-g r" 'my/next-reference-or-label)
(keymap-set org-mode-map "M-g R" 'my/previous-reference-or-label)
(defvar-keymap my/TeX-ref-map
  :repeat t
  "r" 'my/next-reference-or-label
  "R" 'my/previous-reference-or-label)

Jumping to definition: xref support, more hacks

With the cursor at a reference, jump to the corresponding label using Emacs’ standard xref mechanism:

Code below. Again, not pretty.

xref support for RefTeX: code

(require 'xref)
(require 'reftex-ref)

(cl-defmethod xref-backend-identifier-at-point ((_backend (eql reftex)))
  (when (looking-back "\\\\\\(?:page\\|eq\\|auto\\|c\\)?ref{[-a-zA-Z0-9_*.:]*"
                                      (line-beginning-position))
		    (reftex-this-word "-a-zA-Z0-9_*.:")))

(cl-defmethod xref-backend-definitions ((_backend (eql reftex)) prompt)
  (unless (symbol-value reftex-docstruct-symbol) (reftex-parse-one))
  (when-let* ((docstruct (symbol-value reftex-docstruct-symbol))
              (data (assoc prompt docstruct))
              (label (nth 0 data))
              (file  (nth 3 data))
              (buffer (or (find-buffer-visiting file)
                          (reftex-get-file-buffer-force
                           file (not reftex-keep-temporary-buffers))))
              (re (format reftex-find-label-regexp-format (regexp-quote label)))
              (found (with-current-buffer buffer
                       (or (re-search-backward re nil t)
                           (progn (goto-char (point-min))
                                  (re-search-forward
                                   (format reftex-find-label-regexp-format2
                                           (regexp-quote label))
                                   nil t))))))
    (list (xref-make prompt (xref-make-buffer-location
                             buffer found)))))

(cl-defmethod xref-backend-apropos ((_backend (eql reftex)) pattern)
  (xref-backend-definitions 'reftex pattern))

(defun reftex-xref ()
  "Function to activate buffer-local backend.
Add this function to `xref-backend-functions' to use xref to find
function and variable definitions in the same buffer.

This should have a low priority among xref backends."
  'reftex)

;; Eldoc support via xref, because why not
(defun reftex-xref--display-in-eldoc (callback)
  "Display reference label location in Eldoc.

Call CALLBACK if possible."
  (when (cl-intersection
         (ensure-list (get-char-property (point) 'face))
         '(font-lock-keyword-face font-lock-constant-face
           TeX-fold-unfolded-face))
    (save-excursion
      (when-let*
          ((inhibit-redisplay t)
           (_macro (car (reftex-what-macro-safe 1)))
           (key (reftex-this-word "^{}%\n\r, \t"))
           (item (car (xref-backend-definitions 'reftex key)))
           (location (xref-item-location item))
           (pos (xref-buffer-location-position location))
           (docstring
            (with-current-buffer (xref-buffer-location-buffer location)
              (goto-char pos)
              (pcase-let
                  ((`(,start . ,end)
                    (condition-case nil
                        (LaTeX-find-matching-begin)
                      (:success
                       ;; Found \begin{}...\end{}
                       (cons (point)
                             (progn (forward-char 1)
                                    (LaTeX-find-matching-end)
                                    (point))))
                      (error
                       ;; No \begin, find \section{} or Org heading instead
                       (goto-char pos)
                       (re-search-backward
                        (pcase major-mode
                          ('org-mode "^*")
                          (_ (concat "\\(" (LaTeX-outline-regexp)
                                     "\\|\\`\\)")))
                        (line-beginning-position -2)
                        t)
                       (cons (line-beginning-position)
                             (line-end-position))))))
                (buffer-substring-no-properties start end)))))
        (if-let* ((prop-and-ov
                   (get-char-property-and-overlay
                    (xref-buffer-location-position location)
                    (pcase major-mode
                      ('org-mode 'org-overlay-type)
                      ('latex-mode 'category))))
                  (_ (memq (car prop-and-ov)
                           '(org-latex-overlay preview-overlay)))
                  (ov (cdr prop-and-ov)))
            (funcall callback (propertize docstring 'display
                                          (overlay-get ov 'preview-image)))
          (funcall callback docstring))))))

(defun reftex-eldoc-activate ()
  (add-hook 'eldoc-documentation-functions
            'reftex-xref--display-in-eldoc
            nil :local))

(defun reftex-xref-activate ()
  "Activate reftex support using xref."
  (add-hook 'xref-backend-functions
            'reftex-xref nil :local))

(provide 'reftex-xref)

To turn on xref support for references in Org mode:

(use-package reftex-xref
  :after (org latex)
  :hook (org-mode . reftex-xref-activate))

Auto-preview reference at point: eldoc support

If you’re jumping to label definitions with xref just to look at the corresponding equations, we can skip that step with previews of the referenced object. With eldoc-mode Eldoc is Emacs’ standard and recommended way of showing contextual help. enabled:

Equations and figures can be pretty tall, so if you’re not using a dedicated Eldoc buffer (or eldoc-box) you might want to change the height that the echo area can expand to.

(setq eldoc-echo-area-use-multiline-p t
      max-mini-window-height 0.4)

To activate reference previews as an Eldoc documentation function, we use the helper function defined as part of reftex-xref above.

(use-package reftex-xref
  :after (org latex)
  :hook (org-mode . reftex-eldoc-activate))

Yep, reftex-dcr exists

Earlier I mentioned that we would be clumsily recreating features RefTeX already includes. Indeed, most of the above features, like jumping to label definitions and previewing the reference at point, are already provided in the reftex-dcr library, included with RefTeX. The problem is that it predates Emacs’ xref and eldoc APIs (if not their libraries), and like most 90s Emacs packages it reinvents all the wheels. This is painful on more than an aesthetic level – I prefer to have one well-supported and convenient way for each semantic task. The mental model is simpler, as are the required keyboard gymnastics.

Thankfully, there’s enough of a separation between the data-parsing RefTeX backend and the UI that I was able to drive a wedge through and beat it into a more consistent shape. Needless to say, this shape is not very robust. Emacs makes it very easy to cobble together an 80% solution – I spent much longer writing this little explainer than I did actually slapping together the code. Due to a pies ≫ fingers situation, I’m not going to be able to get these hacks to a stage beyond “it works for me”. The interest around this topic suggests that perhaps someone might be interested in developing the idea further and modernizing RefTeX, or working on a better cross-referencing system for Org mode. Have at it!