Helm Bibtex

** Predefined searches

For convenience, frequent searches can be captured in commands and bound to key combinations. Below is example code that defines a search for publications authored by “Jane Doe” and binds the search command to C-x p.

Helm-bibtex:

#+BEGIN_SRC elisp (defun helm-bibtex-my-publications (&optional arg) "Search BibTeX entries authored by “Jane Doe”.

With a prefix ARG, the cache is invalidated and the bibliography reread." (interactive "P") (helm-bibtex arg nil "Jane Doe"))

;; Bind this search function to Ctrl-x p: (global-set-key (kbd "C-x p") 'helm-bibtex-my-publications) #+END_SRC

Ivy-bibtex:

#+BEGIN_SRC elisp (defun ivy-bibtex-my-publications (&optional arg) "Search BibTeX entries authored by “Jane Doe”.

With a prefix ARG, the cache is invalidated and the bibliography reread." (interactive "P") (when arg (bibtex-completion-clear-cache)) (bibtex-completion-init) (ivy-read "BibTeX Items: " (bibtex-completion-candidates) :initial-input "Jane Doe" :caller 'ivy-bibtex :action ivy-bibtex-default-action))

;; Bind this search function to Ctrl-x p: (global-set-key (kbd "C-x p") 'ivy-bibtex-my-publications) #+END_SRC

** Change the available actions :PROPERTIES: :CUSTOM_ID: change-actions :END:

Pressing on a publication triggers the “default action” which is opening the PDF associated with the publication, if present, or its URL or DOI otherwise. Pressing in helm-bibtex or M-o in ivy-bibtex instead displays an action menu listing the available actions. Here is the list of all available actions along with their functions (these are the generic action functions, for helm-bibtex the function names start with helm-bibtex- instead of bibtex-completion-, and for ivy-bibtex they start with ivy-bibtex- instead):

• Open PDF, URL or DOI: bibtex-completion-open-any
• Open PDF file (if present): bibtex-completion-open-pdf
• Open URL or DOI in browser: bibtex-completion-open-url-or-doi
• Insert citation: bibtex-completion-insert-citation
• Insert reference: bibtex-completion-insert-reference
• Insert BibTeX key: bibtex-completion-insert-key
• Insert BibTeX entry: bibtex-completion-insert-bibtex
• Attach PDF to email: bibtex-completion-add-PDF-attachment
• Edit notes: bibtex-completion-edit-notes
• Show entry: bibtex-completion-show-entry
• Add PDF to library: bibtex-completion-add-pdf-to-library

Helm-bibtex: The action list can be modified through the commands helm-add-action-to-source and helm-delete-action-from-source. For instance, the following adds a new action helm-bibtex-open-annotated-pdf (see [[#annotated][above]]) just after the first item in the list above:

#+BEGIN_SRC emacs-lisp (helm-add-action-to-source "Open annotated PDF (if present)" 'helm-bibtex-open-annotated-pdf helm-source-bibtex 1) #+END_SRC

If the last, numerical argument in helm-add-action-to-source is omitted, the new action is added at the end of the list. Since the default action is simply the first entry in the list of actions, the default action can be changed by deleting an action and re-inserting it at the top of the list. Below is an example showing how to make “Insert BibTeX key” the default action:

#+BEGIN_SRC emacs-lisp (helm-delete-action-from-source "Insert BibTeX key" helm-source-bibtex) (helm-add-action-to-source "Insert BibTeX key" 'helm-bibtex-insert-key helm-source-bibtex 0) #+END_SRC

Ivy-bibtex: The default action and the additional available actions are set separately. The default action is controlled by the variables ivy-bibtex-default-action and ivy-bibtex-default-multi-action, with the latter intended for lists of marked entries (see [[#apply-actions-to-multiple-entries][Apply actions to multiple entries]]). For example, the following code changes the default action to "insert BibTeX key":

#+BEGIN_SRC emacs-lisp (setq ivy-bibtex-default-action 'ivy-bibtex-insert-key) #+END_SRC

In the same way, the following code sets the default action for lists of marked entries to "insert BibTeX key" which will insert a nice comma-separated list of keys:

#+begin_src emacs-lisp (setq ivy-bibtex-default-multi-action 'ivy-bibtex-insert-key) #+end_src

The additional actions are set by passing the desired action list to the command ivy-set-actions. For instance, the following codes keeps only two available actions in addition to the default one:

#+BEGIN_SRC emacs-lisp (ivy-set-actions 'ivy-bibtex '(("p" ivy-bibtex-open-any "Open PDF, URL, or DOI" ivy-bibtex-open-any) ("e" ivy-bibtex-edit-notes "Edit notes" ivy-bibtex-edit-notes))) #+END_SRC

The letters p and e are the key bindings for the two actions in the action menu. The key binding o is reserved for the default action. The second appearance of the action in this code alerts ivy that the action can handle lists of marked entries. It can safely be omitted if the right thing to do is simply apply the action to each entry in turn.

If you only want to add new actions at the end of the action list, you can alternatively use the command ivy-add-actions. For instance, the following adds a new action ivy-bibtex-open-annotated-pdf (see [[#annotated][above]]) at the end of the action list:

#+BEGIN_SRC emacs-lisp (ivy-add-actions 'ivy-bibtex '(("P" ivy-bibtex-open-annotated-pdf "Open annotated PDF (if present)" ivy-bibtex-open-annotated-pdf))) #+END_SRC

** Create new actions :PROPERTIES: :CUSTOM_ID: create-actions :END:

Creating a new action for helm-bibtex or ivy-bibtex can be done in three steps. For an example see [[#annotated][Action for opening annotated PDFs]] above.

The first and main step is to create a generic action function bibtex-completion- (e.g. bibtex-completion-open-annotated-pdf). This function should take as single argument a list of BibTeX keys and perform the action on the corresponding BibTeX entries.

The second step is to tailor the generic action function for helm-bibtex or ivy-bibtex, so that it will be run in the correct buffer and receive the keys of the selected entries).

Helm-bibtex: This is simply done with:

#+BEGIN_SRC emacs-lisp (helm-bibtex-helmify-action bibtex-completion- helm-bibtex-) #+END_SRC

Ivy-bibtex: This is simply done with:

#+BEGIN_SRC emacs-lisp (ivy-bibtex-ivify-action bibtex-completion- ivy-bibtex-) #+END_SRC

The third and final step is to make the tailored action function helm-bibtex- or ivy-bibtex- available in helm-bibtex or ivy-bibtex by adding it to the action menu. See [[#change-actions][Change the available actions]].

** Window size

Helm-bibtex: By default helm-bibtex uses the entire frame to display the bibliography. This can be changed by setting the variable helm-bibtex-full-frame to nil, in which case helm’s standard is used (typically vertical split, with the helm search being shown in the lower window).

Ivy-bibtex: Ivy-bibtex always displays the bibliography in the minibuffer. The variable ivy-height controls the number of lines for the minibuffer window in all ivy commands.

** Templates for new notes :PROPERTIES: :END:

Bibtex-completion populates new notes with some basic information about the publication. In the case of just one note file for all publications, new entries look like the following example:

#+BEGIN_EXAMPLE

• Gigerenzer, G. (1998): We need statistical thinking, not statistical rituals :PROPERTIES: :Custom_ID: Gigerenzer1998 :END: #+END_EXAMPLE

The title of the new section consists of the author names, the year, and the title of the publication. The property Custom_ID specifies the BibTeX key of the entry (it’s named Custom_ID for compatibility with org-ref).

In the case of one file per publication, a new notes file contains a title in the following format:

#+BEGIN_EXAMPLE #+TITLE: Notes on: Gigerenzer, G. (1998): We need statistical thinking, not statistical rituals #+END_EXAMPLE

If other formats are desired, the templates for new notes can be changed using the customization variables bibtex-completion-notes-template-one-file and bibtex-completion-notes-template-multiple-files.

** File type of note files

By default bibtex-completion assumes that note files are in org-mode format. However, any other format can be used as well. In the case of just one notes file, it is enough to set bibtex-completion-notes-path to point to the desired file. In the case of multiple note files, the type of the files can be specified using the customization variable bibtex-completion-notes-extension. For example, if Markdown is the desired file type:

#+BEGIN_SRC emacs-lisp (setq bibtex-completion-notes-path "/path/to/notes") (setq bibtex-completion-notes-extension ".md") #+END_SRC

If the file type is set to something else than org-mode, the templates for new note files need to be adjusted as well. See the section above for details.

** Refresh bibliography when new PDFs and notes are added Bibtex-completion automatically reloads the bibliography when a .bib file is changed on disk. However, bibtex-completion does not watch PDFs and notes. Thus, when a new PDF or note is added, the bibliography must be manually reloaded to display the PDF and note symbols correctly (via prefix argument, e.g. C-u M-x helm-bibtex). Unfortunately, implementing automatic reloading for PDFs and notes is not entirely straightforward since bibtex completion is quite flexible in how PDFs and notes can be handled. But for simple setups, there is an easy solution: just watch the bibtex-completion-library-path and bibtex-completion-notes-path directories and reload the bibliography when they change. Example for the PDF directory:

#+BEGIN_SRC emacs-lisp (setq tmalsburg-pdf-watch (file-notify-add-watch bibtex-completion-library-path '(change) (lambda (event) (bibtex-completion-candidates)))) #+END_SRC

• Usage :PROPERTIES: :CUSTOM_ID: usage :END: ** Search publications

Use M-x helm-bibtex or M-x ivy-bibtex to start a new search. The default fields for searching are: author, title, year, BibTeX key, and entry type. Regular expressions can be used. Example searches:

Everything published by Janet Fodor:

#+BEGIN_EXAMPLE janet fodor #+END_EXAMPLE

All PhD theses:

#+BEGIN_EXAMPLE phdthesis #+END_EXAMPLE

Lyn Frazier's PhD thesis:

#+BEGIN_EXAMPLE phdthesis frazier #+END_EXAMPLE

Publications about eye tracking. A regular expression is used to match various spellings (“eyetracking”, “eye tracking”, “eye-tracking”):

#+BEGIN_EXAMPLE eye.?tracking #+END_EXAMPLE

Conference presentations in 2013:

#+BEGIN_EXAMPLE 2013 inproceedings #+END_EXAMPLE

Publications from 2010 and 2011:

#+BEGIN_EXAMPLE (2010|2011) #+END_EXAMPLE

Articles co-authored by David Caplan and Gloria Waters:

#+BEGIN_EXAMPLE article waters caplan #+END_EXAMPLE

Search for articles by David Caplan that are /not/ co-authored by Gloria Waters:

#+BEGIN_EXAMPLE article caplan !waters #+END_EXAMPLE

** Search in the local bibliography

Use helm-bibtex-with-local-bibliography or ivy-bibtex-with-local-bibliography to start a search in the current buffer's "local bibliography", instead of the "global bibliography" defined by bibtex-completion-bibliography. If the current file is a BibTeX file, that bibliography is going to be used. If the current file is a LaTeX file, reftex will be used to determine the local bibliography from the standard LaTeX bibliography commands \bibliography and \addbibresource. If the file is an org file, the local and/or global org bibliography is used (as specified using the new #+BIBLIOGRAPHY: key word and the variable org-cite-global-bibliography). If no local bibliography can be found, the global bibliography (bibtex-completion-bibliography) will be used.

** Search in entries with notes

Use helm-bibtex-with-notes or ivy-bibtex-with-notes to search only among entries that have notes. Particularly useful in combination with [[https://github.com/zaeph/org-roam-bibtex][org-roam-bibtex.el]].

** Search the word under the cursor

A common use case is where a search term is written in a document (say in your LaTeX manuscript) and you want to search for it in your bibliography. In this situation, just start helm-bibtex or ivy-bibtex and enter M-n. This inserts the word under the cursor as the search term. (This is a helm / ivy feature and can be used in all helm / ivy commands, not just helm-bibtex / ivy-bibtex.) Note that it is also possible to use BibTeX keys for searching. So if your cursor is on a BibTeX key (e.g., in a LaTeX cite command) you can start helm-bibtex or ivy-bibtex, hit M-n and see the entry associated with that BibTeX key. Special case: you want to open the PDF associated with the BibTeX key under the cursor: M-x helm-bibtex M-n RET or M-x ivy-bibtex M-n RET. This is of course shorter if you bind helm-bibtex or ivy-bibtex to a convenient key (see [[#key-bindings][Key-bindings]]).

** Actions for selected publications

The available actions are:

• Open a PDF if present, or a URL or DOI (default action)
• Open the URL or DOI in browser
• Insert citation
• Insert reference
• Insert BibTeX key
• Insert BibTeX entry
• Attach PDF to email
• Edit notes
• Show entry
• Add PDF to library

Helm-bibtex: Select an entry and press to execute the default action. Alternatively, press TAB (tabulator) to see a list of all available actions, execute one of them and exit helm-bibtex.

Ivy-bibtex: Select an entry and press to execute the default action. Alternatively, press M-o to see a list of all available actions, execute one of them and exit ivy-bibtex.

** Apply actions to multiple entries :PROPERTIES: :CUSTOM_ID: apply-actions-to-multiple-entries :END:

Helm-bibtex: Start helm-bibtex, enter the search expression, move the cursor to the matching entry and enter C- (control + space bar) to mark this entry, optionally change your search expression, mark more entries, finally press or to execute an action for all selected entries at once and exit helm-bibtex.

Ivy-bibtex: Start ivy-bibtex, enter the search expression, move the cursor to the matching entry and enter C- (control + space bar) to mark this entry, optionally change your search expression, mark more entries, finally press to execute the default action on all the selected entries or M-o to choose another action. Press S- (shift + space bar) to un-mark a marked entry.

** A colleague asks for copies of your new papers

Helm-bibtex: Start an email to your colleague (C-x m) and execute helm-bibtex. Search for your new publications and mark them with C-, then press to execute the action “Attach PDF to email”. Then M-x helm-resume (the publications are still marked) and press to execute the action “Insert BibTeX entry”. Optionally insert more human readable references using M-x helm-resume and to execute the action “Insert reference”. Send email (C-c C-c). Done. This takes less than 10 seconds.

Ivy-bibtex: Start an email to your colleague (C-x m) and execute ivy-bibtex. Search for your new publications and and mark them with C-, then press C-M-o a to execute the action “Attach PDF to email” while keeping ivy open. Then press M-o b to execute the action “Insert BibTeX entry” or insert more human readable references using M-o r to execute the action “Insert reference”. Send email (C-c C-c). Done. This takes less than 10 seconds.

Of course, this assumes that you’re sending email from Emacs, e.g. via [[http://www.djcbsoftware.nl/code/mu/mu4e.html][Mu4e]].

** Tag publications

Helm-bibtex and ivy-bibtex have powerful search capabilities but some common searches cannot be performed simply because the relevant information is typically not represented in BibTeX files. For example, bibtex-completion doesn’t know whether a conference presentation was a talk or a poster because both are represented as inproceedings. So if you want to compile a list of your conference talks (e.g., for your CV), that’s not possible – not without some additional work. One solution is to “tag” publications. Tags are like keywords except that they don’t represent the content of a publications but meta data. Example:

#+BEGIN_SRC bibtex @inproceedings{BibtexKey2015, author = {Jane Doe and Monika Mustermann}, title = {This is the title}, crossref = {XYZ-conference-2015}, keywords = {keyword1, keyword2}, pages = {10}, tags = {poster}, } #+END_SRC

Since tags is not a standard BibTeX field, bibtex-completion by default doesn’t consider it when searching. In order to be able to search for tags, we therefore have to tell bibtex-completion that the tags field is relevant, too:

#+BEGIN_SRC elsip (setq bibtex-completion-additional-search-fields '(tags)) #+END_SRC

There are many other ways in which tags can be used. For example, they can be used to mark articles that you plan to read or important articles or manuscripts in progress, etc. Be creative.

** Insert LaTeX cite commands :PROPERTIES: :CUSTOM_ID: latex-cite :END:

The action for inserting a citation command into a LaTeX document prompts for the citation command and, if applicable, for the pre- and postnote arguments. The prompt for the citation command has its own minibuffer history, which means that previous inputs can be accessed by pressing the key for helm-bibtex or M-p for ivy-bibtex. By pressing it is also possible to access the list of all citation commands defined in biblatex (except for multicite commands and volcite et al. which have different argument structures). The prompt also supports auto-completion via the tab key. If no command is entered, the default command is used. The default command is defined in the customization variable bibtex-completion-cite-default-command. By default, helm-bibtex and ivy-bibtex prompt for pre- and postnotes for the citation. This can be switched off by setting the variable bibtex-completion-cite-prompt-for-optional-arguments to nil.

** Force reloading of the bibliography

Bibtex-completion caches the bibliography to prevent a costly reread when a new query is started. However, bibtex-completion does not check whether new PDFs or notes were added since the last read and hence the symbols indicating the presence or absence of these items may be incorrect. A reread can be forced using a prefix argument.

Helm-bibtex: Either do C-u M-x helm-bibtex or C-u followed by whatever key binding you use to invoke helm-bibtex.

Ivy-bibtex: Either do C-u M-x ivy-bibtex or C-u followed by whatever key binding you use to invoke ivy-bibtex.

** Import BibTeX from CrossRef

Helm-bibtex: Start helm-bibtex and enter search terms. Then select “CrossRef” in the section titled “Fallback options”. (You can use the left and right arrow keys to switch between sections.)

Ivy-bibtex: Start ivy-bibtex and enter search terms. Then press M-o f to see the list of fallback options and select "CrossRef".

This will use [[https://github.com/cpitclaudel/biblio.el][biblio.el]] to search the CrossRef database. In the results list, place the cursor on the entry of interest and hit c to copy the BibTeX for that entry or i to insert it at point. Press q to close the buffer with the search results. See the [[https://github.com/cpitclaudel/biblio.el/blob/master/README.md][documentation of biblio.el]] for details.

*** Searching on a different source Sometimes the search term will not yield the desired results on the first fallback source selected and you may wish to pick another fallback option with the same search term. For this you can use helm-resume (or ivy-resume) to get back to the initial helm (ivy) menu with the last search term pre-entered allowing you to efficiently choose another option.

• Advanced usage (a.k.a. hacks)

Below I provide code that was useful for me or other users. Note that this code may make assumptions that do not hold in your setup. Read the code carefully before executing it and make changes as needed.

** Convert multiple note files to one notes file

The code below reads all note files in your bibtex-completion-notes-path and creates a new notes file containing a section for each publication. This code assumes that bibtex-completion is still configured for multiple note files and that you want to store the notes in the file notes.org in your bibtex-completion-notes-path. The code also adds a level to all org headlines found in the individual note files (because top-level headings are used for the publications in the new notes file). If a note file doesn't have a corresponding entry in the bibliography, it is ignored.

#+BEGIN_SRC elisp (let ((note-files (directory-files bibtex-completion-notes-path t "^[^.]+\.org$")) (bibtex-completion-notes-path (f-join bibtex-completion-notes-path "notes.org"))) (cl-loop for note-file in note-files for key = (f-no-ext (f-filename note-file)) do (condition-case nil (progn (bibtex-completion-edit-notes key) (insert (with-temp-buffer (insert-file-contents note-file) (replace-regexp "^*" "**") (buffer-string)))) (error nil)))) #+END_SRC

** Create a BibTeX file containing only specific entries

Say you want to create a BibTeX file containing only entries that you cited in an article, then you can use the following code to populate the new BibTeX file with entries:

#+BEGIN_SRC elisp (progn (switch-to-buffer (generate-new-buffer "my_new_bibliography.bib")) (--map (insert (bibtex-completion-make-bibtex it)) (-distinct '("Key1" "Key2")))) #+END_SRC

If LaTeX is used to write the article, grep and sed can be used to extract the cited keys:

#+BEGIN_SRC sh grep '\entry{' manuscript.bbl | sed 's/^.\entry{([^}])}.*$/\1/' #+END_SRC

** Create a list with the paths of all PDFs in your bibliography

This can be useful if you’d like to transfer all your PDFs to another directory or similar.

#+BEGIN_SRC elisp (flatten-tree (mapcar (lambda (entry) (bibtex-completion-find-pdf entry)) (bibtex-completion-candidates))) #+END_SRC

** Reverse order of entries

Helm-bibtex and ivy-bibtex display entries in the order in which they appear in the BibTeX file reversed. This way, entries that were added at the bottom of the BibTeX file show up at the top when searching. There is currently no support for sorting but if you want to reverse the order of entries you can use the code below:

#+BEGIN_SRC elisp (advice-add 'bibtex-completion-candidates :filter-return 'reverse) #+END_SRC

** Use ivy-bibtex as an org-cite-follow-processor

As mentioned [[https://github.com/tmalsburg/helm-bibtex#follow-processor-for-helm][here]], the default org-open-at-point on a org citation will take you to the corresponding bibliography entry. The following code will change this behavior to instead open ivy-bibtex when following an org citation by entering RET or clicking on it:

#+BEGIN_SRC elisp (org-cite-register-processor 'my-ivy-bibtex-org-cite-follow :follow (lambda (_ _) (ivy-bibtex)))

(setq org-cite-follow-processor 'my-ivy-bibtex-org-cite-follow) #+END_SRC

• Troubleshooting

** Helm-bibtex doesn’t show any entries

This usually happens when a BibTeX file isn’t well-formed. Common problems are opening quotes or parentheses that don’t have matching counterparts. Unfortunately, Helm swallows the error message that is generated in these cases and just shows an empty buffer.

One way to diagnose the problem is to call the function for reading BibTeX directly and to see what error message it produces:

#+BEGIN_SRC elisp (bibtex-completion-candidates) #+END_SRC

If you see

#+BEGIN_SRC forward-sexp: Scan error: "Unbalanced parentheses", 181009, 512282 #+END_SRC

this means that there is an unmatched opening parenthesis at the position 181009. To find this parenthesis, open the BibTeX file and do M-: (goto-char 181009) RET. You can also use the command M-x bibtex-validate RET to check for errors. Fix any problems and try again.