Travis CI integration for emacs packages

to run automated tests

This post will show how to add simple make-based testing support for running automated emacs `ert` tests.

The following utilities will be available on the development machine:

  • `make update` will install the development dependencies
  • `make compile` will compile the .el files
  • `make test` will run the `ert` tests
  • `make clean` will remove the compiled files

The Travis build will fail with an error when:

  • a compilation warning or error occurs
  • an automated test fails

This will be the resulting directory structure, where `.el` is the hypothetical package we’d like to test:

├── .travis.yml         ;; Travis CI config
├── .elpa               ;; contains installed deps
├── Makefile            ;; shortcuts to test/make-*.el
├── <my-package>.el     ;; package being tested
└── test
    ├── elpa.el         ;; initialize package.el
    ├── tests.el        ;; automated tests
    ├── make-compile.el ;; compile *el files
    ├── make-test.el    ;; run automated tests
    └── make-update.el  ;; install dependencies

These files have to be modified, the rest can be copied as is:

  • `test/make-compile.el` contains the dev dependencies of the package
  • `test/tests.el` contains the automated tests

The rest of the files don’t need to be modified. However, if needed, they can easily be changed since each one is small, simple, serves one purpose, thus easy to tweak.


This file is the entry point for Travis CI.

# .travis.yml
sudo: true
dist: precise
language: emacs-elisp
    - emacs=emacs-snapshot

  - sudo add-apt-repository -y ppa:ubuntu-elisp
  - sudo apt-get update -qq
  - sudo apt-get install -qq $emacs

  - make update
  - make compile
  - make test


The Makefile is used for nothing but shortcuts to running the tasks.

    emacs -batch -l test/make-update.el

compile: clean
    emacs -batch -l test/elpa.el -l test/make-compile.el

    emacs -batch -l test/elpa.el -l test/make-test.el

    rm -f *.elc

.PHONY: update compile test clean


Initializes package.el.

(setq package-user-dir
      (expand-file-name (format ".elpa/%s/elpa" emacs-version)))
(add-to-list 'load-path default-directory)


This file compiles `*.el` files in the package root directory.

;;  bail out on compilation warnings and errors
(setq byte-compile-error-on-warn t)
(setq byte-compile--use-old-handlers nil)

;; compile *.el files
(dolist (file (file-expand-wildcards "*.el"))
  (unless (byte-compile-file file)
    (kill-emacs 1)))


This file runs the tests in `tests/tests.el`.

(let* ((project-tests-file "tests.el")
       (current-directory (file-name-directory load-file-name))
       (project-test-path (expand-file-name "." current-directory))
       (project-root-path (expand-file-name ".." current-directory)))

  ;; add the package being tested to 'load-path so it can be 'require-d
  (add-to-list 'load-path project-root-path)
  (add-to-list 'load-path project-test-path)

  ;; load the file with tests
  (load (expand-file-name project-tests-file project-test-path) nil t)

  ;; run the tests


This file installs dependencies in the `.elpa` directory.

The `dev-packages` variable should be modified per the package’s needs. This example adds the `evil` and `evil-test-helpers` packages as dependencies for illustrative purpose.

;; list of the all the dependencies, including the dev dependencies
(defvar dev-packages '(evil evil-test-helpers))

;; initialize package.el
(setq package-user-dir
      (expand-file-name (format ".elpa/%s/elpa" emacs-version)))
(message "installing in %s ...\n" package-user-dir)
(setq package-archives
      '(("melpa" . "")
        ("gnu" . "")))

;; install dependencies
(dolist (package dev-packages)
  (unless (package-installed-p package)
      (package-install package))))

;; upgrade dependencies
  (package-list-packages t)
  (condition-case nil
        (package-menu-execute t))
     (message "All packages up to date"))))


This file contains the unit tests for `my-package`, the package being tested. This example tests a hypothetical function `my-package-add-numers`.

(require 'ert)
(require 'my-package)

(ert-deftest sample-test ()
  (ert-info ("test function my-package-add-numers")
    (should (eq 3 (my-package-add-numers 1 2))

.gitignore (optional)


The described approach is simple in the sense that it doesn’t add any dependencies to the package, other than `make`. Everything else is included with emacs - package.el, ert.el, etc.

The obvious disadvantage is the wordiness - this method involves multiple files.

See also:

  • cask - this seems to be a tool designed for this purpose solely. Haven’t tried it yet.
  • evm - a tool which allows installing multiple versions of emacs. Seems entangled with cask, but doesn’t require it. This tool can be used to run the tests against multiple versions of emacs, not sure if it can be achieved without pulling in cask as a dependency