Travis CI integration for emacs packages

Sep. 10, 2017

This post will show how to add simple make-based testing support for running automated emacs `ert` tests.

The following utilities will be available on the development machine:

`make update` will install the development dependencies
`make compile` will compile the .el files
`make test` will run the `ert` tests
`make clean` will remove the compiled files

The Travis build will fail with an error when:

a compilation warning or error occurs
an automated test fails

This will be the resulting directory structure, where `<my-package>.el` is the hypothetical package we’d like to test:

.
├── .travis.yml         ;; Travis CI config
├── .elpa               ;; contains installed deps
├── Makefile            ;; shortcuts to test/make-*.el
├── <my-package>.el     ;; package being tested
└── test
    ├── elpa.el         ;; initialize package.el
    ├── tests.el        ;; automated tests
    ├── make-compile.el ;; compile *el files
    ├── make-test.el    ;; run automated tests
    └── make-update.el  ;; install dependencies

These files have to be modified, the rest can be copied as is:

`test/make-compile.el` contains the dev dependencies of the package
`test/tests.el` contains the automated tests

The rest of the files don’t need to be modified. However, if needed, they can easily be changed since each one is small, simple, serves one purpose, thus easy to tweak.

.travis.yml

This file is the entry point for Travis CI.

# .travis.yml
sudo: true
dist: precise
language: emacs-elisp
env:
  matrix:
    - emacs=emacs-snapshot

before_install:
  - sudo add-apt-repository -y ppa:ubuntu-elisp
  - sudo apt-get update -qq
  - sudo apt-get install -qq $emacs

script:
  - make update
  - make compile
  - make test

Makefile

The Makefile is used for nothing but shortcuts to running the tasks.

update:
	emacs -batch -l test/make-update.el

compile: clean
	emacs -batch -l test/elpa.el -l test/make-compile.el

test:
	emacs -batch -l test/elpa.el -l test/make-test.el

clean:
	rm -f *.elc

.PHONY: update compile test clean

test/elpa.el

Initializes package.el.

(setq package-user-dir
      (expand-file-name (format ".elpa/%s/elpa" emacs-version)))
(package-initialize)
(add-to-list 'load-path default-directory)

test/make-compile.el

This file compiles `*.el` files in the package root directory.

;;  bail out on compilation warnings and errors
(setq byte-compile-error-on-warn t)
(setq byte-compile--use-old-handlers nil)

;; compile *.el files
(dolist (file (file-expand-wildcards "*.el"))
  (unless (byte-compile-file file)
    (kill-emacs 1)))

test/make-test.el

This file runs the tests in `tests/tests.el`.

(let* ((project-tests-file "tests.el")
       (current-directory (file-name-directory load-file-name))
       (project-test-path (expand-file-name "." current-directory))
       (project-root-path (expand-file-name ".." current-directory)))

  ;; add the package being tested to 'load-path so it can be 'require-d
  (add-to-list 'load-path project-root-path)
  (add-to-list 'load-path project-test-path)

  ;; load the file with tests
  (load (expand-file-name project-tests-file project-test-path) nil t)

  ;; run the tests
  (ert-run-tests-batch-and-exit))

test/make-update.el

This file installs dependencies in the `.elpa` directory.

The `dev-packages` variable should be modified per the package’s needs. This example adds the `evil` and `evil-test-helpers` packages as dependencies for illustrative purpose.

;; list of the all the dependencies, including the dev dependencies
(defvar dev-packages '(evil evil-test-helpers))

;; initialize package.el
(setq package-user-dir
      (expand-file-name (format ".elpa/%s/elpa" emacs-version)))
(message "installing in %s ...\n" package-user-dir)
(package-initialize)
(setq package-archives
      '(("melpa" . "http://melpa.org/packages/")
        ("gnu" . "http://elpa.gnu.org/packages/")))
(package-refresh-contents)

;; install dependencies
(dolist (package dev-packages)
  (unless (package-installed-p package)
    (ignore-errors
      (package-install package))))

;; upgrade dependencies
(save-window-excursion
  (package-list-packages t)
  (condition-case nil
      (progn
        (package-menu-mark-upgrades)
        (package-menu-execute t))
    (error
     (message "All packages up to date"))))

test/tests.el

This file contains the unit tests for `my-package`, the package being tested. This example tests a hypothetical function `my-package-add-numers`.

(require 'ert)
(require 'my-package)

(ert-deftest sample-test ()
  (ert-info ("test function my-package-add-numers")
    (should (eq 3 (my-package-add-numers 1 2))

.gitignore (optional)

.elpa/
*.elc

The described approach is simple in the sense that it doesn’t add any dependencies to the package, other than `make`. Everything else is included with emacs - package.el, ert.el, etc.

The obvious disadvantage is the wordiness - this method involves multiple files.