Emacs is an insanely liberating hacking environment, many of the intricate systems designed for it help me organize and automate the different facets of my life. A key part of the ecosystem is the collection of Org Mode libraries which cover many things from exporting documents and helping me keep track of habits to an intuitive citation system that can be combined with other integral tools, like the magnificent citeproc.
When I initially started using Emacs, it was to distract myself from my old Vim habits and to learn something new. I was comfortable with my tools, my ways and ideologies, but it was a couple of presentations that helped me realize the vast difference between how I go about accomplishing a task and how others are doing it.
I'm currently leveraging ox-publish, an immensely extensible and highly documented facility of Org Mode, to build and architecture this website, a website whose first commit happened around seven months ago. I've added lots of tiny features since then: preference-based colorschemes, syntax highlighting, citations, and a tiny module that helps me navigate my blog post directory a lot more efficiently, to name a few.
This has also been my first attempt at abstracting the codebase of a website so that it can span multiple sub-projects, like liaison and darkman.el, both of which are independent projects whose source code exists outside of this website's source tree.
One major pain point to my current system was dependency/packagement
management, an area which can make or break a programming
language. Emacs 29 thankfully added a new
which we can use to specify a custom
the packages we install from say, a publishing script, will not
interfere with the user's general environment.
To give you an example, we can consider citeproc a dependency of this project, the way we'd go about handling this dependency follows this general ruleset:
- Configure the project to use a separate initialization directory
- Install the dependency at the beginning of the publishing script
- Require the library like any other
To address the first point, we might consider using a Makefile which will help us streamline the publishing process.
build: emacs --quick --init-directory=.emacs.d --script publish.el --funcall org-publish-all
build recipe is invoked, the emacs batch processor will
create a new
.emacs.d directory adjacent to the
Makefile and use
that to store the source code of the packages we'll install later on.
We're also loading a file named
publish.el whose purpose is to
handle the publishing aspects of the project, this interacts with
ox-publish to configure how the project should turn out when it is
Since the packages we'd like to use aren't not available by default, we'll have to go out and fetch them like we would any other third-party package. This involves some preparation which we can easily abstract into two main functions: the first takes care of initializing the package archives while the second installs the packages provided in a list.
(require 'package) (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t) (defun op-package--initialize () "Initialize the package manager and its archives." (package-initialize) (unless package-archive-contents (package-refresh-contents))) (defun op-package-install (packages) "Install the list of PACKAGES." (op-package--initialize) (dolist (pkg packages) (unless (package-installed-p pkg) (package-install pkg))))
Let's put our functions to use, we'll install citeproc – and htmlize, because why not?
(op-package-install '(citeproc htmlize))
We'll jump a couple lines down, and require the two libraries, actually we won't require htmlize as it's sort of an exception, that's because ox-publish will require it when it needs it.
This next part, which you may or may not care about, configures
citeproc as a citation processor for the
csl backend and
specifically the IEEE citation style.
(require 'oc) (require 'oc-csl) (setq org-cite-global-bibliography (list (expand-file-name "assets/refs.bib")) org-cite-csl-styles-dir (expand-file-name "assets/csl/styles") org-cite-csl-locales-dir (expand-file-name "assets/csl/locales") org-cite-export-processors '((html . (csl "ieee.csl")) (latex . biblatex) (t . simple)))
That's pretty cool and all, but we should get back to the build
process, it is after all what ties everything together. Let's go take
build target we just wrote for a spin and see what it generates.
Importing package-keyring.gpg... Contacting host: melpa.org:443 Package refresh done Contacting host: elpa.gnu.org:443 Package refresh done Contacting host: elpa.nongnu.org:443 Package refresh done ... Checking /home/grtcdr/projects/grtcdr.tn/.emacs.d/elpa/citeproc-20230125.1818... Done (Total of 26 files compiled, 2 skipped) Package ‘citeproc’ installed. ... Checking /home/grtcdr/projects/grtcdr.tn/.emacs.d/elpa/htmlize-20210825.2150... Done (Total of 1 file compiled, 2 skipped) Package ‘htmlize’ installed.
The packages we require should have been installed and placed, along
with their dependencies, in their new separate environment. We can
verify this by inspecting the contents of the
relative to the root of our project – let's do that:
.emacs.d └── elpa ├── citeproc-20230125.1818 ├── dash-20221013.836 ├── f-20230116.1032 ├── htmlize-20210825.2150 ├── parsebib-20221007.1402 ├── s-20220902.1511 └── string-inflection-20220910.1306
The most crucial procedure in the build process is that of the
org-publish-all function, which thanks to our neat setup, is now
able to build the project with all of the libraries we've added along
Publishing file /home/grtcdr/projects/grtcdr.tn/src/now.org using ‘org-html-publish-to-html’ Publishing file /home/grtcdr/projects/grtcdr.tn/src/contact.org using ‘org-html-publish-to-html’ Publishing file /home/grtcdr/projects/grtcdr.tn/src/index.org using ‘org-html-publish-to-html’ ...
Now, what if a dependency of ours isn't available in any package
archives? How do we tackle this problem efficiently? I think
package-vc, a library introduced in Emacs 29, is the answer.
We can extend the
op-package-install function we previously added
with the capacity to download packages from a repository. First,
import the library somewhere near the top of the publishing script:
package-vc-install would take as its first parameter a URL
to the repository we wish to download, but that's just plain
ugly. Instead, we'll introduce an elegant abstraction that will
receive a tiny bit of information identifying the repository and then
translate that to a functional URL.
We'll start by defining our version control providers, e.g. GitHub, SourceHut, etc. and then create a function that can construct the URL bearing in mind the nuances between the different providers.
(defvar op-package--vc-providers '(:github "https://github.com" :sourcehut "https://git.sr.ht") "Property list of version control providers and their associated domains.") (defun op-package--vc-repo (provider slug) "Construct the base URL of a repository from SLUG depending on the PROVIDER." (let ((domain (plist-get op-package--vc-providers provider))) (cond ((eq provider :sourcehut) (format "%s/~%s" domain slug)) (t (format "%s/%s" domain slug)))))
Although SourceHut prefixes usernames with a tilde, we managed to
offload that formatting task to the
our interactions with
op-package-install can therefore remain
Additionally, it would be beneficial to have
our sole entrypoint to installing packages, wherever they may be. It
does however need to undergo a tiny adjustment for that to be
(defun op-package-install (packages) "Install the list of PACKAGES." (op-package--initialize) (dolist (pkg packages) (if (plistp pkg) (let ((provider (car pkg)) (slug (cadr pkg))) (when (plist-member op-package--vc-providers provider) (let* ((base (op-package--vc-repo provider slug)) (package (intern (file-name-base slug)))) (unless (package-installed-p package) (package-vc-install base :last-release))))) (unless (package-installed-p pkg) (package-install pkg)))))
It's a pretty big transformation, but the function can now take a list
such as this one
'(citeproc htmlize (:github "grtcdr/liaison")),
installing citeproc and htmlize from the usual package archives
and liaison from the repository in which it resides.
I'm currently using this setup to try and deliver a great reading experience for you. These tools have empowered me to create an environment that is unique to me, an environment whose elements are evolving alongside my progression as a writer and developer, and an environment whose components can be easily swapped with one another. It's a place that provides me with an opportunity for continuous research, development and improvement. If you're considering using ox-publish as a foundation for your next project, now's the time.
I'd like to thank Sacha Chua for including this publication in her "Emacs News" posting of February 13, 2023. I'm honored to have been included in her weekly digest and grateful to be a part of this lively community.