Moving blog to ox-hugo

It’s refresh time! Since January 2019 I’ve been blogging on this website and it’s been a rewarding experience. I originally researched many blogging solutions that involved using Emacs Org mode. I settled with org-static-blog and it worked pretty well for the most part. It gave just enough structure and mechanism to enable me to work on authoring content without dealing with the details of website building.

Enter 2021, I have used org-static-blog for a couple years, and detailed some of my challenges with it below. I re-researched solutions for org-mode web publishing and found that ox-hugo + hugo was surprisingly easy to use, once you get over the initial hurdles. It also addressed many of the issues I was encountering. Spoiler: I definitely recommend it to other org-mode users.

§Challenges with org-static-blog

§Soft forking org-static-blog is hard

I didn’t like the rapid direction of change in project, with it breaking my workflow every month or so, so I decided to mitigate those rapid changes by maintaining a soft-fork. For the most part, this translated into merging changes from upstream every couple months, testing, and fixing the changes. This was also a good place to test my personal changes and contribute back changes.

Looking at the Git history I made the following changes, some are marked with upstreamed if they made it upstream:

C-u org-static-blog-publish RET to force render

upstreamed

Add more disgusting CDATA escapes

upstreamed

Add postamble to every page

closed, but it worked well enough for my own uses, so I kept in in my soft-fork. More on this below.

After the PR rejection, I just maintained my soft-fork for awhile, occasionally merging in upstream changes. After some time, a big PR was merged in that that restructured how org-static-blog was organized. These changes made maintaining my soft-fork untenable. At this point, I decided to just maintain my copy of org-static-blog separately. It effectively was a dead copy, but it worked well enough for my needs.

Putting aside the fork maintenance issues, there are some usability issues with org-static-blog I hope to address.

§Messy Git worktree

Previously I’d just commit the published HTML files in the same repository as the org-mode posts. I also kept drafts in there, but had to move them into the publish-posts area when previewing. My git work tree tended to get very dirty. I frequently forgot to stage/unstage the HTML output. This led to a lot of fixup commits.

§Hyperlinking is cumbersome

Hyperlinking to local content was a bit tricky. You had to either symlink in your static directory or manually copy-paste paths relative to the publish directory into your org-mode document. Even if you symlink in, and therefore could tab-complete paths, linking to other posts (HTML documents) was nontrivial.

§Org Export HTML is quirky

org-static-blog uses org html export, where controlling the html formatting is not very easy. At one point I wanted to use Tachyons to style org-mode HTML documents, but promptly gave up realizing there is no easy way to add CSS utilities to the exported HTML documents. Additionally, I wanted to add more styling or change the layout and it was always a hack. Just look at my old css file. It’s not very maintainable, even for such a simple website!

When exporting syntax highlighting, the colors are derived from your current emacs setup, so a dark-mode syntax highlighting setup will conflict with a light-mode website, vice versa. See the following screenshot for an example.¹

Finally, none of the automatic page anchors were stable. Every time a document was re-exported the anchors would change. This means internal page anchors could not be used reliably when hyperlinking. There are hacks to work around this, but it didn’t feel like it would work in every situation.

§Migrating the content

Instead of requiring each post be in a separate org-mode document, ox-hugo treats each top-level node as a different post. This simplifies authoring new content, as there is just the one file to deal with. It also reflects the common practice of using few, large org-mode documents. I have to move posts each with their own org-mode document into a single org-mode file.

My initial plan was to script it in elisp, such that I just let it run, and check the result. I could have done this, but figured if it took more than 3 hours, I might be wasting my time, because I only have 20 posts. 180 minutes decided 20 posts is 9 minutes per post. I am very thankful my collection isn’t very large, as I would have to write some more robust elisp and do a great deal more testing, then automate it.

Much to the reader’s disdain, I went the manual route. It turned out it was super simple. On the old site, every post was in its own file in the /posts/ directory. In the new site, ever post would be a top-level node in content-org/all-posts.org. I had to fix the location of include directives and copy images into a new directory, otherwise all I had to do was the following steps per document:

1. Open the org-mode file on the old site in posts/.
2. Every post had a #+title: header on the top line. So I copied the title via C-s SPACE RET C-SPACE C-e M-w. (Search for the first space, exit search, mark to end of line, copy that text.)
3. Create a new top-level heading in the all-posts.org file, then type C-y to yank.
4. Type C-c C-c on the newly created heading to set the tags, e.g. C-c C-c computing:gentoo RET to set the tags to computing and gentoo.
5. Type C-x C-c p EXPORT_FILE_NAME RET cool_post_filename_here RET to set the export filename.
6. Type C-x C-c p EXPORT_DATE RET 2021-03-21T13:00:00-05:00 RET to set the post time.
7. Switch back to the old post file, then run C-u 11 M-x winny/increment-footnotes RET to renumber the footnotes starting at 12. This would allot me to simply copy-paste footnotes without redoing the references. (See source at end of this section).
8. Put mark towards the beginning of the document (M-<) then run M-% static/image RET images RET to fix the image urls.
9. Increase the depth of org-mode notes in the old document (so there are no level 1 headings).
10. Copy from the first paragraph to * Footnotes and paste under the new heading in the new document.
11. Copy the old document’s * Footnotes content under the new document’s * Footnotes heading.

The above manual steps seem a bit much, but I estimate there are about 50-100 keystrokes per document. It’s not that many for such a big task. There is a lot of room to automate this with macros or more emacs lisp, but given I’d only do this once, and I just want to get it done as quickly as possible, it didn’t make sense to try to automate it.

§Source for winny/increment-footnotes

(defun winny/increment-footnotes (count)
  "Increment all footnote numbers in buffer by `COUNT'."
  (interactive "p")
  (unless count
    (setq count 1))
  (save-excursion
    (goto-char (point-min))
    (while (re-search-forward "\\[fn:\\([0-9]+\\)\\]" nil t)
      (message "m")
      (replace-match (number-to-string (+ count (string-to-number (match-string 1))))
                     nil nil nil 1))))

§Retaining the RSS URL

My old site used /rss.xml for the main RSS document. Hugo uses /index.xml by default. One can add the following lines to their config.toml to configure this (source):

[outputFormats.RSS]
baseName = "rss"  # use `rss.xml' instead of `index.xml'.

§Retaining Permalinks from org-static-blog

Dead links are the specter of the internet. I’d prefer to allow old org-static-blog URLs to redirect to their updated locations. It turns out GitLab pages supports a _redirects file, similar to how Netlify does it. I simply dropped a _redirects file into the public/ directory that is built by GitLab’s CI. It contains lines like the following:

/2021-03-14-some-tips-when-copying-recovering-disks.html /posts/some-tips-when-coping-recovering-disks/ 301
/2021-02-14-you-want-sudo--i-or-su--.html /posts/you-want-sudo-i-or-su-dash/ 301
/2020-11-03-about-my-keyboard-choices.html /posts/about-my-keyboard-choices/ 301

A fun tip from tejr:

I usually start with 302, test the redirect, and then switch it to 301.

The idea behind this advice is your browser will always retry a URL that had given a 302 response (temporary redirect), but will probably not retry a URL that had given a 301 response (permanent redirect). I think this is especially important when editing a live website’s redirects — you don’t want to accidentally tell the entire world your site belongs at a mistaken URL redirect.

§Creating the theme

I have virtually no control over HTML layout on the old blog, I could customize the <head>, content preamble, and content postable elements, but not much else. This meant to get the most of the styling out of the old blog system, I’d simply tack on more CSS themes, with lots of clever selectors. Even with these limitations I really enjoyed the theme I created. So I invested some effort into creating a likable theme for Hugo.

§Iteration #1: Tachyons and hugo-bare-min-theme

This part took a long time. My first iteration used Tachyons and was based off hugo-bare-min-theme — the theme used for the ox-hugo test website. While one has a great deal of control over HTML output in Hugo, the exact markup of exported Markdown is not easy to control. As a result, it wasn’t possible to use Tachyons utility CSS classes on things like Markdown paragraphs, so the CSS started looking like a mess, much like my older website’s CSS. Below is a screenshot of this theme:

§Iteration #2: Tailwind CSS and hugo-theme-tailwindcss-starter

I asked around and everyone kept telling me to try Tailwind CSS, especially with the @apply and customize configuration parameters. I was very skeptical at first, because adding additional tooling to build webpages is a rabbit hole. It tends to end up with the entire project being difficult to maintain. After some extensive searching, I found a usable Hugo theme — hugo-theme-tailwindcss-starter — that integrated TailwindCSS in way that did not require any special gotchas. Simply npm install some tools, then run hugo server. Here is a screenshot of my current theme. It’s themes all the way down!

§TailwindCSS versus Tachyons

Some nice things about TailWindCSS over Tachyons:

• My stylesheet size went from about 70 KB (Tachyons) down to 13 KB (TailwindCSS). The original TailWindCSS stylesheet is over 8 MB, but after minification and CSS selector purging, it is a mere 13 KB.
• Using @apply and tailwind.config.js it becomes trivial to apply CSS to HTML fragments I cannot directly add Utility CSS classes on. There is no analogue in Tachyons.
• It is trivial to add custom color classes and other things via tailwind.config.js. It is possible to do this in Tachyons, but requires manually adding selectors to each css include that uses the color variables. In Tailwind CSS I just add a single line to tailwind.config.js and the CSS utility classes are added without any additional work.

§Integration into GitLab CI

I used GitLab CI with org-static-blog to automatically update this website. I generated the HTML files using M-x org-static-blog-publish RET and commit them to the Git repository. On git push Gitlab would simply deploy the HTML files from Git. I didn’t feel confident enough in my org-static-blog to run emacs in batch mode within GitLab CI, thereby generating the HTML files outside of the git repository.

With Hugo and ox-hugo, one simply runs a single hugo command. Because I added in a dependency on npm packages, I just had to roll my own ci.sh script that ran on the alpine Docker image, but it really was pretty simple:

#!/bin/sh
set -eu
apk add hugo git npm
cd themes/winny2-theme
npm i
cd -
ln -s themes/winny2-theme/node_modules node_modules
npm install -g postcss-cli autoprefixer

hugo

The symlink was necessary to ensure npm knew about the installed node_modules from the project root directory. See also: the current ci.sh and its .gitlab-ci.yml.

§The publishing workflow

When working with Hugo + ox-hugo, I start up hugo in the root repository directory using hugo serve --disableFastRender and point a browser towards http://localhost:1313/ . Then visit ./content-org/all-posts.org, creating a new top-level node at the very bottom, just before the Footnotes node. Be sure to add the mandatory property EXPORT_FILE_NAME (via C-c C-x p EXPORT_FILE_NAME RET your-file-name RET). ox-hugo uses this to determine the markdown filename for export. After authoring the post, export the org file to Hugo markdown using C-c C-e H A. This will output markdown files according to the hugo_base_dir setting at the top of your org-mode document. In my case it exports to the content/ directory relative to this document’s parent directory, (i.e. ../content/).

Voilà! You’ll see the post in your browser. Hugo will automatically reload the webpage as well. To mark the post as a draft, simply mark the top-level node as a TODO (using C-c C-t). It won’t be published when one runs hugo, but you can still export it to markdown, and preview it locally. Once happy with the content, simply run git add content content-org && git commit && git push. Wait a moment, and your content will be live thanks to CI/CD.

§Closing remarks

This migration could have a greater “cool factor” with some more automation, but I am pleased I was able to do a complete migration within a weekend. Going forward the website should be a lot easier to update because org-static-blog and Org HTML Export won’t get in the way of doing things. Additionally the theme and HTML layout is directly customizable now, instead of just piling in a lot of CSS to fix bad layout.

After kicking the tires with TailwindCSS, I can see why everyone is so enamored with it. It works very well, is simple to augment, and certainly feels like the natural successor to Tachyons.

So far so good. I don’t know if this new publishing system will stick. This is my second post; insofar it has been a lot easier to use than the old system. I’ve enjoyed the amount of customization available in Hugo and the modularity of the process. Stay tuned for any hiccups :).

§Thanks

Thanks to bard for spotting grammatical errors in the post.