Moving blog to ox-hugo
It’s refresh time! Since January 2019 I’ve been blogging on this website and it’s been a rewarding experience. I originally researched many blogging solutions that involved using Emacs Org mode. I settled with org-static-blog and it worked pretty well for the most part. It gave just enough structure and mechanism to enable me to work on authoring content without dealing with the details of website building.
Enter 2021, I have used org-static-blog for a couple years, and detailed some of my challenges with it below. I re-researched solutions for org-mode web publishing and found that ox-hugo + hugo was surprisingly easy to use, once you get over the initial hurdles. It also addressed many of the issues I was encountering. Spoiler: I definitely recommend it to other org-mode users.
§Challenges with org-static-blog
§Soft forking org-static-blog is hard
I didn’t like the rapid direction of change in project, with it breaking my workflow every month or so, so I decided to mitigate those rapid changes by maintaining a soft-fork. For the most part, this translated into merging changes from upstream every couple months, testing, and fixing the changes. This was also a good place to test my personal changes and contribute back changes.
Looking at the Git history I made the following changes, some are marked with upstreamed if they made it upstream:
- C-u org-static-blog-publish RET to force render
- Add more disgusting CDATA escapes
- Add postamble to every page
- closed, but it worked well enough for my own uses, so I kept in in my soft-fork. More on this below.
After the PR rejection, I just maintained my soft-fork for awhile, occasionally merging in upstream changes. After some time, a big PR was merged in that that restructured how org-static-blog was organized. These changes made maintaining my soft-fork untenable. At this point, I decided to just maintain my copy of org-static-blog separately. It effectively was a dead copy, but it worked well enough for my needs.
Putting aside the fork maintenance issues, there are some usability issues with org-static-blog I hope to address.
§Messy Git worktree
Previously I’d just commit the published HTML files in the same repository as the org-mode posts. I also kept drafts in there, but had to move them into the publish-posts area when previewing. My git work tree tended to get very dirty. I frequently forgot to stage/unstage the HTML output. This led to a lot of fixup commits.
§Hyperlinking is cumbersome
Hyperlinking to local content was a bit tricky. You had to either symlink in your static directory or manually copy-paste paths relative to the publish directory into your org-mode document. Even if you symlink in, and therefore could tab-complete paths, linking to other posts (HTML documents) was nontrivial.
§Org Export HTML is quirky
org-static-blog uses org html export, where controlling the html formatting is not very easy. At one point I wanted to use Tachyons to style org-mode HTML documents, but promptly gave up realizing there is no easy way to add CSS utilities to the exported HTML documents. Additionally, I wanted to add more styling or change the layout and it was always a hack. Just look at my old css file. It’s not very maintainable, even for such a simple website!
When exporting syntax highlighting, the colors are derived from your current emacs setup, so a dark-mode syntax highlighting setup will conflict with a light-mode website, vice versa. See the following screenshot for an example.1
Finally, none of the automatic page anchors were stable. Every time a document was re-exported the anchors would change. This means internal page anchors could not be used reliably when hyperlinking. There are hacks to work around this, but it didn’t feel like it would work in every situation.
§Migrating the content
Instead of requiring each post be in a separate org-mode document, ox-hugo treats each top-level node as a different post. This simplifies authoring new content, as there is just the one file to deal with. It also reflects the common practice of using few, large org-mode documents. I have to move posts each with their own org-mode document into a single org-mode file.
My initial plan was to script it in elisp, such that I just let it run, and check the result. I could have done this, but figured if it took more than 3 hours, I might be wasting my time, because I only have 20 posts. 180 minutes decided 20 posts is 9 minutes per post. I am very thankful my collection isn’t very large, as I would have to write some more robust elisp and do a great deal more testing, then automate it.
Much to the reader’s disdain, I went the manual route. It turned out it was
super simple. On the old site, every post was in its own file in the
directory. In the new site, ever post would be a top-level node in
content-org/all-posts.org. I had to fix the location of
and copy images into a new directory, otherwise all I had to do was the
following steps per document:
- Open the org-mode file on the old site in
- Every post had a
#+title:header on the top line. So I copied the title via
C-s SPACE RET C-SPACE C-e M-w. (Search for the first space, exit search, mark to end of line, copy that text.)
- Create a new top-level heading in the
all-posts.orgfile, then type
C-c C-con the newly created heading to set the tags, e.g.
C-c C-c computing:gentoo RETto set the tags to computing and gentoo.
C-x C-c p EXPORT_FILE_NAME RET cool_post_filename_here RETto set the export filename.
C-x C-c p EXPORT_DATE RET 2021-03-21T13:00:00-05:00 RETto set the post time.
- Switch back to the old post file, then run
C-u 11 M-x winny/increment-footnotes RETto renumber the footnotes starting at 12. This would allot me to simply copy-paste footnotes without redoing the references. (See source at end of this section).
- Put mark towards the beginning of the document (
M-<) then run
M-% static/image RET images RETto fix the image urls.
- Increase the depth of org-mode notes in the old document (so there are no level 1 headings).
- Copy from the first paragraph to
* Footnotesand paste under the new heading in the new document.
- Copy the old document’s
* Footnotescontent under the new document’s
The above manual steps seem a bit much, but I estimate there are about 50-100 keystrokes per document. It’s not that many for such a big task. There is a lot of room to automate this with macros or more emacs lisp, but given I’d only do this once, and I just want to get it done as quickly as possible, it didn’t make sense to try to automate it.
(defun winny/increment-footnotes (count) "Increment all footnote numbers in buffer by `COUNT'." (interactive "p") (unless count (setq count 1)) (save-excursion (goto-char (point-min)) (while (re-search-forward "\\[fn:\\([0-9]+\\)\\]" nil t) (message "m") (replace-match (number-to-string (+ count (string-to-number (match-string 1)))) nil nil nil 1))))
§Retaining the RSS URL
My old site used
/rss.xml for the main RSS document. Hugo uses
by default. One can add the following lines to their
config.toml to configure
[outputFormats.RSS] baseName = "rss" # use `rss.xml' instead of `index.xml'.
§Retaining Permalinks from org-static-blog
Dead links are the specter of the internet. I’d prefer to allow old
org-static-blog URLs to redirect to their updated locations. It turns out
GitLab pages supports a
_redirects file, similar to how Netlify does it. I
simply dropped a
_redirects file into the
public/ directory that is built
by GitLab’s CI. It contains lines like the following:
/2021-03-14-some-tips-when-copying-recovering-disks.html /posts/some-tips-when-coping-recovering-disks/ 301 /2021-02-14-you-want-sudo--i-or-su--.html /posts/you-want-sudo-i-or-su-dash/ 301 /2020-11-03-about-my-keyboard-choices.html /posts/about-my-keyboard-choices/ 301
A fun tip from tejr:
I usually start with 302, test the redirect, and then switch it to 301.
The idea behind this advice is your browser will always retry a URL that had given a 302 response (temporary redirect), but will probably not retry a URL that had given a 301 response (permanent redirect). I think this is especially important when editing a live website’s redirects — you don’t want to accidentally tell the entire world your site belongs at a mistaken URL redirect.
§Creating the theme
I have virtually no control over HTML layout on the old blog, I could customize
<head>, content preamble, and content postable elements, but not much
else. This meant to get the most of the styling out of the old blog system,
I’d simply tack on more CSS themes, with lots of clever selectors. Even with
these limitations I really enjoyed the theme I created. So I invested some
effort into creating a likable theme for Hugo.
§Iteration #1: Tachyons and hugo-bare-min-theme
This part took a long time. My first iteration used Tachyons and was based off hugo-bare-min-theme — the theme used for the ox-hugo test website. While one has a great deal of control over HTML output in Hugo, the exact markup of exported Markdown is not easy to control. As a result, it wasn’t possible to use Tachyons utility CSS classes on things like Markdown paragraphs, so the CSS started looking like a mess, much like my older website’s CSS. Below is a screenshot of this theme:
§Iteration #2: Tailwind CSS and hugo-theme-tailwindcss-starter
I asked around and everyone kept telling me to try Tailwind CSS, especially
@apply and customize configuration parameters. I was very skeptical
at first, because adding additional tooling to build webpages is a rabbit hole.
It tends to end up with the entire project being difficult to maintain. After
some extensive searching, I found a usable Hugo theme —
hugo-theme-tailwindcss-starter — that integrated TailwindCSS in way that did
not require any special gotchas. Simply
npm install some tools, then run
hugo server. Here is a screenshot of my current theme. It’s themes all the
§TailwindCSS versus Tachyons
Some nice things about TailWindCSS over Tachyons:
- My stylesheet size went from about 70 KB (Tachyons) down to 13 KB (TailwindCSS). The original TailWindCSS stylesheet is over 8 MB, but after minification and CSS selector purging, it is a mere 13 KB.
tailwind.config.jsit becomes trivial to apply CSS to HTML fragments I cannot directly add Utility CSS classes on. There is no analogue in Tachyons.
- It is trivial to add custom color classes and other things via
tailwind.config.js. It is possible to do this in Tachyons, but requires manually adding selectors to each css include that uses the color variables. In Tailwind CSS I just add a single line to
tailwind.config.jsand the CSS utility classes are added without any additional work.
§Integration into GitLab CI
I used GitLab CI with org-static-blog to automatically update this
website. I generated the HTML files using
M-x org-static-blog-publish RET
and commit them to the Git repository. On
git push Gitlab would simply
deploy the HTML files from Git. I didn’t feel confident enough in
my org-static-blog to run emacs in batch mode within GitLab CI, thereby
generating the HTML files outside of the git repository.
With Hugo and ox-hugo, one simply runs a single hugo command. Because I added
in a dependency on npm packages, I just had to roll my own
ci.sh script that
ran on the alpine Docker image, but it really was pretty simple:
#!/bin/sh set -eu apk add hugo git npm cd themes/winny2-theme npm i cd - ln -s themes/winny2-theme/node_modules node_modules npm install -g postcss-cli autoprefixer hugo
The symlink was necessary to ensure npm knew about the installed
from the project root directory. See also: the current
ci.sh and its
§The publishing workflow
When working with Hugo + ox-hugo, I start up hugo in the root repository
hugo serve --disableFastRender and point a browser towards
http://localhost:1313/ . Then visit
./content-org/all-posts.org, creating a new top-level node at the very bottom, just
before the Footnotes node. Be sure to add the mandatory property
C-c C-x p EXPORT_FILE_NAME RET your-file-name RET).
ox-hugo uses this to determine the markdown filename for export. After
authoring the post, export the org file to Hugo markdown using
C-c C-e H A.
This will output markdown files according to the
hugo_base_dir setting at the
top of your org-mode document. In my case it exports to the
directory relative to this document’s parent directory, (i.e.
Voilà! You’ll see the post in your browser. Hugo will automatically reload the
webpage as well. To mark the post as a draft, simply mark the top-level node
as a TODO (using
C-c C-t). It won’t be published when one runs
you can still export it to markdown, and preview it locally. Once happy with
the content, simply run
git add content content-org && git commit && git push. Wait a moment, and your content will be live thanks to CI/CD.
This migration could have a greater “cool factor” with some more automation, but I am pleased I was able to do a complete migration within a weekend. Going forward the website should be a lot easier to update because org-static-blog and Org HTML Export won’t get in the way of doing things. Additionally the theme and HTML layout is directly customizable now, instead of just piling in a lot of CSS to fix bad layout.
After kicking the tires with TailwindCSS, I can see why everyone is so enamored with it. It works very well, is simple to augment, and certainly feels like the natural successor to Tachyons.
So far so good. I don’t know if this new publishing system will stick. This is my second post; insofar it has been a lot easier to use than the old system. I’ve enjoyed the amount of customization available in Hugo and the modularity of the process. Stay tuned for any hiccups :).
Thanks to bard for spotting grammatical errors in the post.