Switching website to GitLab Pages

Previously I detailed how I set up blog.winny.tech using GitHub for source code hosting and Caddy’s git plugin for deployment. This works well and I used a similar setup with my homepage. The downside is I host the static web content and I am tied to using Caddy.¹ I imagine simpler is better, so I opted to host my static sites — https://winny.tech/ & https://blog.winny.tech/ — with GitLab pages.

§What’s wrong with Caddy?

Caddy is very easy to get started with, but it has its own set of trade-offs. Over the last few years, I’ve noticed multiple hard-to-isolate performance quirks, some of which were likely related to the official Docker image. In particular, I had built a Docker image of Caddy with webdav support, and the overall performance tanked to seconds per request, even with webdav disabled. I still have no clue what happened there; instrumenting Caddy through Docker appeared nontrivial, so I gave up on webdav support, reverted to my old Docker based setup, and everything was fast, once again.

There is a good amount of inflexibility in Caddy, such as the git plugin’s limitation to deploy to a non-root folder of the web root. And its rewrite logic is usually what you want, but not nearly as flexible as nginx’s.

Asking questions on their IRC is usually met with no response of any kind, which indicates to me that the project’s community isn’t very active.

The move to Caddy v2 is unwelcoming; I don’t want to relearn yet another set of config files and quirks, especially weeding through the layer of configuration file format adapters and the abstracted-away configuration options, so I rather just use Certbot and some other HTTPD that won’t change everything for the fun of it.²

Until recently Caddy experimented with a pretty dubious monetizing strategy. HackerNoon published an article detailing how it worked. In short: they plastered text all over their website claiming you “need to buy a license” to use Caddy commercially, though that claim was never true. Caddy was always covered by Apache License 2.0. Instead, you needed a commercial license in the narrow use-case that your organization wants to use Caddy’s prebuilt release binaries as offered on their website. It is good they stopped this scheme, but it leaves a bad taste with the community, and with me, and discourages me from relying on the project moving forward.

§Why GitLab Pages instead of GitHub Pages?

I have used both GitHub Pages and GitLab Pages in the past. My experience with GitHub Pages is it’s relatively inflexible and difficult to see what is going to be published, and has a CI/CD setup only useful for certain Jekyll based sites. GitLab pages, on the other hand, lets you set up any old Docker-based CI/CD workflow, so it is possible to render a blog with GitLab CI of any static site generating software. The IEEE-CS student chapter I am a part of does just this. We use a combination of static redirect sites and a Pelican-powered static website. There are a large number of example repositories for most of the popular ways to publish a static website, including Gatsby, Hugo, and Sphinx. Needless to say GitLab Pages puts GitHub pages to shame in terms of flexibility.

§Setting up GitLab Pages

There are two steps in setting up GitLab Pages. These are the most important ideas related to GitLab pages; how to navigate the site is something the reader must experience for oneself. Nothing beats experimentation and reading the docs. Make sure to refer to the official GitLab Pages documentation for further details.

§1) Getting GitLab Pages deploy your git repository

Before getting started, make sure GitLab Pages is activated for your project. Visit it via Settings → Pages on your project. Most of the Pages settings are rooted in that webpage.

How GitLab Pages CI/CD deploys your site is specific to your software or lack of software. If you are simply setting up a static website on GitLab Pages, a simple .gitlab-ci.yml will work for you:

pages:
  stage: deploy
  script:
  - mkdir .public
  - cp -rv -- * .public/  # Note the `--'
  - mv .public public
  artifacts:
    paths:
    - public
  only:
  - master

This simply tells GitLab CI/CD to copy everything not starting with a . into the public folder. By the way, one cannot change the public folder path. It does not appear possible to use something like artifacts: paths: ["."] to deploy the entire git repository.

There is a GitLab CI/CD YAML lint website³ (and web API). Additionally, there is a reference documentation for the .gitlab-ci.yml schema. Please note, it will often yield confusing error messages. For example it is invalid to omit a script key, but the error message is Error: root config contains unknown keys: pages. Take the error messages with a grain of salt.

Once you have what seems like the .gitlab-ci.yml that you want, commit it to your git repository, and push to GitLab. Check progress under CI/CD → Pipelines. If everything works out, you should be able to view the website on GitLab Page’s website — e.g. https://winny.tech.gitlab.io/blog.winny.tech. The format of the above url (visible in Settings → Pages) is https://<namespace>.gitlab.io/<project>. If you can’t view your website, check the CI/CD pipeline’s logs, and inspect the artifacts ZIP — which is also available from the CI/CD piplines page. Chances are you need to edit the .gitlab-ci.yml or tweak the scripts used in the YAML file.

§2) Hosting the GitLab Pages site on your (sub-)domain

All the tasks in this section use Settings → Pages using the “New Domain” or “Edit” webpages.

To set up GitLab pages on your domain, you need to first prove ownership of that specific domain via a specially constructed TXT record, then configure that specific domain to point to GitLab Pages via a CNAME or A record. In general I recommend using an A record because you can stuff any other records you please on the same domain.

Simply add an A record on your DNS setup as so: yourdomain.com. A 35.185.44.232.⁴ If everything works, after the DNS updates it can take anywhere from seconds to the rest of your SOA TTL (Time-To-Live). Visiting your domain should now provide a GitLab Pages placeholder page with a 4xx error code.

Next prove to GitLab you own the domain. Create the TXT record as indicated in the GitLab Pages management website. The string to the left of TXT should be the name/subdomain, and the string to the right of TXT is the value. Alternately you can put the entire string into the value field of a TXT record (?!).

Note, the above two sub-steps are independent; one can validate the domain before adding the record to point it to GitLab, and vice versa.

§GitLab Pages Gotchas

There are a few gotchas about GitLab Pages. Some of them are related to GitLab Pages users not being familiar with all of the DNS RFCs. Others are simply because GitLab Pages has quirks too.

§CNAME on apex domain is a no-no

Make sure you do not use a CNAME record on the apex domain. Use an A record instead. Paraphrasing from the ServerFault answer: RFC 2181 clarifies a CNAME record can only coexist with records of types SIG, NXT, and KEY RR. Every apex domain must contain NS and SOA records, hence a CNAME on the apex domain will break things.

§CNAME and TXT cannot co-exist

The above also is true for TXT and CNAME on the same subdomain. For example if one adds TXT somevaluehere and CNAME example.com to the same domain, say hello.example.com, things will not behave correctly.

If we have a look at the GitLab Pages admin page, the language is mildly confusing, stating “To verify ownership of your domain, add the above key to a TXT record within to your DNS configuration.” At first, I thought “somewhere in your configuration” means “place this entire string as the right hand side of a TXT record on any subdomain in your configuration”. This does work, as such I have

blog.winny.tech. IN A   35.185.44.232
blog.winny.tech. IN TXT "_gitlab-pages-verification-code.blog.winny.tech TXT gitlab-pages-verification-code=99da5843ab3eabe1288b3f8b3c3d8872"

But they probably didn’t mean that, Surely I should have this instead:

blog.winny.tech IN A   35.185.44.232
_gitlab-pages-verification-code.blog.winny.tech IN TXT gitlab-pages-verification-code=99da5843ab3eabe1288b3f8b3c3d8872

I feel a bit silly after realizing this is what the GitLab Pages folks intended for me to do, but it really was not clear to me, especially given how when clicking in the TXT record’s text-box it highlights the entire string, instead of allowing the user to copy the important bits (such as the TXT’s key) into whatever web management UI they might be using for DNS.

§The feedback loop for activation of the domain is slow

It can take awhile for a domain to be activated by GitLab Pages after the initial deploy. Things to look for: you should get a GitLab Pages error page on your domain if you set up the CNAME or A record correctly. The error is usually “Unauthorized (401)”, but it can be other errors.

The other place to look is verify your domain is in the “Verified” state on the GitLab Pages admin website.

§The feedback loop for activation of LetsEncrypt HTTPS is huge

Sometimes GitLab pages will seemingly never activate your LetsEncrypt support for HTTPS access. If this happens, a discussion suggests the best solution is to remove that domain from your GitLab Pages setup, and add it again. You will likely have to edit the TXT record used to claim domain ownership. This also worked for me, when experiencing the same issue.

§Make sure to enable GitLab Pages for all users

See this ticket.

§Conclusion

GitLab pages isn’t perfect, but this should streamline what services my VPS hosts, and give me more freedom to fiddle with my VPS configuration and deployment. I look forward to rebuilding my VPS with cdist, ansible, or saltstack. While that happens, my website will be up thanks to GitLab pages. Also, I imagine GitLab Pages is a bit more resilient to downtime than a budget VPS provider.

The repositories with .gitlab-ci.yml files for both this site, and winny.tech are public on GitLab official hosting. Presently it is the simplest setup possible, simply deploying pre-generated content already checked into git, but the possibilities are endless.