Migrating from Emacs 26 to Emacs 27 on Gentoo

Published: Sunday, Nov 28, 2021
Last modified: Sunday, Nov 28, 2021 (5493b8f)
Figure 1: Behold!  Emacs 27!

Figure 1: Behold! Emacs 27!

When upgrading to Emacs 27 there were quite a few weird things I had to address.

My Emacs is installed via Gentoo Portage. The USE flags I have set (to enable/disable features at build time) essentially configure my Emacs to be like Lucid Emacs builds. Here’s the USE flags:

Xaw3d acl alsa athena cairo dbus dynamic-loading gif gmp gui
imagemagick inotify jpeg lcms libxml2 png source ssl svg threads
tiff toolkit-scroll-bars xft xpm zlib -aqua -games -gconf -gfile
-gpm -gsettings -gtk -gzip-el -harfbuzz -json -kerberos -livecd
-m17n-lib -mailutils -motif -selinux -sound -systemd -wide-int
-xwidgets

Make note that cairo support is enabled. This manifests one of the gotchas.

Bitmap font support has been removed

In my ~/.Xdefaults I have the following line:

*font: -*-terminus-bold-*-*-*-*-320-*-*-*-*-*-*

Unfortunately, this causes my Emacs to exit with code 255 after printing the following error:

Font ‘-*-terminus-bold-*-*-*-*-320-*-*-*-*-*-*’ is not defined

Looking in the NEWS file, there is a mention of bitmap support has been removed due to cairo building with pango now.

It turns out setting a catchall Xresources setting to configure font for all old-style applications can cause problems with Emacs which uses Emacs.font resource to configure a font. Insofar the other X applications appear to use *font* only for old-style bitmap fonts.

In order to keep the *font setting, override Emacs.font:

Emacs.font: Mono-12

Now Emacs 27 launches on my workstation.

Quick background. On my workstation I have Gentoo with OpenRC. To set up Emacs Daemon, install app-emacs/emacs-daemon, then create a symbolic link to “instantiate the service” (i.e. cd /etc/init.d && ln -s emacs.winston emacs). Then simply run rc-service emacs.winston start (don’t forget to rc-update add emacs.winston default).

Anyway, I’m not sure went on here, but /etc/init.d/emacs.winston start consistently launched Emacs 26. I did eselect emacs set emacs-27 to configure the emacs and emacsclient programs in my path for Emacs 27. No luck. I also set EMACS in /etc/conf.d/emacs.winston, but that did not work either. No clue what went on here, but the solution was simple - don’t deal with the problem - emerge -avc emacs:26.

emacsclient: can’t find socket

Now Emacs Daemon for Emacs 27 successfully launched, this caused a new problem when running emacsclient -c to create a new frame.

emacsclient: can't find socket; have you started the server?
emacsclient: To start the server in Emacs, type "M-x server-start".
emacsclient: No socket or alternate editor.  Please use:

        --socket-name
        --server-file      (or environment variable EMACS_SERVER_FILE)
        --alternate-editor (or environment variable ALTERNATE_EDITOR)

After a quick search on the Gentoo bug tracker, this bug looked appropriate: #794649: app-emacs/emacs-daemon fails to set the socket location emacsclient 27 expects by default.

This bug references upstream Emacs bugreport: #33847: 27.0.50; emacsclient does not find server socket.

The TL;DR on Gentoo OpenRC does not have XDG_RUNTIME_DIR set when the service is started via rc-service emacs.winston start. Emacs Daemon then proceeds to create the socket file under /tmp. Once I log in to my desktop, elogind sets XDG_RUNTIME_DIR and emacsclient looks there, does not find a socket file, then gives up. In the bugreport there was a possible solution identified to do just that - search for the old socket file in /tmp.

There might be a patch floating around (check Archlinux, Debian, etc), but I wanted to address this upstream issue without a patch if possible. It’s just less work and portable to other systems that don’t patch this surprising behavior. The solution was to create file /etc/profile.d/emacs-daemon.sh with the following contents:

EMACS_SOCKET_NAME="/tmp/emacs$(id -u)/server"
export EMACS_SOCKET_NAME

Log out and log back in to load the new profile.d environment in a new session. emacsclient -c should now work because it will look for the socket file in /tmp.

There was in issue with my Emacs configuration that prevents use-package from being installed on first startup. This appears to be caused by package-refresh-contents never being called and (package-install 'use-package) fails to find the package to install.

It would be pretty slow to always execute (package-refresh-contents) on every startup, adding tens of seconds to startup time. The workaround is to add a first-boot functionality:

(defconst winny/initialized-cookie-path
  (concat user-emacs-directory "/.w-firstboot-initialized"))
(unless (file-exists-p winny/initialized-cookie-path)
  ;; Run this body only on first startup.
  (message "Detected first boot.  Doing initialization steps...")
  (package-refresh-contents)
  (with-temp-buffer
    (write-file winny/initialized-cookie-path))
  (message "First boot-only initialization complete."))

On first startup the file ~/.emacs.d/.w-firstboot-initialized will not exist. This allows the unless form’s body to execute. Besides writing messages to the *Messages* buffer and running the critical section ((package-refresh-contents)), the with-temp-buffer expression writes out an empty file to ~/.emacs.d/.w-firstboot-initialized.

Now a git clone --recursive git@github.com:winny-/emacs.d.git .emacs.d && emacs works without special user intervention. One downside of my configuration.org setup with use-package is the slow initialization. It takes at least a couple minutes to install all the packages during first startup. Maybe one day my Emacs startup be fast and instantaneous from git clone

Using an old Supermicro IPMI to configure broken networking

Published: Monday, Aug 9, 2021
Last modified: Monday, Aug 9, 2021 (34f8d54)

The goal of this post is to demonstrate the usefulness of IPMI even in hobbyist or personal use. Anything that means less touching physical machines to power cycle them, or fix network misconfigurations, can save a lot of time.

I had broken my NAS’s networking by adding a bridge and attaching the existing ethernet device to it. I forgot to configure the ethernet device to not try to fetch an IP address (via DHCP), but instead only fetch an IP address on the bridge itself. This had the surprising effect of both the bridge (br0) and ethernet (eth0) being given the same IP, as they both had the same mac address (as the bridge was talking to my LAN over the eth0 slave device). Woops, stuff happens.

This NAS does not have HDMI, DVI, or DisplayPort. I do have a VGA monitor, but it was packed up in storage. First thought was I could use a serial console, but it appears I hadn’t configured the serial console correctly, as I got a blank terminal. Then I remembered, this thing has IPMI with full VGA console access. There is a nasty catch, which is what this post is about.

For anyone curious, here’s the specs for my NAS. It’s not very exciting, software configuration barely works, but is reliable. A TODO is to ansible-ize the configuration. The motherboard, cpu were purchased used off ebay, it was a good decision, as it was reasonably inexpensive.

Anyway back to solving that pesky networking issue…

Using an Ubuntu VM

After installing Ubuntu 20.04 in VirtualBox, I set up the guest additions (e.g. run sudo apt install build-essential followed by running the VBoxLinuxAdditions.run off the Guest Additions ISO.) Then I Tried opening the console just for fun. I was greeted by a warning that my Java was out of date (a Browser alert()), which I ignored.

After clicking on the console, I got a prompt to do something with the .jnlp java web start file. Ubuntu didn’t know what to do with it so I gave did some looking around and found this helpful stock overflow post. It says I should just install icedtea-netx (e.g. sudo apt install icedtea-netx). After installing it, then clicking on the console thumbnail again, I was again prompted to do something with the file, but was given the option to “Open with IcedTea Web Start (default)”. After clicking OK, I was greeted with a Java application asking it can trust the web start application as the digital signature could not be verified. Here are the two warning dialogues. I’d recommend doing this is a throw away VM, it’s old crufty software.

After clicking “Run” and “Yes” at the security prompts, I was greeted with the VGA console.

WinXP for the fun of it anyways?

I had initially set up Windows XP assuming I wouldn’t be able to get this to work on a modern linux distro. But as the above section demonstrates, it’s possible to do it on Ubuntu 20.04. But I figure It’d be fun to leave this section in here for the reader’s enjoyment. I learned my lesson: try stuff before you assume anything.

WinXP is available on archive.org free of change. I don’t know how the licensing works, though it has been on archive.org for many years with CD keys, so it must be fine, or Microsoft doesn’t care.

After installing Windows XP (and installing VirtualBox guest additions), you might need to install a browser that can access websites that use strict TLS/SSL ciphers for HTTPS. I found this very cool website that describes how to get started with Windows XP in 2021. Folks still run it unironically for fun and profit. Crazy! But I’m also that crazy, so let’s grab a good browser.

Enter Mypal. It has a cute mascot of a Raccoon. It also seemed to work more reliably than the version of Artic Fox I tried (it kept hanging ???). You can download it here from its GitHub releases page. Enjoy the following screenshot!

Now we have a modern-ish web browser we can access the webpages that offer Java installers for Windows XP. Unfortunately, Oracle seems to have removed all references to legacy Java builds that work on Windows XP. I get an error message when trying the old Java releases on Windows XP:

An hour of Google head-bashing later, I found this link to a questionable website called Filepuma that offers a usable Java installer. For the esteemed reader, here is the output of sha256sum:

43edfa7ecde47309cb7af2ce81dd856f7299e47ae26aa387fc5db541a016bea4  Java_Runtime_Environment_(32bit)_v7.0.5.exe

Running it, one gets the usual security warning followed by a setup screen. Success!

Finally, accessing the console

After a reboot, I fired up Internet Exploder again, logged into the IPMI webpage. After clicking on the console preview, the interactive VGA console automatically opened up in Java WebStart. Fanstatic! I was able to log in to the console and fix the networking issue. All from my desk, without touching the affected computer.

No need to reinstall your OS

Published: Thursday, Jul 8, 2021
Last modified: Thursday, Jul 8, 2021 (a1b05c7)

A fantastic “feature” of Linux, BSD, and even Windows 10 is you don’t really need to reinstall to migrate an installation to a new computer. A common misunderstanding is if you get a new PC, you must use the new OS install, or install a new copy of your OS. If you’re intending on replacing an existing PC (and disposing of or re-purposing the old one), there is probably no need to reinstall your OS and deal with user data migration.

On Linux if you rip a SSD out of any x86 box, it will probably boot on any other machine, including virtualization software. One example of this is the laptop I am currently typing away on. It is the third laptop for the same Linux installation. Each time I simply either moved the SSD over or duplicated the SSD onto another SSD. Sometimes I have to modify some configuration, as I use Gentoo on my workstation and laptop and it’s not going to do everything for me automagically (or otherwise surprise me with some unwelcome, opinionated defaults).

My NAS has been migrated between multiple rebuilds with no problem. Even switching between SAS Passthrough cards is not a problem. It runs Alpine… but the distro hardly matters in this discussion. All distros will handle booting on random equipment reasonably well. My Desktop has survived a complete rebuild (new everything) and many partial rebuilds. It’s the same install from years ago.

Sometimes on IRC I hear of people having the same install from ago. I think it might be more common in the Gentoo community, but I don’t see this being a distro specific thing, only that Gentoo is a bit less surprising when things break. Clear error messages and doesn’t hide it behind a bunch of tooling that takes a long time to learn.

Example migration

I recently swapped out my HP Elitebook 820 G4 for a XPS 13. Both devices have NVMe SSDs. I saw four options to perform the installation migration:

  1. Install the old SSD into the new laptop;
  2. Remove the old SSD. Install into a USB SSD enclosure. Then copy its contents over (using ddrescue) to the new SSD (using a LiveCD environment);
  3. Boot the old laptop with a LiveCD, make an image of the SSD onto an external drive, then boot the new laptop with a LiveCD, write the image onto the new SSD from the external drive;
  4. Or Boot both laptops with a LiveCD, network them, then the old SSD data to the new SSD on the other machine using the network.

I opted for option 4.

First step is to boot both laptops with live media. I prefer GRML because it’s pretty simple and no frills, yet it ships with normal desktop stuff such as audio, if you really need it. It has a boot option to load the entire live media into ram, thereby allowing you to remove the bootable media. (One could also PXE boot to achieve similar effect.) I simply booted each laptop from the same usb, using the option to load into memory. Next up was to set up a network. In this case I had 1Gb ethernet interfaces on both devices (the XPS 13 has ethernet via a USB-C dock). I simply ran grml-network on both, said no to DHCP, set each to a RFC 1918 address in the same configured subnet. Ping one host from the other and verify it works. Next is a bit muddy, because it in theory did not work perfectly, but it did achieve the intended results.

I ran nc -l 1234 | pv > /dev/nvme0n1 on the new machine. Then I ran pv /dev/nvme0n1 | nc 192.168.0.1 1234 on the donor machine. It took about 2.5 hours for coping an entire 1TB SSD. In retrospect I believe this could have been faster had I used 10Gb ethernet or a USB enclosure, but it got the job done.

One gotcha is when the imaging finished, nc/pv hung on both sides. I could tell they were finished because no more progress was being made. And they were at the progress of the size of the donor machine’s storage. This is not a good thing to experience, so any recommendations to get better nc ... | pv ... to exit is welcome 😁.

Next up was booting. I just rebooted and it worked. Note: I set up my grub installs to boot both with MBR BIOS systems and UEFI systems. So tell your friends, you don’t need to pick ‘n choose how to boot your PC, just boot it either way if you set up grub to support both.

Migrating Windows Installs

I don’t know if it is realistically feasible to migrate a pre-Windows 10 install, but with Windows 10, one can simply yank the OS disk and install it in the new PC. If you need to copy tho install onto a soldered SSD, or are upgrading the storage, you can just image the entire original storage device onto the destination storage device. One can use gparted to resize Windows. Surprisingly, Windows will handle this - previously Windows would get upset if the NTFS C: partition is resized without special treatment.

Alternatively, there are vendor tools that can streamline this process, such as Samsung Data Migration tool.1

One gotcha - even if you successfully migrate your Windows 10 install to a new box, you may have to reactivate the license… you’ll find out.

Some other gotchas

If you need to resize an install, it might be feasible without much trouble. If you’re growing a Linux install, usually one can just resize the partitions/block devices in gparted/parted, then run grow2fs(8) on the filesystems. If you’re using LUKS encryption it appears to take its size from the size of the block device it resides on. AS for LVM2, see pvresize(8).

Shrinking is a bit trickier because you will have to adjust the LVM2/LUKS/filesystems before shrinking the partition itself. Shrinking can be trickier because doing partition table math is hard. You’ll have to carefully ensure you shrink the filesystem, then shrink the block devices, and partitions in order. I haven’t quite figured it out but it feels like guesswork to get the sizes just right, so it might be easiest to shrink the filesystem, LVM2 physical volume, LUSK, etc to be extra small, but a bit bigger than whatever they contain. Then just “grow” each filesystem/block device after the partition is finally shrank. This way you don’t need to do exact math. Recap: Shrink everything be too small starting with the filesystem working your way out, then grow everything inside the partition with growing the filesystem last.

Backups! Backups! Backups! If you are doing anything with data, it doesn’t exist if you don’t have backups that you can restore from. Ideally you want at least two copies. When imaging anything, it’s usually a good idea to keep a copy of the original image until you’re satisfied with your handywork. Better safe than sorry. Data storage is cheap… or if you afford backups for your storage, you can’t afford your storage (as it doesn’t exist without backups).

Conclusion

I think my details were a bit streamlined, but I wanted to keep this short. If the reader takes anything away from this post, most modern OS installs are portable between hardware, and are easy to install onto new storage. Reinstalls shouldn’t be a thing of necessity, but an action of policy only - if you are re-purposing a machine, by all means reinstall, but no need to do it when migrating.

As always I should shout out a live media to use for this post — GRML linux is great for this.


  1. Thanks smokey991 for suggesting this. ↩︎