Installing a cloud server with full disk encryption

Written by ADRN — March 3, 2021

Installing a cloud server with full disk encryption

I am installing a new server on Kimsufi. I want full-disk & swap encryption. Let’s do it.

Guides:

Overview of steps

  • Boot the server in rescue mode.

  • Wipe filesystem, format partitions:

  • Install and configure Debian (including disk cryptography and decryption through SSH).

  • Boot the server & setup automated decryption from a remote (in case of reboot).

Detailed process

Boot the server in rescue mode

  • Go to the Kimsufi admin panel, click “NetBoot”, select “Rescue”, pick “rescue64-pro”, clock “Next”, “Confirm”, and then click the “Reboot” button on the admin panel.

  • Kimsufi will mail you the root password for you to log into the server in rescue mode.

  • Will likely cause an error from your SSH client, since the fingerprint of the server differs from the usual one.

Wipe filesystem & format partitions

Target partition table:

        +-----------+--------------------+-----------+--------------------+
Name:   | /dev/sda1 | /dev/sda2          | /dev/sda3 | /dev/sda4          |
        |           |                    |           |                    |
        |           |                    |           |                    |
        |           | +------------------+           | +------------------+
        |           | | /dev/mapper/swap |           | | /dev/mapper/main |
Format: | none      | | swap             | ext4      | | ext4             |
Size:   | 1MiB      | | 8GiB             | 512MiB    | | remaining space  |
Flags:  | bios_grub | |                  |           | |                  |
Mount:  |           | | swap             | /boot     | | /                |
        +-----------+-+------------------+-----------+-+------------------+

We use GPT partition table layout (without UEFI), which demands a small bios_grub partition at the beginning of the drive (stores GRUB’s core.img). Hence /dev/sda1. See:

  • https://en.wikipedia.org/wiki/BIOS_boot_partition
  • https://www.gnu.org/software/grub/manual/grub/html_node/BIOS-installation.html#BIOS-installation
  • https://wiki.archlinux.org/index.php/partitioning#Partition_table
  • https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Disks#Partition_tables

  1. Wipe filesystem, format partitions

    • Wipe all information about previous filesystem:

      DANGER!!! DANGER!!! WIPES EVERYTHING FROM YOUR DISK!!!

      wipefs -a /dev/sda

    • Create GPT disk layout

      # Create GPT layout
parted -a optimal /dev/sda mklabel gpt
# Create first 1MiB bios_grub partition 
parted /dev/sda -a optimal mkpart bios_grub 0% 1MiB
parted /dev/sda set 1 bios_grub on
# Create third 8GiB swap partition
parted /dev/sda -a optimal mkpart swap 1MiB 8001MiB
# Create second 512MiB /boot partition
parted /dev/sda -a optimal mkpart boot 8001MiB 8513MiB
# Create last / partition using all remaining space 
parted /dev/sda -a optimal mkpart main 8513MiB 100%
# Set first partition as bootable

    • Format partitions

      • The ext4 boot partition

        Format it like so:

        mkfs.ext4 /dev/sda3

      • The encrypted swap partition

        All configuration of the encrypted swap partition comes after the OS installation.

      • The encrypted main partition

        # Set it up using cryptsetup (I searched for good parameters):
cryptsetup --type luks2 --cipher aes-xts-plain64 --hash sha256 --iter-time 2000 --key-size 512 luksFormat /dev/sda4

# Get the UUID, save it for later:
cryptsetup luksDump /dev/sda4 | grep UUID | awk '{print $2}'
# Example output: 09762476-ba7c-4732-8856-44a716c23339

# Decrypt the partition:
cryptsetup open /dev/sda4 main

# Format it to ext4:
mkfs.ext4 /dev/mapper/main

  2. Install and configure Debian

    • Mount the partitions 

      # Decrypt the main partition (if not already done):
cryptsetup open /dev/sda4 main
# mount it:
mount /dev/mapper/main /mnt 
# Mount the boot partition:
mkdir /mnt/boot
mount /dev/sda3 /mnt/boot

    • Bootstrap latest stable Debian into mounted partitions

      debootstrap --arch amd64 stable /mnt http://ftp.fr.debian.org/debian/

    • Mount system partitions

      mount -o bind /dev /mnt/dev
mount -t proc proc /mnt/proc
mount -t sysfs sys /mnt/sys
mount -t devpts devpts /mnt/dev/pts

    • Chroot

        chroot /mnt /bin/bash

    • Fill out crypttab and fstab (UUID is the output of above cryptsetup luksDump)

      /etc/crypttab

      cat << EOF > /etc/crypttab
# <name> <device>                                  <password>   <options>
main     UUID=09762476-ba7c-4732-8856-44a716c23339 none         luks
swap     /dev/sda2                                 /dev/urandom swap,noearly,cipher=aes-xts-plain64,size=512
EOF

      /etc/fstab

      cat << EOF > /etc/fstab
# <filesystem>   <dir> <type> <options>          <dump> <pass>
/dev/mapper/main /     ext4   errors=remount-ro  0      1
/dev/sda3        /boot ext4   defaults           0      2
/dev/mapper/swap none  swap   sw                 0      0
EOF

    • Do the /proc/mounts -> /etc/mtab symlink

      ln -s /proc/mounts /etc/mtab

      Why? Linux From Stratch and our guide on OpsBlog both say it’s needed.

    • Choose a hostname & DNS domain

      hammerhead & hammerhead.luxeylab.net

    • Fill in network-related files

      Beware of your ethernet interface name! It’s only after failing with eth0 that the boot logs informed me that the iface name was eno1.

      /etc/network/interfaces

      cat << EOF > /etc/network/interfaces
######################################################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# See the interfaces(5) manpage for information on what options are
# available.
######################################################################

# loopback interface
auto lo
iface lo inet loopback

# eno1 through DHCP
auto eno1
iface eno1 inet dhcp
EOF

      /etc/resolv.conf

      cat << EOF > /etc/resolv.conf 
# OVH public DNS
nameserver 213.186.33.99
# https://servers.opennicproject.org/edit.php?srv=ns2.he.de.dns.opennic.glue 
nameserver 172.104.136.243
EOF

      /etc/hosts

      cat << EOF > /etc/hosts
127.0.0.1 localhost
127.0.1.1 hammerhead hammerhead.luxeylab.net 

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

213.186.33.116 ws.ovh.com
198.245.48.4 ws.ovh.ca
5.135.179.11 hammerhead hammerhead.luxeylab.net 
EOF

      /etc/hostname

      echo "hammerhead" > /etc/hostname

      /etc/timezone

      dpkg-reconfigure tzdata

    • Update APT repositories in /etc/apt/sources.list

      cat << EOF > /etc/apt/sources.list
deb http://ftp.fr.debian.org/debian buster main
deb-src http://ftp.fr.debian.org/debian buster main

deb http://ftp.fr.debian.org/debian-security/ buster/updates main
deb-src http://ftp.fr.debian.org/debian-security/ buster/updates main

deb http://ftp.fr.debian.org/debian buster-updates main
deb-src http://ftp.fr.debian.org/debian buster-updates main
EOF

    • Configure locales & keyboard

      # Refresh packages list
apt update
# Install & configure locales 
apt install locales && dpkg-reconfigure locales
# Install & configure keyboard 
apt install console-setup console-data && dpkg-reconfigure keyboard-configuration

    • Pick up a kernel & install it

      apt-cache search linux-image
# I selected the meta-package `linux-image-amd64` which is the latest 
apt install linux-image-amd64 linux-headers-amd64

    • Install additional necessary software

      apt install cryptsetup dropbear grub-pc man-db ssh

      Because we use a GPT table layout but no UEFI, we install grub-pc and not grub-efi-amd64.

      Do think about installing bash-completion and ufw someday.

    • Set up SSH keys and the rest (where <PUBLIC_SSH_KEY> is your client SSH public key)

      mkdir /root/.ssh && chmod 600 /root/.ssh
echo "<PUBLIC_SSH_KEY>" > /root/.ssh/authorized_keys
echo "<PUBLIC_SSH_KEY>" > /etc/dropbear-initramfs/authorized_keys

    • Configure dropbear-initramfs (SSH on boot)

      sed -i 's/#DROPBEAR_OPTIONS=/DROPBEAR_OPTIONS=\"-p 2222 -c \/bin\/cryptroot-unlock\"/' /etc/dropbear-initramfs/config

      This changes the SSH listen post to 2222, and enforces that the only command run from dropbear-initramfs is cryptroot-unlock.

    • Configure sshd custom listen port (SSH after boot)

      sed -i 's/#Port 22/Port 2223/' /etc/ssh/sshd_config

    • [Optional] Authorize root password login 

      # Remove -s (disable password login) from dropbear-initramfs
sed -i 's/local flags=\"Fs\"/local flags=\"F\"/' /usr/share/initramfs-tools/scripts/init-premount/dropbear

    • Update GRUB and initramfs

      update-grub && update-initramfs -u

    • Clean up before leaving 

      exit
umount /mnt/{boot,dev/pts,dev,proc,sys}
umount /mnt
cryptsetup luksClose main

  3. Boot the server & setup automated decryption from a remote

    • I configured my laptop’s ~/.ssh/config like so:

      Host hammerhead-decrypt
    User root
    Hostname <SERVER_URL_OR_IP>
    Port 2222
    IdentityFile <PUBLIC_SSH_KEY_PATH>
    # Prevents signatures mismatch
    UserKnownHostsFile ~/.ssh/known_hosts_hammerhead-decrypt
Host hammerhead
    User root
    Hostname <SERVER_URL_OR_IP>
    Port 2223
    IdentityFile <PUBLIC_SSH_KEY_PATH>

    • And here is how I connect to the server:

      laptop$ ssh hammerhead-decrypt 
Please unlock disk main: <PASSWORD>
# [...]
Connection to <SERVER_URL_OR_IP> closed.
laptop$ ssh hammerhead 
# [...]
root@hamerhead:~# echo A winner is you!
A winner is you!

Automatically decrypt the drive from a remote server

It is desirable to have a daemon running on a remote server, to automatically decrypt the drive when the encrypted server reboots without warning.

The remote server is called a key escrow. One must be particularly careful about the escrow’s security, since it holds the decryption keys for our server.

Trinity recommends Tang and Clevis.

This article is under the CC-BY-SA license.

Log in, or use your Fediverse account to interact with this article

Comments

No comments yet. Be the first to react!