I’ve been keeping my eyes open for a new development workstation powerful enough for Scala compilation in a laptop form factor. When I saw that Lenovo had a Black Friday sale on an extraordinarily inexpensive laptop ($129!) with good build quality but anemic specs, I had an idea: why not use this cheap laptop as a thin client for a remote development environment? I have a relatively snappy little server at home that can do all of the heavy lifting, which should mean that the laptop gets great performance without sacrificing on cost, weight, or battery life.
My Linux server runs headless Debian, which is great for a server but not really ideal for my development workflow. So the plan will be to use Docker to run Arch in a container. I’ll expose the container to the outside internet so that I can access it wherever I am in the world. It would be possible to do all of this with vagrant — there’s even an archlinux box - but I’m looking for something a little lighter-weight than a dedicated virtual machine. systemd-nspawn is perhaps more fit-for-purpose when running a whole guest OS in a container, but it lacks some of the convenience features provided by Docker, especially around automation and network configuration. Besides, this is a good opportunity to get some hands-on experience with Docker. It’s worth pointing out, though, that we’re accepting some risk by using containers because if there are ever any forward-incompatible kernel ABI changes then our Arch guest’s libc might try to make system calls that the Debian host kernel doesn’t understand. I’ve also found that security features in Docker mean that not everything in the guest OS works right out of the box, but that’s just something I’m going to deal with as it comes.
Before we move on, let’s discuss the different options available for how to remotely control my development environment from the laptop. LTSP and Thinstation don’t seem to exactly align with what I’m trying to do. I know from experience that basic X forwarding over SSH won’t give me anywhere near adequate performance (low latency, compatability with low-bandwidth WAN links) for comfortable use. Xpra adds some tricks and H.264 video compression to make X forwarding more bearable, so that might work — it even has an HTML5 client that works over SSL, in case you only have access to a web browser. TigerVNC/Xvnc over SSH would probably work okay. From my research, though, it looks like NoMachine has the best technology out there for remote desktop on Linux, and fortunately the freeware version of their product supports a virtual framebuffer via an embedded X server. If FOSS is a hard requirement for you, an old version (c. 2013) of their technology was open source, and it lives on in the X2Go project. From reports I see on the web, it seems that X2Go provides better performance than VNC and Xpra, so that would be my second choice. But I’m going with NoMachine.
Step zero is to make sure that Docker is installed on the host server. The version of Docker in the Debian repository is pretty old (even in Debian testing) so we’ll want to get it directly from Docker’s APT repo. The official instructions are fine but this is what I did instead:
sudo -i curl -fs https://download.docker.com/linux/debian/gpg | gpg --dearmor > /etc/apt/trusted.gpg.d/docker.asc.gpg echo "deb [arch=amd64] https://download.docker.com/linux/debian buster stable" > /etc/apt/sources.list.d/docker.list apt-get update apt-get install docker-ce docker-ce-cli
Now we need a Dockerfile specifying how to build the image for our development environment. This is what I came up with; feel free to customize it for your needs.
FROM archlinux/base ARG username=brian WORKDIR /root # The default mirrors are sometimes not synchronized. Updating the mirrorlist first is necessary for the Dockerfile to reliably run without failures. RUN pacman -Syu --noconfirm \ && pacman -S --noconfirm reflector \ && reflector --verbose --fastest 5 --age 6 --save /etc/pacman.d/mirrorlist \ && pacman -Syu --noconfirm \ && pacman -S --noconfirm --needed man man-pages nano openssh iputils procps-ng base-devel git systemd-sysvcompat \ && pacman -S --noconfirm --needed xfce4 xfce4-goodies gvfs pulseaudio ttf-roboto ttf-ubuntu-font-family ttf-dejavu # Set the system time zone. RUN ln -sf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime # Create our user *before* mounting the home directory as a volume so that it won't be owned by root. RUN useradd --create-home --user-group --groups wheel --shell /bin/bash $username RUN echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/wheel # Enable OpenSSH daemon with password-based authentication disabled. You can remove some of this if you don't have authorized_keys set up. RUN systemctl enable sshd RUN sed -i 's/UsePAM yes/UsePAM no/' /etc/ssh/sshd_config RUN sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config COPY --chown=$username:$username authorized_keys /home/brian/.ssh/authorized_keys # Set up key-based auth for NX, though as we'll see it doesn't actually work. You can remove this if you don't have authorized_keys set up. RUN mkdir -p /home/brian/.nx/config RUN cp /home/brian/.ssh/authorized_keys /home/brian/.nx/config/authorized.crt RUN chown -R brian:brian /home/brian/.nx RUN chmod 0600 /home/brian/.nx/config/authorized.crt # Patch makepkg so that it will let us run it as root. # Evil, but not as dumb as switching to a user only for makepkg to prompt for a nonexistent sudo password to install dependencies. RUN cp --preserve=all /usr/bin/makepkg /usr/bin/makepkg.old RUN sed -i 's/exit $E_ROOT/#exit $E_ROOT/g' /usr/bin/makepkg # Build/install NoMachine. The vanilla installer doesn't support Arch but there's an AUR package which patches the Fedora support to work. # The installer script checks /proc/1 to see if we're running systemd so if we want it to install the `.service` file we have to rename the shell. # You can remove the DefaultDesktopCommand line if NoMachine autodetects your desktop environment. See https://www.nomachine.com/AR10K00725 RUN git clone https://aur.archlinux.org/nomachine.git WORKDIR nomachine RUN makepkg -sric --noconfirm RUN cp /bin/bash systemd-sh RUN sed -i 's_#!/bin/bash_#!/root/nomachine/systemd-sh_' /usr/NX/scripts/setup/nxserver RUN /usr/NX/scripts/setup/nxserver --enableservices RUN sed -i 's_#!/root/nomachine/systemd-sh_#!/bin/bash_' /usr/NX/scripts/setup/nxserver RUN sed -i 's_DefaultDesktopCommand /usr/bin/startxfce4_DefaultDesktopCommand "/usr/bin/dbus-launch /usr/bin/startxfce4"_' /usr/NX/etc/node.cfg RUN sed -i 's/EnableUPnP NX/#EnableUPnP NX/' /usr/NX/etc/server.cfg RUN echo "UDPPort 4000" >> /usr/NX/etc/server.cfg RUN echo "CreateDisplay 1" >> /usr/NX/etc/server.cfg RUN echo "DisplayOwner \"$username\"" >> /usr/NX/etc/server.cfg RUN echo "#EnableNXClientAuthentication 1" >> /usr/NX/etc/server.cfg RUN echo "#AcceptedAuthenticationMethods NX-private-key" >> /usr/NX/etc/server.cfg WORKDIR /root RUN rm -rf nomachine # Build/install yay, an AUR helper. You can remove this section if you use a different helper or prefer to run makepkg manually. RUN git clone https://aur.archlinux.org/yay.git WORKDIR yay RUN makepkg -sric --noconfirm WORKDIR /root RUN rm -rf yay # Put makepkg back the way it was. RUN mv -f /usr/bin/makepkg.old /usr/bin/makepkg # Since we're running systemd as our PID1 we need Docker to send it the SIGRTMIN+3 stop signal that systemd expects rather than the default SIGTERM. STOPSIGNAL RTMIN+3 VOLUME /home/$username CMD [ "/sbin/init" ]
Note that we’re using systemd as our entry point. Typically you would not have an init system running in a container at all. It conventionally suffices to set up the filesystem and start your containerized application as PID1 directly. In our case, however, it would be nice to have systemd running so that we can install multiple services and start/stop them with systemctl. We want a full operating system, not just a single application, after all. Running systemd also helps us by reaping zombie processes which might otherwise be ignored by an application-specific entry point.
Put our Dockerfile in a directory and build an image from it. Change
brian to your own username:
sudo -i mkdir dev cp ~brian/.ssh/authorized_keys . nano Dockerfile docker build -t dev --build-arg username=brian .
Now create a new container and start it up in the background. We need a couple of extra options so that systemd can run as PID1 in a non-privileged container. The
SYS_PTRACE capability seems to be needed for NoMachine to start up its embedded X server and is useful for debugging, but be careful - if your kernel is older than version 4.8 then programs can use ptrace to bypass seccomp filters.
SYS_RESOURCE is useful and relatively harmless, and stops sudo from complaining at us when it tries to call
--shm-size is required to prevent Chrome from running out of memory, because it makes use of
/dev/shm which is only 64MB by default. We restate the
/home/$username volume mount here so that Docker gives it a name and will reuse the same volume across
docker run invocations; if you want to start completely fresh, remove the
-v devvol:/home/brian option or run
docker volume rm devvol.
docker run -d -v devvol:/home/brian -p 2222:22 -p 4000:4000 --name dev --hostname dev --cap-add=SYS_PTRACE --cap-add=SYS_RESOURCE --shm-size=1g --restart always --tmpfs /tmp:exec --tmpfs /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro dev
For now we can start a new root shell in the container so that we can set a password for our user. It’s a good idea to do this interactively so that your password isn’t recorded into
docker exec -it dev bash passwd brian exit
I had one additional step, which was to update my firewall configuration on the host server to permit IPv4 packets to pass through to the container via the Docker bridge network device.
sudo -i cat > /etc/ufw/applications.d/nxserver <<-ENDOFMESSAGE [nxserver] title=NX protocol server description=Part of NoMachine, a remote desktop solution. ports=4000 ENDOFMESSAGE ufw app update nxserver ufw allow from 192.168.0.0/16 to 0.0.0.0/0 app nxserver ufw limit log proto tcp from 192.168.0.0/16 to 0.0.0.0/0 port 2222
If you’re a Chrome user, running it in a container is a bit tricky because it uses user namespaces for sandboxing. This feature is disabled by default in Debian for security reasons so I had to run this in order to enable it on the host kernel (note that this specific sysctl knob will not work in other distros):
echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/00-local-userns.conf service procps restart
The other thing you’ll need in order to run Chrome in your container is Jess Frazelle’s Chrome seccomp profile. Add
--security-opt seccomp=/path/to/chrome.json to your
docker run command in order to apply it to your container. Because this will replace the default seccomp profile completely, it can actually be more restrictive than the default profile in some ways. In particular, syscalls needed to use strace are allowed by the default profile but not by Jess’s Chrome profile. Her profile also doesn’t allow
statx, which can break Qt apps. I created a merged profile that includes both the syscalls needed to run Chrome and the syscalls granted by the default profile.
In order to remote into your server, download and install the freeware NoMachine Enterprise Client. Create a new connection, selecting the NX protocol (the freeware version of the server software doesn’t support SSH) and entering the relevant hostname or IP address. You can select “Use UDP communication for multimedia data” if you change the port to 4000.
The Dockerfile copies your server’s
~/.nx/config/authorized.crt but I was unable to get key-based authentication to work (connections fail if
1 and this guy’s fix doesn’t work for me) so password-based authentication is left enabled. Use a strong password if you expose this to the internet!
This section is a collection of pointers for Docker beginners to help with common tasks that you might want to perform with your container.
/var/lib/docker/volumes/on the host. Do not try to insert files into the container this way. For inbound file transfers, you should use the send/receive functionality built into NoMachine, use SCP/SFTP, or mount a network share into your container.
docker stop dev. To start it again, run
docker start dev. Don’t call
docker runagain as it will create a brand new container.
docker rm dev. To delete the image, run
docker image rm dev. To delete unused volumes, run
docker volume prune.
docker runwith the
--restart alwaysargument, our container should automatically come back up if it crashes or if the host reboots.
docker runoptions and you don’t want to lose all of your data, you can
docker committo build a new image from the current state of the container. You can then
docker runthe new image with your updated options.
After repeated iterations of committing and re-running your image, you may find that its layers are consuming excessive disk space. You can export the filesystem of your container and import it into a new image with the command
docker export dev | docker import --change 'CMD [ "/sbin/init" ]' - dev-flat:latest
Note that the only thing preserved by this export/import operation is the filesystem, so we have to re-supply the
CMD with a
--change argument. We also need to supply the
STOPSIGNAL instruction but predictably this instruction isn’t currently supported by
docker import — make sure to add
--stop-signal=$(kill -l RTMIN+3) to your
docker run command.