UFW & Docker: Fixing a Security Issue

August 29, 2019    ufw docker security devops sysadmin hosting linux

I recently changed hosting providers, and as a result had to rebuild a bunch of the infrastructure that hosts this site. While going through this process, I ran into an interesting security issue, and I thought I’d document while it was fresh in my mind.

When first setting up the server I went through a few simple steps to increase its security and decrease the potential attack footprint (the cool kids call this process “hardening”). These steps included:

  • SSH: Turning off root logins
  • SSH: Turning off password-based authentication altogether
  • Enabling UFW and allowing SSH connections only (denying all other connections)

I then went on to install Portainer, which is a nice web-based GUI for managing Docker containers that I had used in the past. It’s not “enterprise-ready” like some other solutions out there, but it’s quite simple and good enough for my use case.

I used Portainer’s docker-compose-like syntax to set up a bunch of services on the machine, including the web server for this site. I then opened my browser to test this site out, and lo and behold, everything worked great on the first try!

Great success!

Wait a second.

It took me a few minutes but I realized that I shouldn’t be able to access the web site, since I had explicily set up UFW before to only allow SSH connections. I double checked my UFW configuration:

colin@fajita:~$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere                  
22/tcp (v6)                ALLOW IN    Anywhere (v6)

colin@fajita:~$

Default deny on incoming connections, and only port 22 is open. Yet, I can still access my site which is running on port 443. What gives?

After doing some research, I discovered that by default, Docker adds rules to iptables (the actual kernel-level firewall that UFW wraps) in a way that bypasses the rules that UFW sets up for you. Ugh.

The best solution I found was provided in the ufw-docker GitHub repo. You add a short set of iptables commands to the /etc/ufw/after.rules file and then you’re set. Once I implemented these changes, I could no longer connect to the website, as I expected. To allow incoming connections, I had to use this (slightly longer than normal) UFW command:

ufw route allow proto tcp from any to any port 80

It turns out that with this workaround, you need to use the slightly longer ufw route allow syntax any time you want to open a port that is mapped to a Docker container. This is not a horrible inconvenience for me, as I have a single reverse proxy running on ports 80 and 443 that forwards connections to all of the websites that I host on this machine, so I only needed to open those ports once.

I hope this helps someone out there!