Connection refused? Docker networking and how it impacts your image


Can’t connect to the server running in your container? Let’s see why, and how to fix it, starting with an example.

Let’s say you run the following command on your computer:

$ FLASK_APP=exampleapp:app flask run
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

You can point a browser at http://127.0.0.1:5000/ and look at the resulting web page.

If, however, you package it in a Dockerfile, and then run it:

$ docker run --rm -p 5000:5000 example
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

This time when you point a browser at http://127.0.0.1:5000 you get a connection reset or connection refused error.

What’s going on? To understand how to solve this, you need to know a minimal amount about how Docker’s networking works. In particular, this article will cover:

  • Networking namespaces, and how Docker uses them.
  • What docker run -p 5000:5000 does, and why our example above doesn’t work.
  • How to fix your image so the server is accessible.

Networking without Docker

Let’s start with our first scenario: you run a server directly inside your operating system, and then connect to it. I’m going to assume the main OS is Linux, for simplicity of explanation. Docker runs on non-Linux OSes like macOS by running a Linux virtual machine, but the practical consequences are the same.

Your operating system has multiple network “interfaces”. For example, on my computer (with output shortened for clarity):

$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 172.17.0.1 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 wlp0s20u8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.7.202

In this output we see three network interfaces:

  • We’ll ignore docker0 for now.
  • lo is the loopback interface, with IPv4 address 127.0.0.1: it’s your own computer, addressable in-memory without any networking hardware.
  • wlp0s20u8 is my WiFi card, with IPv4 address 192.168.7.202, and when I talk to computers on the Internet the packets are sent via that interface.

Let’s go back to our starting, working example—you run a server listening on 127.0.0.1, and then connect to it. We can visualize it like this:

Network namespaces

You’ll notice the image above talks about a “Default network namespace”. So what’s that?

Docker is a system for running containers: a way to isolate processes from each other. It builds on a number of Linux kernel features, one of which is network namespaces—a way for different processes to have different network devices, IPs, firewall rules, and so on.

By default, each container run by Docker has its own network namespace, with its own IPs:

$ docker run --rm -it busybox
/ # ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02 inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0

So this container has two interfaces, eth0 and lo, each with their own IP addresses. But because this is a different network namespace, these are different interfaces than the default namespace we saw above.

To make it clear what this means, let’s run the Flask server inside a Docker container, and then diagram the results:

$ docker run --rm example * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

The resulting network setup looks like this:

Now it’s clear why there’s a connection refused: the server is listening on 127.0.0.1 inside the container’s network namespace. The browser is connecting to 127.0.0.1 in the main, default network namespace. But those are different interfaces, so no connection is made.

How do we connect the two network namespaces? With Docker port-forwarding.

Docker run port-forwarding (is not enough)

If we run docker run with -p 5000:5000, it will forward from all interfaces where the Docker daemon is running (for our purposes, the main network namespace) to the external IP address of the containter.

To break it down explicitly: -p 5000:5000 means redirecting traffic from port 5000 on all interfaces in the main network namespace to the container’s port 5000 on its external interface. -p 8080:80 would redirect traffic from port 8080 on all interfaces in the main network namespace to port 80 on the container’s external interface. And so on.

(We’re doing port 5000 specifically because that’s where our Docker image is listening, Flask’s default port.)

So let’s run a container, and then look at a diagram to visually see what that means:

$ docker run --rm -p 5000:5000 example * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

And now we see the second problem: the server is listening on 127.0.0.1 inside the container network namespace, but the port forwarding is going to the external IP, 172.17.0.2.

Thus, a connection reset or refused.

The solution: listen on all interfaces

Port forwarding can only connect to a single destination—but you can change where the server process is listening. You do this by listening on 0.0.0.0, which means “listen on all interfaces”.

Note: Outside the specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

Want a best-practices Dockerfile and build system? Check out my Production-Ready Python Containers product.

FROM python:3.7-slim-stretch
RUN pip install flask
COPY . .
ENV FLASK_APP=exampleapp:app
CMD ["flask", "run", "--host", "0.0.0.0"]

Now the network diagram looks like this:

Takeaways

  1. By default, containers run in their own network namespaces, with their own IP addresses.
  2. docker run -p 5000:5000 will forward from all interfaces in the main network namespace (or more accurately, the one where the Docker daemon is running) to the external IP in the container.
  3. You therefore need to listen on the external IP inside the container, and the easiest way to do that is by listening on all interfaces: 0.0.0.0.