Port forward with DevContainer and Docker Compose

The Problem

You need multiple containers for your development setup and use a docker-compose.yml file to declare the requirements for your services. After starting the setup you see the port for one service in the VSCode ports tab but none of the ports from the other service containers.

If you have no time for background information, jump directly to the working solution.

Example Setup

Let’s define a simple example setup to explore the problem space. Accessing services in development differs from the requirements in production.

We have a typical node application with two other required services with a docker-compose.yml setup:

services:
  service_a:
    build:
      context: ../service_a/
      dockerfile: Dockerfile
    volumes:
      - ../service_a:/workspace:cached
  service_b:
    build:
      context: ../service_b/
      dockerfile: Dockerfile
    volumes:
      - ../service_b:/workspace:cached
    working_dir: /workspace
  db:
    image: postgres:latest
    restart: unless-stopped
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres
volumes:
  postgres-data:

VSCode will allow us to add a devcontainer.json setup based on this file which will look like this:

{
    "name": "Node.js & PostgreSQL",
    "dockerComposeFile": "docker-compose.yml",
    "service": "service_a",
    "workspaceFolder": "/workspace",
    "customizations": {
        "vscode": {
           "extensions": [
                "dbaeumer.vscode-eslint"
            ]
        }
    },
    "remoteUser": "node"
}

As part of the assisted setup we will be asked for the main service which will be added as "service": "service_a", to the config.

Let’s run this setup with DevContainer: Rebuild and Reopen in Container and look at the options we get.

(1) we see only the files of the service_a in the explorer panel of VS Code:

(2) starting the node app node index.js will make VS Code detect the port and offers the option to preview the port in the browser:

(3) switching to the PORTS tab will show the currently forwarded ports:

How is everthing related

Based on the current setup:

• VS Code is only installed in the main container which is service_a
• all ports are only defined with EXPOSE xxx in the docker files of the services
• all ports are unbound
• only the ports from the main container are auto-detected and for devcontainer port forwarding
• Port from other containers need to be defined with "host:port" in devcontainer.json portFoward directive.

How can this be fixed?

IMPORTANT: host names are only valid with character a-z0–9 and the - dash. All other character e.g. _ underline create unvalid hostname ! Check you docker-compose.yml service names which become hostnames to only contain vaild characters!

(A) devcontainer.json / portFoward

This solution is currently (Nov 2022) not supported by Github Codespaces!

(1) fix the service names in docker-compose.yml. Change service_a to servicea and service_b to serviceb.

(2) in devcontainer.json uncomment the portFoward directive. Use the format "host:port" to specify ports on none main containers and change to:

"forwardPorts": [
    3000,
    "serviceb:3000",
    "db:5432"
],

(3) Optional you can enrich the display of the ports with better lables or define additional settings — e.g. UPC:

"portsAttributes": {
    "3000": {
        "label": "Service A (Main)"
    },
    "serviceb:3000": {
        "label": "Service B"
    },
    "db:5432": {
        "label": "Database Postgress"
    }
},

Diagram of the port mapping:

The green ports are defined by VS Code which tries to do a 1:1 from the ports of the containers but will increase the port if containers use the same port and will use a different port if the port is used by an other service on the local computer.

You find all relevant code for the example in my Github repository

VS Code with all fowarded ports, database access and previews of both services:

(B) Ports mapping

This solution does not work with Github Codespaces and any setup where docker is not installed on the local computer!

Use a ports: mapping in the docker-compose.yml file to make the required ports available on the host system e.g.:

db:
      ports:
         - "5432:5432"

This does work with Docker Desktop (MacOS, Windows, Linux) where VS Code and Docker run on the same computer but will fail on setups where docker runs on a different host or in the cloud (vscode.dev). Mapping is not visible in the VS Code ports tab.

(C) IP sharing

WARNING: The IP sharing setup cannot handle container which expose their service on the same port. This needs a fallback to docker compose

Use the IP address of one container db for different containers with the very bad documented option network_mode: service:[service name], which needs to be added to all containers network_mode: service:db except for the service db:

services:
   service_a:
      build:
         context: ../service_a/
         dockerfile: Dockerfile
      volumes:
         - ../service_a:/workspace:cached
      network_mode: service:db
   service_b:
      build:
         context: ../service_b/
         dockerfile: Dockerfile
      volumes:
         - ../service_b:/workspace:cached
      working_dir: /workspace
      network_mode: service:db
   db:
      image: postgres:latest
      restart: unless-stopped
      volumes:
         - postgres-data:/var/lib/postgresql/data
      environment:
         POSTGRES_PASSWORD: postgres
         POSTGRES_USER: postgres
         POSTGRES_DB: postgres
volumes:
   postgres-data:

to make the port always forwarded (optional) add this to your devcontainer.json:

"forwardPorts": [3001,3002,5432],

Tipp: you can specify the exact attributes of a port and add a description with the parameter portsAttributes.

"portsAttributes": {
   "3001": {"label": "Service A (Main)"},
   "3002": {"label": "Service B"},
   "5432": {"label": "Database Postgress"}
},

Diagram of the port mapping:

Github repository (branch c-ip-sharing-example) with example Code & Setup

VS Code with all fowarded ports, database access and previews of both services:

(D) Other options

1. TCP Port forwarding — e.g. adding the following script in devcontainer.json to postCreate:

socat TCP4-LISTEN:<PORT-TO-FORWARD>,reuseaddr,fork TCP:<SERVICE-NAME>:<PORT-TO-FORWARD> & 
socat TCP4-LISTEN:3306,reuseaddr,fork TCP:cs-mysql:3306 &

1. start a container with a http proxy e.g. Tinyproxy and wire up all existing secondary container. This will allow to simulate a production environment with all related production domains.
2. use a revers proxy as ingress — e.g. Traefik

Originally published at https://www.heissenberger.at.