Bug: ExpressVPN refuses to connect to Montreal

[JanitorialMess]

Is this urgent?

None

Host OS

Debian Bullseye

CPU arch

x86_64

VPN service provider

ExpressVPN

What are you using to run the container

docker run

What is the version of Gluetun

Running version v3.40.0 built on 2024-12-25T22:01:25.675Z (commit e890c50)

What's the problem 🤔

If I set SERVER_CITIES to Montreal it refuses to connect and keeping trying without showing any errors. If I don't set the city, and just set Canada as a country, gluetun will always connect to Toronto (the only other available city). I checked the available servers list and I can ping the hostname for Montreal. I also noticed that the same hostname is specified twice. Montreal only has one entry on the desktop client.
I tested the images tagged as latest, v3 and even v3.35 from a year ago and none of them worked.

| Canada | Montreal | `canada-montreal-ca-version-2.expressnetw.com` | ❌ | ✅ |
| Canada | Montreal | `canada-montreal-ca-version-2.expressnetw.com` | ❌ | ✅ |

Changing the log level to debug does not give any useful additional information. Running in privileged mode did not help either.

Share your logs (at least 10 lines)

2025-02-21T21:27:12Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.10 and family v4
2025-02-21T21:27:12Z INFO [routing] local ethernet link found: eth0
2025-02-21T21:27:12Z INFO [routing] local ipnet found: 172.17.0.0/16
2025-02-21T21:27:12Z INFO [firewall] enabling...
2025-02-21T21:27:12Z INFO [firewall] enabled successfully
2025-02-21T21:27:12Z INFO [storage] creating /gluetun/servers.json with 20776 hardcoded servers
2025-02-21T21:27:12Z INFO Alpine version: 3.20.3
2025-02-21T21:27:12Z INFO OpenVPN 2.5 version: 2.5.10
2025-02-21T21:27:12Z INFO OpenVPN 2.6 version: 2.6.11
2025-02-21T21:27:12Z INFO IPtables version: v1.8.10
2025-02-21T21:27:12Z INFO Settings summary:
├── VPN settings:
|   ├── VPN provider settings:
|   |   ├── Name: expressvpn
|   |   └── Server selection settings:
|   |       ├── VPN type: openvpn
|   |       ├── Countries: canada
|   |       ├── Cities: montreal
|   |       └── OpenVPN server selection settings:
|   |           └── Protocol: UDP
|   └── OpenVPN settings:
|       ├── OpenVPN version: 2.6
|       ├── User: [set]
|       ├── Password: [set]
|       ├── Network interface: tun0
|       ├── Run OpenVPN as: root
|       └── Verbosity level: 1
├── DNS settings:
|   ├── Keep existing nameserver(s): no
|   ├── DNS server address to use: 127.0.0.1
|   └── DNS over TLS settings:
|       ├── Enabled: yes
|       ├── Update period: every 24h0m0s
|       ├── Upstream resolvers:
|       |   └── cloudflare
|       ├── Caching: yes
|       ├── IPv6: no
|       └── DNS filtering settings:
|           ├── Block malicious: yes
|           ├── Block ads: no
|           ├── Block surveillance: no
|           └── Blocked IP networks:
|               ├── 127.0.0.1/8
|               ├── 10.0.0.0/8
|               ├── 172.16.0.0/12
|               ├── 192.168.0.0/16
|               ├── 169.254.0.0/16
|               ├── ::1/128
|               ├── fc00::/7
|               ├── fe80::/10
|               ├── ::ffff:127.0.0.1/104
|               ├── ::ffff:10.0.0.0/104
|               ├── ::ffff:169.254.0.0/112
|               ├── ::ffff:172.16.0.0/108
|               └── ::ffff:192.168.0.0/112
├── Firewall settings:
|   └── Enabled: yes
├── Log settings:
|   └── Log level: info
├── Health settings:
|   ├── Server listening address: 127.0.0.1:9999
|   ├── Target address: cloudflare.com:443
|   ├── Duration to wait after success: 5s
|   ├── Read header timeout: 100ms
|   ├── Read timeout: 500ms
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   └── Enabled: no
├── HTTP proxy settings:
|   └── Enabled: no
├── Control server settings:
|   ├── Listening address: :8000
|   ├── Logging: yes
|   └── Authentication file path: /gluetun/auth/config.toml
├── Storage settings:
|   └── Filepath: /gluetun/servers.json
├── OS Alpine settings:
|   ├── Process UID: 1000
|   └── Process GID: 1000
├── Public IP settings:
|   ├── IP file path: /tmp/gluetun/ip
|   ├── Public IP data base API: ipinfo
|   └── Public IP data backup APIs:
|       ├── ifconfigco
|       ├── ip2location
|       └── cloudflare
└── Version settings:
    └── Enabled: yes
2025-02-21T21:27:12Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.10 and family v4
2025-02-21T21:27:12Z INFO [routing] adding route for 0.0.0.0/0
2025-02-21T21:27:12Z INFO [firewall] setting allowed subnets...
2025-02-21T21:27:12Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.10 and family v4
2025-02-21T21:27:12Z INFO [dns] using plaintext DNS at address 1.1.1.1
2025-02-21T21:27:12Z INFO [http server] http server listening on [::]:8000
2025-02-21T21:27:12Z INFO [healthcheck] listening on 127.0.0.1:9999
2025-02-21T21:27:12Z INFO [firewall] allowing VPN connection...
2025-02-21T21:27:12Z INFO [openvpn] OpenVPN 2.6.11 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2025-02-21T21:27:12Z INFO [openvpn] library versions: OpenSSL 3.3.2 3 Sep 2024, LZO 2.10
2025-02-21T21:27:12Z INFO [openvpn] TCP/UDP: Preserving recently used remote address: [AF_INET]107.152.206.69:1195
2025-02-21T21:27:12Z INFO [openvpn] UDPv4 link local: (not bound)
2025-02-21T21:27:12Z INFO [openvpn] UDPv4 link remote: [AF_INET]107.152.206.69:1195
2025-02-21T21:27:18Z INFO [healthcheck] program has been unhealthy for 6s: restarting VPN (healthcheck error: dialing: dial tcp4: lookup cloudflare.com on 1.1.1.1:53: write udp 172.17.0.10:42230->1.1.1.1:53: write: operation not permitted)
2025-02-21T21:27:18Z INFO [healthcheck] 👉 See https://github.com/qdm12/gluetun-wiki/blob/main/faq/healthcheck.md
2025-02-21T21:27:18Z INFO [healthcheck] DO NOT OPEN AN ISSUE UNLESS YOU READ AND TRIED EACH POSSIBLE SOLUTION
2025-02-21T21:27:18Z INFO [vpn] stopping
2025-02-21T21:27:18Z INFO [vpn] starting
2025-02-21T21:27:18Z INFO [firewall] allowing VPN connection...
2025-02-21T21:27:18Z INFO [openvpn] OpenVPN 2.6.11 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2025-02-21T21:27:18Z INFO [openvpn] library versions: OpenSSL 3.3.2 3 Sep 2024, LZO 2.10
2025-02-21T21:27:18Z INFO [openvpn] TCP/UDP: Preserving recently used remote address: [AF_INET]144.168.163.5:1195
2025-02-21T21:27:18Z INFO [openvpn] UDPv4 link local: (not bound)
2025-02-21T21:27:18Z INFO [openvpn] UDPv4 link remote: [AF_INET]144.168.163.5:1195
2025-02-21T21:27:29Z INFO [healthcheck] program has been unhealthy for 11s: restarting VPN (healthcheck error: dialing: dial tcp4: lookup cloudflare.com on 1.1.1.1:53: write udp 172.17.0.10:57377->1.1.1.1:53: write: operation not permitted)
2025-02-21T21:27:29Z INFO [healthcheck] 👉 See https://github.com/qdm12/gluetun-wiki/blob/main/faq/healthcheck.md
2025-02-21T21:27:29Z INFO [healthcheck] DO NOT OPEN AN ISSUE UNLESS YOU READ AND TRIED EACH POSSIBLE SOLUTION
2025-02-21T21:27:29Z INFO [vpn] stopping

Share your configuration

sudo docker run -it --rm --cap-add=NET_ADMIN \
    -e VPN_SERVICE_PROVIDER=expressvpn \
    -e OPENVPN_USER=<user> \
    -e OPENVPN_PASSWORD=<pass> \
    -e SERVER_COUNTRIES=Canada \
    -e SERVER_CITIES=Montreal \
    --device /dev/net/tun:/dev/net/tun \
    qmcgaw/gluetun:v3

[github-actions]

@qdm12 is more or less the only maintainer of this project and works on it in his free time.
Please:

• do not ask for updates, be patient
• 👍 the issue to show your support instead of commenting
  @qdm12 usually checks issues at least once a week, if this is a new urgent bug,
  revert to an older tagged container image

[JanitorialMess]

Ok so I think I know where the issue is. The OVPN config created by gluetun for Montreal leads to this

2025-02-24T06:25:01Z INFO [openvpn] Connection profiles [0]:
2025-02-24T06:25:01Z INFO [openvpn]   proto = udp
2025-02-24T06:25:01Z INFO [openvpn]   local = '[UNDEF]'
2025-02-24T06:25:01Z INFO [openvpn]   local_port = '[UNDEF]'
2025-02-24T06:25:01Z INFO [openvpn]   remote = '144.168.163.5'
2025-02-24T06:25:01Z INFO [openvpn]   remote_port = '1195'

The IP listed here has nothing to do with the hardcoded hostname for Montreal which when resolved shows this as of today

; <<>> DiG 9.16.50-Debian <<>> canada-montreal-ca-version-2.expressnetw.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58675
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;canada-montreal-ca-version-2.expressnetw.com. IN A

;; ANSWER SECTION:
canada-montreal-ca-version-2.expressnetw.com. 300 IN A 45.91.23.231
canada-montreal-ca-version-2.expressnetw.com. 300 IN A 45.91.23.111
canada-montreal-ca-version-2.expressnetw.com. 300 IN A 45.91.23.176

I tested Toronto too and the IP in the OVPN cfg matches one of ones returned by dig.
I do not have any custom DNS resolver set up and I have tested and replicated the issue on 2 machines in different countries with different operating systems. Creating my own OVPN config from scratch following the template used by gluetun and just filling in the correct IP instead resolves the issue.
I also noticed when passing in a custom config with those IP to gluetun, it uses ipinfo.io which resolves all of them to the wrong region. ipinfo.io returns Toronto as the region for those addresses whereas ip.me correctly shows Montreal.