Image may be NSFW.
Clik here to view.
I got a chance to revisit my docker swarm deployment this week after a bit of a break. I was a little curious about my setup because when I spoke to some of our ‘Project Hatchway‘ engineers, I was told that I should be able to launch a single instance of Nginx in Docker Swarm (“docker service create –replicas 1 -p 8080:80 –name web nginx”) and I should be able to access the web service using the following command from any swarm node – “curl 127.0.0.1:8080”. This was not what I was seeing. When I launched the Nginx service, the curl command was successful on the container host where the service was running, but on every other host/node in the swarm cluster, I got a “Failed connect/connection refused”. So why wasn’t it working?
Eventually I traced it to yet another firewall issue on the container hosts/swarm nodes (using Centos 7). It seems that the overlay network needed some ports opened to work as well. These are the ports that I figured out needed to be opened on the firewall of my swarm nodes:
- 7946/tcp – port for “control plane” discovery communication
- 7946/udp – port for “control plane” discovery communication
- 4789/udp – port for “data plane” overlay network traffic
I used the following command on Centos 7 to modify the firewall:
[root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=7946/tcp --permanent [root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=7946/udp --permanent [root@centos-swarm-master ~]# firewall-cmd --zone=public --add-port=4789/udp --permanent [root@centos-swarm-master ~]# firewall-cmd --reload
To verify that the changes took place, I used the following command:
[root@centos-swarm-master ~]# firewall-cmd --list-all public (active) target: default icmp-block-inversion: no interfaces: ens192 sources: services: dhcpv6-client ssh ports: 2379/tcp 4789/udp 2377/tcp 7946/udp 7946/tcp 2380/tcp protocols: masquerade: no forward-ports: sourceports: icmp-blocks: rich rules:
The other ports related to Swarm, which is discussed here, and ETCD, which is for vFile (which I haven’t yet blogged about – watch this space). With these ports opened, we have allowed our docker overlay network to communicate between Swarm nodes. Now if I launch a single replica for the Nginx web service and retry the curl test on different nodes, lets see what happens:
[root@centos-swarm-master ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS rxspku5i98cc vFileServerSharedVol replicated 1/1 luomiao/samba-debian *:30000->445/tcp [root@centos-swarm-master ~]# docker service create --replicas 1 -p 8080:80 --name web nginx xvtzr79sb0fdut85yssxd7z1n overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [root@centos-swarm-master ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS rxspku5i98cc vFileServerSharedVol replicated 1/1 luomiao/samba-debian *:30000->445/tcp xvtzr79sb0fd web replicated 1/1 nginx:latest *:8080->80/tcp [root@centos-swarm-master ~]# curl 127.0.0.1:8080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> [root@centos-swarm-master ~]#
Let’s switch to a worker node, and retry the same test.
[root@centos-swarm-w1 ~]# curl 127.0.0.1:8080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> [root@centos-swarm-w1 ~]#
Success! Now that my overlay network is working successfully, I can reach a single instance of a service working on docker swarm from any of the nodes in the cluster.
The post Validating overlay network when docker swarm running on Centos VMs on vSphere appeared first on CormacHogan.com.